A nail-hole assembly method for complex scenes based on obstacle avoidance and diffusion guidance

Abstract

The present invention discloses a nail hole assembly method for complex scenes based on obstacle avoidance diffusion guidance, which is specifically divided into two stages: in the search stage, the system uses a point cloud as input and collision loss as a constraint condition, and generates a trajectory that avoids obstacles near the hole and reaches the target hole position through a diffusion model; in the alignment and insertion stage, the point cloud is used as input, and the visual servo network outputs the increments of the terminal joint coordinates and posture changes, and continuously iterates the translation increment and rotation increment through closed-loop control to optimize the terminal insertion posture. Compared with existing solutions, the present invention can adapt to complex assembly scenarios where the target hole position is located on a non-flat, non-horizontal surface, and supports the realization of obstacle avoidance and assembly tasks under the premise of considering spatial constraints. Based on the encoding of the three-dimensional spatial relationship with the target hole position, combined with the constraint condition of collision loss, the present invention significantly improves the accuracy and efficiency of nail hole assembly, and provides an effective solution for automated assembly in complex scenes.

Description

Technical Field

[0001] The present invention relates to the field of robot imitation learning assembly, and in particular to a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance. Background Art

[0002] As a core component of industrial applications, robotic assembly has long been a hot research area, among which nail-hole assembly is considered to be the basis for the assembly of various components. Robotic nail-hole assembly has broad application prospects in large-scale object assembly as well as small component assembly. Existing nail-hole assembly methods are usually targeted at relatively simple scenarios, such as when the target hole is located on a flat horizontal surface. In such scenarios, it is only necessary to obtain the two-dimensional coordinates of the hole, and then the joint angles of the robot arm can be obtained through inverse kinematics to complete the assembly. However, in actual assembly tasks, the position of the hole is often more complex. For example, the hole may be located on a non-horizontal surface of other workpieces, or even on the inner wall of other workpieces. In this case, it is necessary to plan the motion trajectory of the robot arm, comprehensively considering the complex constraints in space, to ensure that the robot arm can avoid obstacles and complete the nail hole insertion operation.

[0003] Existing studies usually reduce complexity by simplifying the problem, such as imposing strict constraints on the distance between the target and the end effector. However, under these simplified conditions, these methods often have difficulty working when there are more complex posture differences between the target and the end effector, such as significant rotational deviations or large initial alignment errors. In particular, for rotational deviations, even slight tilt angles can cause problems, because many methods usually assume that the insertion direction is always aligned with the z-axis of the hole during training. In addition, methods based on torque feedback control have limitations due to the need for physical contact, while methods based on visual servoing are susceptible to error accumulation. Therefore, existing methods find it difficult to effectively deal with large initial alignment errors, and assembly failure often occurs when the initial distance between the end effector and the target position is far.

[0004] In summary, existing nail-hole assembly methods have significant shortcomings in dealing with pose variations, avoiding obstacles in complex scenes, and large initial alignment errors. Therefore, designing a flexible, precise, and robust assembly strategy to overcome the limitations of existing methods and improve assembly accuracy and reliability has become a key issue that needs to be addressed. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned existing nail hole assembly methods in dealing with posture differences, complex scene obstacle avoidance and large initial alignment errors, and to provide a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A complex scene nail hole assembly method based on obstacle avoidance and diffusion guidance, comprising the following steps:

[0008] The point cloud data collected by the depth camera is used as the input of the diffusion-based trajectory generation model. With the collision loss as the constraint condition, the cross-attention mechanism is used to conditionally control the trajectory generation of the diffusion-based trajectory generation model based on the point cloud data, generating an end-effector trajectory that avoids obstacles and reaches the target hole.

[0009] The global features of point cloud data are extracted through a visual servo network, and the global features are input into a multi-layer perceptron network to predict the increments of the end-effector joint coordinates and posture. A closed-loop control method is then used to iteratively optimize the translation offset and rotation offset to realize the insertion operation of the end-effector.

[0010] As a preferred technical solution, the diffusion-based trajectory generation model includes two processes: forward diffusion and reverse generation:

[0011] The diffusion-based trajectory generation model takes as input the noisy data obtained by adding noise to the given expert trajectory and the current point cloud data;

[0012] In the forward diffusion process, the noisy data is used as the input of the Unet network, and the Unet network outputs the predicted noise; the Unet network model is optimized by calculating the mean square error and collision loss between the predicted noise and the actual added noise to obtain a trained Unet network;

[0013] In the reverse generation process, the Unet network trained in the forward diffusion is used to denoise the noisy data using a denoising diffusion implicit model, and the cross-attention mechanism is used to guide the denoising process of the Unet network based on the current point cloud data to generate an obstacle avoidance trajectory for the end effector pointing to the vicinity of the hole position.

[0014] As a preferred technical solution, the collision loss is specifically as follows:

[0015]

[0016] in, represents the end coordinate of the i-th trajectory at time t; D j The (·) function represents the signed distance function. For a given closed surface, the signed distance function maps a point in Euclidean space to the minimum distance between the point and the surface; if the point is inside the surface, the signed distance function returns a negative value.

[0017] As a preferred technical solution, the cross-attention mechanism is used to conditionally control the trajectory generation of the diffusion model based on point cloud data, as follows:

[0018] Features are extracted from point cloud data. A query vector is generated based on the current state of the diffusion model, and a key vector and a value vector are generated based on the point cloud features. The similarity between the query vector and the key vector is converted into a weight coefficient through an activation function, and then multiplied by the value vector to obtain a feature representation of the fused environmental information.

[0019] As a preferred technical solution, during the training process of the diffusion-based trajectory generation model, the colors of the pins and holes, the spatial positions of the holes, the obstacles near the holes, and the lighting conditions in the simulation environment are dynamically adjusted to simulate diverse scene changes.

[0020] As a preferred technical solution, the diffusion-based trajectory generation model in the inference stage:

[0021] First, the randomly sampled Gaussian noise is denoised through DDIM; then the current point cloud data is input and the cross-attention mechanism is used to guide the denoising process, thereby generating an obstacle avoidance trajectory pointing to the vicinity of the hole.

[0022] As a preferred technical solution, the process of implementing the insertion operation of the end effector through the visual servo network is as follows:

[0023] Crop the input point cloud data to focus on the pins and holes;

[0024] Extract global features from the cropped point cloud data;

[0025] Input the global features into the multi-layer perceptron to predict the translation offset Δt and rotation offset Δr;

[0026] After the offset prediction is completed, visual servoing is used to iteratively estimate the translation offset Δt and rotation offset Δr, and the posture of the end effector is gradually adjusted to realize the insertion operation of the end effector.

[0027] As a preferred technical solution, the visual servoing optimizes the posture of the end effector, and the specific steps are as follows:

[0028] Record the initial posture of the end effector [R|t], where R is the rotation matrix and t is the translation vector;

[0029] Convert the rotation offset Δr into the form of a rotation matrix ΔR;

[0030] Move the end effector to the next pose [R′|t′], where R′ = ΔR·R and t′ = Δt+t;

[0031] By repeatedly performing the iterative operation until the predicted offset is less than the error tolerance or the preset maximum number of iterations is reached, the insertion instruction is finally executed to complete the task.

[0032] As a preferred technical solution, the loss L of the visual servo insertion fine It consists of two parts:

[0033] L fine =L trans +L rot

[0034] Among them, L trans and L rot Represent the loss of translation offset and rotation offset respectively;

[0035] For the translation offset loss L trans , using the root mean square error to learn the magnitude of the translation offset and the cosine distance to learn the direction of motion:

[0036]

[0037] Where Δt * is the true value of the translation offset; Δt is the predicted translation offset;

[0038] For the rotation offset loss L rot , using the root mean square error to minimize the rotation error:

[0039]

[0040] in, is the true value of the rotation offset, and Δr is the predicted rotation offset.

[0041] As a preferred technical solution, in each iterative optimization, the posture of the end effector is initialized to the true posture of the target hole;

[0042] Randomly generate a negative value of the true translation offset -Δt and a negative value of the true rotation offset -Δθ_r, move the end effector to a new position according to the negative value of the true translation offset -Δt and the negative value of the true rotation offset -Δθ_r, and record the translation offset Δt and the rotation offset Δθ_r;

[0043] The robot is trained using behavioral cloning. During testing, the robot uses visual feedback to iteratively estimate and correct the translation offset Δt and rotation offset Δθ_r of the end effector. Visual servoing gradually optimizes the insertion posture of the end effector through a closed-loop control mechanism until the translation offset and rotation offset fall within the preset threshold or the specified maximum number of iterations is reached.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1) The present method, based on a diffusion-guided search method with obstacle avoidance, effectively addresses the robot's large initial posture error and potential obstacles encountered during movement. By generating an obstacle avoidance trajectory, the end effector is brought close to the target hole position, providing precise initial conditions for subsequent operations. Simultaneously, a 3D point cloud-based visual servo insertion strategy iteratively optimizes the end effector's translational and rotational increments through a closed-loop control strategy, achieving precise insertion in 6 degrees of freedom.

[0046] 2) Unlike traditional methods that rely directly on force sensors for physical contact with the target object, this method utilizes only visual sensors, significantly improving insertion efficiency. Furthermore, compared to other visual sensor-based methods, this method fully utilizes 3D point cloud information, learns the three-dimensional spatial relationship between nail holes, and incorporates collision loss constraints to significantly improve the completion and accuracy of 6-DOF nail-hole assembly tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of the nail hole assembly method for complex scenes based on obstacle avoidance and diffusion guidance of the present invention.

[0048] Figure 2 It is a schematic diagram of the diffusion model training of the complex scene nail hole assembly method based on obstacle avoidance diffusion guidance in the method of the present invention.

[0049] Figure 3 This is a visual servoing flow chart of the complex scene nail hole assembly method based on obstacle avoidance diffusion guidance in the method of the present invention. DETAILED DESCRIPTION

[0050] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0051] Example 1

[0052] In response to the shortcomings of the existing technology, the present invention aims to solve the problems of posture differences, complex scene obstacle avoidance, and large initial alignment errors during the 6-DOF nail hole assembly process, and proposes a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance. The method is divided into two stages: in the hole search stage, a diffusion model is used to take the 3D point cloud as input and the collision loss as a constraint to generate a rough trajectory to guide the end effector to avoid obstacles and approach the hole position; in the insertion stage, the 3D point cloud features are extracted through the visual servo network using Pointnet++, and the features are input into the multi-layer perceptron (MLP) network to predict the increments of the end effector joint coordinates and posture. The closed-loop control method is combined to iteratively optimize the translation offset and rotation offset, thereby achieving precise insertion of the end effector. Through this coarse-to-fine control strategy, the robot can complete the 6-DOF insertion task.

[0053] To achieve the above purpose, the present invention provides a technical solution: a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance, such as Figure 1 The method steps shown include:

[0054] S1. Search based on obstacle avoidance diffusion guidance: Generate the trajectory of the end effector so that it can approach the target hole while avoiding obstacles, thereby effectively solving the problem of large posture error in the robot's initial state and ensuring that it can flexibly avoid obstacles during movement.

[0055] S2. Visual servo insertion based on 3D point cloud: The point cloud data collected by the depth camera is converted into translation increments and rotation increments of the end effector, and the insertion pose is iteratively optimized through closed-loop control to ensure high precision and high robustness of the insertion process.

[0056] Specifically, in step S1, a diffusion-based deep generation model is constructed for obstacle avoidance and rough hole search. Consider the following Gaussian diffusion model, which includes two processes: forward diffusion and reverse generation.

[0057] The forward diffusion process describes the process of adding noise to the given initial expert trajectory data:

[0058]

[0059] Among them, x0 represents the original data of the expert trajectory, x t represents the noisy data, represents a constant that changes with t, ε t Normally distributed noise is added.

[0060] The noisy data is used as the input of the Unet network, and Unet outputs the predicted noise. The mean square error (MSELoss) between the predicted noise and the added noise is calculated as the noise loss; in order to avoid collisions, a collision loss is also added:

[0061]

[0062] in, represents the end coordinate of the i-th trajectory at time t, D j The (·) function represents the signed distance function (SDF). For a given closed surface, its signed distance function SDF maps a point in Euclidean space to the minimum distance between the point and the surface; if the point is inside the surface, the function returns a negative value.

[0063] During the reverse generation process, the UNet network trained in the forward propagation process is used to denoise the noisy data, gradually restoring the data to a distribution as close to the original data as possible. To ensure that the gradients in the reverse generation process can be effectively backpropagated, a deterministic denoising diffusion implicit model (DDIM) is adopted. This approach accelerates the generation process.

[0064] The training process of the diffusion model is as follows Figure 2 As shown in the figure, the point cloud data is first extracted using the PointNet++ network. Then, the diffusion model's trajectory generation is conditionally controlled using a cross-attention mechanism, allowing the generation process to incorporate the features of the current point cloud data. During training, the mean squared error (MSELoss) and collision loss are calculated between the predicted noise and the actual added noise to optimize the model and ultimately train our UNet network.

[0065] In the inference stage of the diffusion model, the randomly sampled Gaussian noise is first denoised through DDIM. Then the current point cloud data is input and the cross-attention mechanism is used to guide the denoising process, thereby generating an obstacle avoidance trajectory pointing to the vicinity of the hole.

[0066] Furthermore, the search based on obstacle avoidance diffusion guidance, the training data comes from the CoppeliaSim simulator, the training data X i Represents point cloud data, Traj irepresents the trajectory, and M represents the number of data points, with each trajectory consisting of 16 steps. During data generation, the model dynamically adjusts the color of the pins and holes, the spatial position of the holes, nearby obstacles, and lighting conditions in the simulation environment to simulate diverse scene changes, thereby improving the model's adaptability to different environments. As training progresses, the model achieves robust control, generating obstacle avoidance trajectories close to the target hole location based on the currently acquired point cloud. This eliminates the need for strict initial constraints, providing reliable initial conditions for the subsequent precise insertion stage.

[0067] In step S2, the visual servo insertion based on the 3D point cloud is used to adjust the robot end position and realize the 6-DOF jacking operation. By estimating the offset between the current position and the target position, the posture of the end effector is optimized to achieve accurate insertion operation. The visual servo process is as follows: Figure 3 First, input point cloud data Crop to where N ′ ∈[0,N] represents the number of point cloud points close to the end effector. By cropping the point cloud data, we can focus more on the local information of the pins and holes, ignore other redundant information, and improve the effectiveness of feature extraction. Based on the cropped point cloud, the module uses the Pointnet++ network to extract global features and inputs them into a multi-layer perceptron (MLP) to predict the translation offset Δt = (Δx, Δy, Δz) and the rotation offset Δr = (Δθ x ,Δθ y ,Δθ z ), where the rotation offset is expressed in Euler angles. After the prediction, the module uses visual servoing (a closed-loop control method) to optimize the posture of the end effector, thereby gradually achieving accurate insertion. Specifically, the posture of the end effector is gradually adjusted by iteratively estimating the translation offset Δt and the rotation offset Δr. First, the initial posture of the end effector is recorded Where R is the rotation matrix and t is the translation vector. The rotation offset Δr is converted into the form of a rotation matrix. We then move the end effector to the next pose Where R′=ΔR·R, t′=Δt+t, and the iterative operation is repeated until the predicted offset is less than the error tolerance or the preset maximum number of iterations is reached, and finally the insertion instruction is executed to complete the task.

[0068] The loss L of the visual servo insertion module based on 3D point cloud fine It consists of two parts:

[0069] L fine =L trans +L rot

[0070] Among them, L trans and L rot Represent the loss of translation offset and rotation offset respectively. For translation offset, the model uses both root mean square error and cosine distance to optimize the translation error:

[0071]

[0072] Where Δt * =(Δx * ,Δy * ,Δz * ) is the true value of the translation offset. The model uses the root mean square error to learn the magnitude of the translation offset and uses the cosine distance to learn the direction of motion. For the rotation offset, the root mean square error is used to minimize the rotation error:

[0073]

[0074] in, is the true value of the rotation offset.

[0075] Furthermore, the training data of the 3D point cloud-based visual servo insertion comes from the CoppeliaSim simulator. The dataset is defined as Where L is the total number of data, X i and A i is a point cloud and action in a scene. The dataset is iterated L times by reverse sampling. In each iteration, the posture of the end effector is first initialized to the true posture of the target hole to ensure that the optimization starts from the standard position. Then, -Δt and -Δθ are randomly generated. r , which are the negative values ​​of the real translation offset and rotation offset respectively. Then, according to -Δt and -Δθ r Move the end effector to a new position and record the translation offset Δt and rotation offset Δθ r As the refined data. After obtaining the data, the model is trained using behavioral cloning. During testing, the robot uses visual feedback to calculate the translation offset Δt and rotation offset Δθ of the end effector. r Perform iterative estimation and correction. Through a closed-loop control mechanism, the insertion pose of the end effector is gradually optimized until the translational and rotational offsets fall within the preset thresholds or the specified maximum number of iterations is reached.

[0076] This paper utilizes the PointNet++ network to extract point cloud features. By introducing a hierarchical feature learning mechanism and a multi-scale grouping strategy, the PointNet++ network effectively improves its understanding of 3D point clouds and its feature extraction capabilities. Its core concept is to construct a multi-layer feature extraction network within the point cloud. Each layer consists of three main modules: sampling, grouping, and feature extraction. First, a representative center point is selected from the input point cloud using the farthest point sampling algorithm. Then, for each center point, neighboring points are searched within a certain radius to form a local point set. Next, feature extraction is performed on each local point set using PointNet or its variants to generate a local feature description. Through its hierarchical structure, the network aggregates point cloud features layer by layer, thereby simultaneously capturing microscopic details and macroscopic global structure. Multi-scale grouping strategies (such as multi-scale grouping and multi-resolution grouping) further enhance the network's robustness to point cloud geometry, ensuring effective feature capture at different scales. Ultimately, through feature aggregation, PointNet++ maps multi-level features to an output space, which can be used for a variety of 3D vision tasks such as classification and segmentation. In the application of the present invention, the final output space part of the PointNet++ network is removed, and the aggregated features are directly extracted for model design, thereby meeting the task requirements of nail-hole assembly.

[0077] In this invention, we introduce extracted 3D point cloud features into a diffusion model via a cross-attention mechanism to train robot trajectory generation. The cross-attention mechanism is an information fusion method in deep learning that enables the diffusion model to efficiently focus on 3D point cloud features, thereby generating a robot motion path that conforms to environmental constraints. Specifically, the cross-attention mechanism dynamically adjusts the model's attention to different features by calculating the similarity between the query, key, and value. In this process, the current state of the diffusion model generates a query vector, and the point cloud features generate key and value vectors. The similarity between the query and key is converted into a weight coefficient through a softmax function and then multiplied by the value vector to obtain a feature representation that incorporates environmental information. This mechanism enables the diffusion model to dynamically extract the most relevant information from the point cloud features when generating trajectories based on the nail hole information in the environment, thereby achieving perception and adaptation to complex 3D environments. By organically combining 3D point cloud features with the diffusion model, the cross-attention mechanism plays a key bridging role in this invention, enabling the trajectory generated by the diffusion model to reach the vicinity of the target hole location.

[0078] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A complex scene nail hole assembly method based on obstacle avoidance diffusion guidance, characterized in that the steps include: The point cloud data collected by the depth camera is used as the input of the diffusion-based trajectory generation model. With the collision loss as the constraint condition, the cross-attention mechanism is used to conditionally control the trajectory generation of the diffusion-based trajectory generation model based on the point cloud data, generating an end-effector trajectory that avoids obstacles and reaches the target hole. The diffusion-based trajectory generation model includes two processes: forward diffusion and reverse generation: The diffusion-based trajectory generation model takes as input the noisy data obtained by adding noise to the given expert trajectory and the current point cloud data; In the forward diffusion process, the noisy data is used as the input of the Unet network, and the Unet network outputs the predicted noise; the Unet network model is optimized by calculating the mean square error and collision loss between the predicted noise and the actual added noise to obtain a trained Unet network; In the reverse generation process, the Unet network trained in the forward diffusion is used to denoise the noisy data using a denoising diffusion implicit model, and the cross-attention mechanism is used to guide the denoising process of the Unet network based on the current point cloud data to generate an obstacle avoidance trajectory for the end effector pointing to the vicinity of the hole position; The global features of point cloud data are extracted through a visual servo network, and the global features are input into a multi-layer perceptron network to predict the increments of the end-effector joint coordinates and posture. A closed-loop control method is then used to iteratively optimize the translation offset and rotation offset to realize the insertion operation of the end-effector.

2. The complex scene nail hole assembly method based on obstacle avoidance and diffusion guidance according to claim 1 is characterized in that: The collision losses are as follows: in, Representative The track in The end coordinates of the moment; The function represents the signed distance function. For a given closed surface, the signed distance function maps a point in Euclidean space to the minimum distance between the point and the surface; if the point is inside the surface, the signed distance function returns a negative value.

3. The complex scene nail hole assembly method based on obstacle avoidance and diffusion guidance according to claim 1 is characterized in that: The cross-attention mechanism is used to conditionally control the trajectory generation of the diffusion model based on point cloud data, as follows: Extract features from point cloud data; The query vector is generated based on the current state of the diffusion model, and the key vector and value vector are generated based on the point cloud features. The similarity between the query vector and the key vector is converted into a weight coefficient through an activation function, and then multiplied by the value vector to obtain a feature representation of the fused environmental information.

4. The complex scene nail hole assembly method based on obstacle avoidance and diffusion guidance according to claim 1 is characterized in that: During the training process of the diffusion-based trajectory generation model, the colors of the pins and holes, the spatial positions of the holes, the obstacles near the holes, and the lighting conditions in the simulation environment are dynamically adjusted to simulate diverse scene changes.

5. The complex scene nail hole assembly method based on obstacle avoidance and diffusion guidance according to claim 1 is characterized in that: The diffusion-based trajectory generation model in the inference phase: First, the randomly sampled Gaussian noise is denoised through DDIM; then the current point cloud data is input and the cross-attention mechanism is used to guide the denoising process, thereby generating an obstacle avoidance trajectory pointing to the vicinity of the hole.

6. The complex scene nail hole assembly method based on obstacle avoidance and diffusion guidance according to claim 1 is characterized in that: The process of implementing the insertion operation of the end effector through the visual servo network is as follows: Crop the input point cloud data to focus on the pins and holes; Extract global features from the cropped point cloud data; Input the global features into the multi-layer perceptron to predict the translation offset and rotation offset ; After the offset prediction is completed, the translation offset is estimated iteratively using visual servoing and rotation offset , and gradually adjust the posture of the end effector to realize the insertion operation of the end effector.

7. The complex scene nail hole assembly method based on obstacle avoidance and diffusion guidance according to claim 6 is characterized in that: The visual servo optimization of the end effector posture is carried out in the following specific steps: Record the initial posture of the end effector ,in is the rotation matrix, is the translation vector; Rotate the offset Converted to the form of rotation matrix ; Move the end effector to the next pose ,in , ; By repeatedly performing the iterative operation until the predicted offset is less than the error tolerance or the preset maximum number of iterations is reached, the insertion instruction is finally executed to complete the task.

8. The complex scene nail hole assembly method based on obstacle avoidance and diffusion guidance according to claim 1 is characterized in that: The visual servoing inserts a loss Consists of two parts composition: in, and Represent the loss of translation offset and rotation offset respectively; For translational offset loss , using the root mean square error to learn the magnitude of the translation offset and the cosine distance to learn the direction of motion: in, is the true value of the translation offset; To predict the translation offset; For rotational offset loss , using the root mean square error to minimize the rotation error: in, is the true value of the rotation offset, is the predicted rotation offset.

9. The complex scene nail hole assembly method based on obstacle avoidance and diffusion guidance according to claim 1 is characterized in that: In each iterative optimization, the posture of the end effector is initialized to the true posture of the target hole; Randomly generate a negative value of the true translation offset -Δt and a negative value of the true rotation offset -∆θ_r, move the end effector to a new position according to the negative value of the true translation offset -Δt and the negative value of the true rotation offset -∆θ_r, and record the translation offset Δt and the rotation offset ∆θ_r; The robot is trained using behavioral cloning. During testing, the robot uses visual feedback to iteratively estimate and correct the translation offset Δt and rotation offset ∆θ_r of the end effector. Visual servoing gradually optimizes the insertion pose of the end effector through a closed-loop control mechanism until the translation and rotation offsets fall within the preset threshold or the specified maximum number of iterations is reached.

Images