Complex scene nail hole assembly method based on obstacle avoidance diffusion guidance

Abstract

The invention discloses a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance, which is specifically divided into two stages: in a search stage, a system takes point cloud as input and collision loss as a constraint condition, and generates a track for avoiding obstacles near a hole and reaching a target hole position through a diffusion model; in the alignment insertion stage, the point cloud serves as input, the visual servo network outputs the tail end joint coordinates and the increment of the posture change, and the translation increment and the rotation increment are continuously iterated through closed-loop control so as to optimize the tail end insertion posture. Compared with an existing scheme, the method can adapt to a complex assembling scene that the target hole site is located on a non-flat and non-horizontal plane, obstacle avoidance and assembling tasks are achieved on the premise that the spatial constraint relation is considered, and based on coding of the three-dimensional space relation between the target hole site and the target hole site, the collision loss constraint condition is combined, and the assembling accuracy is improved. According to the method, the precision and efficiency of nail hole assembly are remarkably improved, and an effective solution is provided for automatic assembly in a complex scene.

Description

Technical Field

[0001] The invention relates to the field of robot imitation learning assembly, and in particular to a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance. Background Art

[0002] As a core component of industrial applications, robotic assembly has long been a hot research area, among which nail-hole assembly is considered to be the basis of multiple component assemblies. Robotic nail-hole assembly has broad application prospects in large-scale object assembly and small component assembly. Existing nail-hole assembly methods are usually aimed at simpler scenarios, such as when the target hole is located on a flat horizontal surface. In such scenarios, only the two-dimensional coordinates of the hole need to be obtained, and the joint angles of the robot arm can be obtained through inverse kinematics to complete the assembly. However, in actual assembly tasks, the position and posture of the hole are often more complex. For example, the hole may be located on the non-horizontal surface of other workpieces, or even in the inner wall of other workpieces. In this case, it is necessary to plan the motion trajectory of the robot arm, comprehensively consider the complex constraints in space, and ensure that the robot arm can avoid obstacles and complete the nail hole insertion operation.

[0003] Existing studies usually reduce complexity by simplifying the problem, such as imposing strict constraints on the distance between the target and the end effector. However, under these simplified conditions, these methods often have difficulty working when there are more complex posture differences between the target and the end effector, such as significant rotational deviations or large initial alignment errors. In particular, for rotational deviations, even slight tilt angles can cause problems because many methods usually assume during training that the insertion direction is always aligned with the z-axis of the hole. In addition, methods based on torque feedback control have limitations due to the need for physical contact, while methods based on visual servoing are susceptible to error accumulation. Therefore, existing methods find it difficult to effectively deal with large initial alignment errors, and often lead to assembly failure when the initial distance between the end effector and the target position is far.

[0004] In summary, the existing nail-hole assembly method has significant shortcomings in dealing with posture differences, complex scene obstacle avoidance, and large initial alignment errors. Therefore, how to design a flexible, accurate, and robust assembly strategy to overcome the limitations of existing methods and improve assembly accuracy and reliability has become a key issue that needs to be solved urgently. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned existing nail hole assembly methods in dealing with posture differences, complex scene obstacle avoidance and large initial alignment errors, and to provide a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A complex scene nail hole assembly method based on obstacle avoidance diffusion guidance, the steps comprising:

[0008] The point cloud data collected by the depth camera is used as the input of the diffusion-based trajectory generation model. The collision loss is used as a constraint condition. The cross-attention mechanism is used to conditionally control the trajectory generation of the diffusion-based trajectory generation model based on the point cloud data, and an end-effector trajectory is generated that avoids obstacles and reaches the target hole.

[0009] The global features of point cloud data are extracted through the visual servo network, and the global features are input into the multi-layer perceptron network to predict the increments of the end-effector joint coordinates and posture. The closed-loop control method is used to iteratively optimize the translation offset and rotation offset to realize the insertion operation of the end-effector.

[0010] As a preferred technical solution, the diffusion-based trajectory generation model includes two processes: forward diffusion and reverse generation:

[0011] The diffusion-based trajectory generation model uses the noisy data obtained by adding noise to the previously given expert trajectory and the current point cloud data as input;

[0012] In the forward diffusion process, the noisy data is used as the input of the Unet network, and the Unet network outputs the predicted noise; the Unet network model is optimized by calculating the mean square error and collision loss between the predicted noise and the actual added noise to obtain a trained Unet network;

[0013] In the reverse generation process, the Unet network trained in the forward diffusion is used to denoise the noisy data using a denoising diffusion implicit model, and the cross-attention mechanism is used to guide the denoising process of the Unet network based on the current point cloud data to generate an obstacle avoidance trajectory of the end effector pointing to the vicinity of the hole position.

[0014] As a preferred technical solution, the collision loss is specifically as follows:

[0015]

[0016] in, represents the terminal coordinates of the ith trajectory at time t; D j The (·) function represents the signed distance function. For a given closed surface, the signed distance function maps a point in Euclidean space to the minimum distance from the point to the surface; if the point is inside the surface, the signed distance function returns a negative value.

[0017] As a preferred technical solution, the cross-attention mechanism is used to conditionally control the trajectory generation of the diffusion model based on point cloud data, as follows:

[0018] Features are extracted from point cloud data; a query vector is generated based on the current state of the diffusion model, and a key vector and a value vector are generated based on point cloud features; the similarity between the query vector and the key vector is converted into a weight coefficient through an activation function, and then multiplied with the value vector to obtain a feature representation of the fused environmental information.

[0019] As a preferred technical solution, during the training process of the diffusion-based trajectory generation model, the colors of the pins and holes, the spatial positions of the holes, the obstacles near the holes, and the lighting conditions in the simulation environment are dynamically adjusted to simulate diverse scene changes.

[0020] As a preferred technical solution, the diffusion-based trajectory generation model in the inference stage:

[0021] First, the randomly sampled Gaussian noise is denoised through DDIM; then the current point cloud data is input and the cross-attention mechanism is used to guide the denoising process, thereby generating an obstacle avoidance trajectory pointing to the vicinity of the hole.

[0022] As a preferred technical solution, the process of implementing the insertion operation of the end effector through the visual servo network is as follows:

[0023] Crop the input point cloud data to focus on the pins and holes;

[0024] Extract global features from the cropped point cloud data;

[0025] The global features are input into the multi-layer perceptron to predict the translation offset Δt and rotation offset Δr;

[0026] After the offset prediction is completed, visual servoing is used to iteratively estimate the translation offset Δt and rotation offset Δr, and the posture of the end effector is gradually adjusted to realize the insertion operation of the end effector.

[0027] As a preferred technical solution, the visual servo optimizes the posture of the end effector, and the specific steps are as follows:

[0028] Record the initial posture of the end effector [R|t], where R is the rotation matrix and t is the translation vector;

[0029] Convert the rotation offset Δr into the form of a rotation matrix ΔR;

[0030] Move the end effector to the next pose [R′|t′], where R′=ΔR·R, t′=Δt+t;

[0031] The iterative operation is repeatedly performed until the predicted offset is less than the error tolerance or the preset maximum number of iterations is reached, and finally the insertion instruction is executed to complete the task.

[0032] As a preferred technical solution, the loss L inserted by the visual servoing fine It consists of two parts:

[0033] L fine =L trans +L rot

[0034] Among them, L trans and L rot They represent the loss of translational offset and rotational offset respectively;

[0035] For the translation offset loss L trans , using the root mean square error to learn the magnitude of the translation offset and the cosine distance to learn the direction of motion:

[0036]

[0037] Among them, Δt * is the true value of the translation offset; Δt is the predicted translation offset;

[0038] For the rotational offset loss L rot , using the root mean square error to minimize the rotation error:

[0039]

[0040] in, is the true value of the rotation offset, and Δr is the predicted rotation offset.

[0041] As a preferred technical solution, in each iterative optimization, the posture of the end effector is initialized to the real posture of the target hole;

[0042] Randomly generate a negative value of the real translation offset -Δt and a negative value of the real rotation offset -Δθ_r, move the end effector to a new position according to the negative value of the real translation offset -Δt and the negative value of the real rotation offset -Δθ_r, and record the translation offset Δt and the rotation offset Δθ_r;

[0043] The robot is trained by behavioral cloning. During testing, the robot uses visual feedback to iteratively estimate and correct the translation offset Δt and rotation offset Δθ_r of the end effector. The visual servoing gradually optimizes the insertion posture of the end effector through a closed-loop control mechanism until the translation offset and rotation offset are reduced to within the preset threshold or the specified maximum number of iterations is reached.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1) The method of the present invention effectively solves the problem of large posture errors in the robot's initial state and the problem of obstacles that may be encountered during movement based on the search guided by obstacle avoidance diffusion. By generating an obstacle avoidance trajectory, the end effector is brought close to the target hole position, providing accurate initial conditions for subsequent operations. At the same time, the visual servo insertion based on the 3D point cloud uses a closed-loop control strategy to iteratively optimize the translation increment and rotation increment of the end effector, thereby achieving precise jacking operation under 6 degrees of freedom.

[0046] 2) The method of the present invention is different from the traditional method that directly relies on force sensors to make physical contact with the target object, and only uses visual sensors, thereby significantly improving the efficiency of plugging. And compared with other methods based on visual sensors, this method makes full use of 3D point cloud information, learns the three-dimensional spatial relationship between nail holes, and combines collision loss constraints to significantly improve the completion and accuracy of the 6-DOF nail hole assembly task. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of the complex scene nail hole assembly method based on obstacle avoidance diffusion guidance of the present invention.

[0048] Figure 2 It is a schematic diagram of diffusion model training of the complex scene nail hole assembly method based on obstacle avoidance diffusion guidance in the method of the present invention.

[0049] Figure 3 It is a visual servoing flow chart of the complex scene nail hole assembly method based on obstacle avoidance diffusion guidance in the method of the present invention. DETAILED DESCRIPTION

[0050] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0051] Example 1

[0052] In view of the deficiencies in the prior art, the present invention aims to solve the problems of posture differences, complex scene obstacle avoidance, and large initial alignment errors in the 6-DOF nail hole assembly process, and proposes a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance. The method is divided into two stages: in the hole search stage, a diffusion model is used to take 3D point cloud as input, and collision loss is used as a constraint condition to generate a rough trajectory to guide the end effector to avoid obstacles and approach the hole position; in the insertion stage, 3D point cloud features are extracted using Pointnet++ through a visual servo network, and the features are input into a multi-layer perceptron (MLP) network to predict the increments of the end effector joint coordinates and posture, and the closed-loop control method is combined to iteratively optimize the translation offset and rotation offset, thereby achieving precise insertion of the end effector. Through this coarse-to-fine control strategy, the robot can complete the 6-DOF insertion task.

[0053] To achieve the above purpose, the technical solution provided by the present invention is: a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance, such as Figure 1 The method steps shown include:

[0054] S1. Search based on obstacle avoidance diffusion guidance: Generate the trajectory of the end effector so that it can approach the target hole while avoiding obstacles, thereby effectively solving the problem of large posture error in the initial state of the robot and ensuring that obstacles can be flexibly avoided during movement.

[0055] S2. Visual servo insertion based on 3D point cloud: The point cloud data collected by the depth camera is converted into the translation increment and rotation increment of the end effector, and the insertion posture is iteratively optimized through closed-loop control to ensure high precision and high robustness of the insertion process.

[0056] Specifically, in step S1, a diffusion-based deep generation model is constructed for obstacle avoidance and rough hole position search. Consider the following Gaussian diffusion model, which includes two processes: forward diffusion and reverse generation.

[0057] The forward diffusion process describes the addition of noise to the given initial expert trajectory data:

[0058]

[0059] Among them, x0 represents the original data of the expert trajectory, x t represents the noisy data, represents a constant that changes with t, ε t Normally distributed noise is added.

[0060] The noisy data is used as the input of the Unet network, and Unet outputs the predicted noise. The mean square error (MSELoss) between the predicted noise and the added noise is calculated as the noise loss. In order to avoid collisions, a collision loss is also added:

[0061]

[0062] in, represents the terminal coordinate of the ith trajectory at time t, D j (·) The function represents the signed distance function (SDF). For a given closed surface, its signed distance function SDF maps a point in Euclidean space to the minimum distance between the point and the surface; if the point is inside the surface, the function returns a negative value.

[0063] In the reverse generation process, the Unet network trained in the forward diffusion is used to denoise the noisy data and gradually restore the data to a distribution as close to the original data as possible. In order to ensure that the gradient in the reverse generation process can be effectively back-propagated, a deterministic denoising diffusion implicit model (Denoising Diffusion Implicit Models, DDIM) is used. This method speeds up the generation process.

[0064] The training process of the diffusion model is as follows Figure 2 As shown. First, the point cloud data is extracted through the PointNet++ network, and then the trajectory generation of the diffusion model is conditionally controlled through the cross-attention mechanism, so that the generation process can combine the characteristics of the current point cloud data. During the training process, the mean square error (MSELoss) and collision loss between the predicted noise and the actual added noise are calculated to optimize the model and finally train our Unet network.

[0065] In the inference stage of the diffusion model, the randomly sampled Gaussian noise is first denoised through DDIM; then the current point cloud data is input and the cross-attention mechanism is used to guide the denoising process, thereby generating an obstacle avoidance trajectory pointing to the vicinity of the hole.

[0066] Furthermore, the search based on obstacle avoidance diffusion guidance uses training data from the CoppeliaSim simulator. X i Represents point cloud data, Traj iRepresents the trajectory, M represents the number of data, and each trajectory consists of 16 steps. In the data generation process, by dynamically adjusting the color of the pins and holes in the simulation environment, the spatial position of the holes, the obstacles near the holes, and the lighting conditions, a variety of scene changes are simulated to improve the model's adaptability to different environments. With the deepening of training, the model can achieve robust control and generate obstacle avoidance trajectories close to the target hole position based on the currently acquired point cloud. There is no need to strictly restrict the initial conditions, which provides a reliable initial condition guarantee for the subsequent precise insertion stage.

[0067] In step S2, the visual servo insertion based on the 3D point cloud is used to adjust the robot end position and realize the 6-DOF jacking operation. By estimating the offset between the current position and the target position, the posture of the end effector is optimized to achieve accurate insertion operation. The visual servo process is as follows: Figure 3 First, input the point cloud data Crop to Where N ′ ∈[0,N] represents the number of point cloud points close to the end effector. By cropping the point cloud data, we can focus more on the local information of the pins and holes, ignore other redundant information, and improve the effectiveness of feature extraction. Based on the cropped point cloud, this module uses the Pointnet++ network to extract global features and inputs them into a multi-layer perceptron (MLP) to predict the translation offset Δt=(Δx,Δy,Δz) and the rotation offset Δr=(Δθ x ,Δθ y ,Δθ z ), where the rotation offset is expressed in Euler angles. After the prediction, the module uses visual servoing (a closed-loop control method) to optimize the posture of the end effector to gradually achieve accurate insertion. Specifically, the posture of the end effector is gradually adjusted by iteratively estimating the translation offset Δt and the rotation offset Δr. First, the initial posture of the end effector is recorded Where R is the rotation matrix and t is the translation vector. The rotation offset Δr is converted into the form of a rotation matrix. We then move the end effector to the next pose Where R′=ΔR·R, t′=Δt+t, and the iterative operation is repeated until the predicted offset is less than the error tolerance or the preset maximum number of iterations is reached, and finally the insertion instruction is executed to complete the task.

[0068] The loss L of the visual servo insertion module based on 3D point cloud fine It consists of two parts:

[0069] L fine =L trans +L rot

[0070] Among them, L trans and L rot Represent the loss of translation offset and rotation offset respectively. For translation offset, the model uses both root mean square error and cosine distance to optimize the translation error:

[0071]

[0072] Among them, Δt * =(Δx * ,Δy * ,Δz * ) is the true value of the translation offset. The model uses the root mean square error to learn the magnitude of the translation offset and uses the cosine distance to learn the direction of motion. For the rotation offset, the root mean square error is used to minimize the rotation error:

[0073]

[0074] in, is the true value of the rotation offset.

[0075] Furthermore, the training data of the 3D point cloud-based visual servo insertion comes from the CoppeliaSim simulator. The data set is defined as Where L is the total number of data, X i and A i is a point cloud and action in a scene. The dataset is iterated L times by reverse sampling. In each iteration, the posture of the end effector is first initialized to the true posture of the target hole to ensure that the optimization starts from the standard position. Subsequently, -Δt and -Δθ are randomly generated. r , which are the negative values ​​of the true translation offset and rotation offset respectively. Then, according to -Δt and -Δθ r Move the end effector to a new position and record the translation offset Δt and rotation offset Δθ r As refinement data. After obtaining the data, the model is trained using behavioral cloning. During testing, the robot uses visual feedback to calculate the translation offset Δt and rotation offset Δθ of the end effector. r Iterative estimation and correction are performed. Through a closed-loop control mechanism, the insertion pose of the end effector is gradually optimized until the translation offset and rotation offset are reduced to within the preset threshold or the specified maximum number of iterations is reached.

[0076] The present invention uses the PointNet++ network to extract point cloud features. The PointNet++ network effectively improves the understanding and feature extraction capabilities of 3D point clouds by introducing a hierarchical feature learning mechanism and a multi-scale grouping strategy. The core idea is to build a multi-layer feature extraction network in the point cloud, and each layer consists of three main modules: sampling, grouping, and feature extraction: First, a representative center point is selected from the input point cloud using the farthest point sampling algorithm; then, for each center point, neighboring points are found within a certain radius to form a local point set; then, PointNet or its variants are used to extract features from each local point set to generate a local feature description. Through the hierarchical structure, the network aggregates the features of the point cloud layer by layer, thereby capturing both microscopic detail information and macroscopic global structure at the same time. Multi-scale grouping strategies (such as multi-scale grouping and multi-resolution grouping) further enhance the network's robustness to the geometric shape of the point cloud, ensuring that features can be effectively captured at different scales. Finally, PointNet++ maps multi-level features to the output space through feature aggregation, which can be used for a variety of 3D vision tasks such as classification and segmentation. In the application of the present invention, the final output space part of the PointNet++ network is removed, and the aggregated features are directly extracted for model design, thereby meeting the task requirements of nail hole assembly.

[0077] In the present invention, we introduce the extracted three-dimensional point cloud features into the diffusion model through the cross-attention mechanism to train the generation of robot trajectories. The cross-attention mechanism is an information fusion method in deep learning, which enables the diffusion model to efficiently focus on the three-dimensional point cloud features, thereby generating a robot motion path that meets environmental constraints. Specifically, the cross-attention mechanism dynamically adjusts the model's attention to different features by calculating the similarity between the query, key, and value. In this process, the current state of the diffusion model generates a query vector, and the point cloud features generate key and value vectors. The similarity between the query and the key is converted into a weight coefficient through the softmax function, and then multiplied by the value vector to obtain a feature representation of the fused environmental information. This mechanism enables the diffusion model to dynamically extract the most relevant information from the point cloud features based on the nail hole information in the environment when generating trajectories, thereby realizing the perception and adaptation of complex three-dimensional environments. By organically combining the three-dimensional point cloud features with the diffusion model, the cross-attention mechanism plays a key bridge role in the present invention, so that the trajectory generated by the diffusion model can reach the vicinity of the target hole position.

[0078] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A complex scene nail hole assembly method based on obstacle avoidance diffusion guidance, characterized in that the steps include: The point cloud data collected by the depth camera is used as the input of the diffusion-based trajectory generation model. The collision loss is used as a constraint condition. The cross-attention mechanism is used to conditionally control the trajectory generation of the diffusion-based trajectory generation model based on the point cloud data, and an end-effector trajectory is generated that avoids obstacles and reaches the target hole. The global features of point cloud data are extracted through the visual servo network, and the global features are input into the multi-layer perceptron network to predict the increments of the end-effector joint coordinates and posture. The closed-loop control method is used to iteratively optimize the translation offset and rotation offset to realize the insertion operation of the end-effector.

2. According to claim 1, a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance is characterized in that: The diffusion-based trajectory generation model includes two processes: forward diffusion and reverse generation: The diffusion-based trajectory generation model uses the noisy data obtained by adding noise to the previously given expert trajectory and the current point cloud data as input; In the forward diffusion process, the noisy data is used as the input of the Unet network, and the Unet network outputs the predicted noise; the Unet network model is optimized by calculating the mean square error and collision loss between the predicted noise and the actual added noise to obtain a trained Unet network; In the reverse generation process, the Unet network trained in the forward diffusion is used to denoise the noisy data using a denoising diffusion implicit model, and the cross-attention mechanism is used to guide the denoising process of the Unet network based on the current point cloud data to generate an obstacle avoidance trajectory of the end effector pointing to the vicinity of the hole position.

3. According to claim 2, a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance is characterized in that: The collision losses are as follows: in, represents the terminal coordinates of the ith trajectory at time t; D j The (·) function represents the signed distance function. For a given closed surface, the signed distance function maps a point in Euclidean space to the minimum distance from the point to the surface; if the point is inside the surface, the signed distance function returns a negative value.

4. According to claim 2, a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance is characterized in that: The cross-attention mechanism is used to conditionally control the trajectory generation of the diffusion model based on point cloud data, as follows: Extract features from point cloud data; The query vector is generated by the current state of the diffusion model, and the key vector and value vector are generated by the point cloud features. The similarity between the query vector and the key vector is converted into a weight coefficient through an activation function, and then multiplied with the value vector to obtain a feature representation of the fused environmental information.

5. According to claim 2, a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance is characterized in that: During the training process of the diffusion-based trajectory generation model, the colors of the pins and holes, the spatial positions of the holes, the obstacles near the holes, and the lighting conditions in the simulation environment are dynamically adjusted to simulate diverse scene changes.

6. The method for complex scene nail hole assembly based on obstacle avoidance diffusion guidance according to claim 1 is characterized in that: The diffusion-based trajectory generation model in the inference phase: First, the randomly sampled Gaussian noise is denoised through DDIM; then the current point cloud data is input and the cross-attention mechanism is used to guide the denoising process, thereby generating an obstacle avoidance trajectory pointing to the vicinity of the hole.

7. According to claim 1, a complex scene nail hole assembly method based on obstacle avoidance diffusion guidance is characterized in that: The process of implementing the insertion operation of the end effector through the visual servo network is as follows: Crop the input point cloud data to focus on the pins and holes; Extract global features from the cropped point cloud data; The global features are input into the multi-layer perceptron to predict the translation offset Δt and rotation offset Δr; After the offset prediction is completed, visual servoing is used to iteratively estimate the translation offset Δt and rotation offset Δr, and the posture of the end effector is gradually adjusted to realize the insertion operation of the end effector.

8. The method for complex scene nail hole assembly based on obstacle avoidance diffusion guidance according to claim 7 is characterized in that: The visual servo optimization of the posture of the end effector has the following specific steps: Record the initial posture of the end effector [R|t], where R is the rotation matrix and t is the translation vector; Convert the rotation offset Δr into the form of a rotation matrix ΔR; Move the end effector to the next pose [R′|t′], where R′=ΔR·R, t′=Δt+t; The iterative operation is repeatedly performed until the predicted offset is less than the error tolerance or the preset maximum number of iterations is reached, and finally the insertion instruction is executed to complete the task.

9. The method for complex scene nail hole assembly based on obstacle avoidance diffusion guidance according to claim 1 is characterized in that: The visual servoing inserts a loss L fine It consists of two parts: L fine =L trans +L rot Among them, L trans and L rot They represent the loss of translational offset and rotational offset respectively; For the translation offset loss L trans , using the root mean square error to learn the magnitude of the translation offset and the cosine distance to learn the direction of motion: Among them, Δt * is the true value of the translation offset; Δt is the predicted translation offset; For the rotational offset loss L rot , using the root mean square error to minimize the rotation error: in, is the true value of the rotation offset, and Δr is the predicted rotation offset.

10. The method for complex scene nail hole assembly based on obstacle avoidance diffusion guidance according to claim 1, characterized in that: In each iterative optimization, the posture of the end effector is initialized to the true posture of the target hole; Randomly generate a negative value of the real translation offset -Δt and a negative value of the real rotation offset -Δθ_r, move the end effector to a new position according to the negative value of the real translation offset -Δt and the negative value of the real rotation offset -Δθ_r, and record the translation offset Δt and the rotation offset Δθ_r; The robot is trained by behavioral cloning. During testing, the robot uses visual feedback to iteratively estimate and correct the translation offset Δt and rotation offset Δθ_r of the end effector. The visual servoing gradually optimizes the insertion posture of the end effector through a closed-loop control mechanism until the translation offset and rotation offset are reduced to within the preset threshold or the specified maximum number of iterations is reached.

Images