Part flexible assembly method, device and equipment based on target posture recognition and reinforcement learning, medium and product

Abstract

The invention discloses a part flexible assembly method, device and equipment based on target posture recognition and reinforcement learning, a medium and a product, and relates to the technical field of robot autonomous assembly control. The method comprises the following steps: inputting acquired image information into a part posture recognition model to obtain posture prediction information; then a PBVS method is adopted to perform motion control on a mechanical arm end effector of the assembly robot according to the coordinate transformation matrix, so that coarse positioning of the mechanical arm end effector for clamping the part to the insertion hole is achieved; adopting a reinforcement learning strategy network to adjust the tail end pose state information of the roughly positioned part, and determining tail end pose increment action information; and part assembling is conducted on the basis of the tail end pose increment action information, and motion information of a mechanical arm tail end executor is adjusted by adopting an admittance model constructed on the basis of contact force information fed back by assembling. According to the method, the precision, efficiency and robustness of autonomous flexible assembly of the assembly robot can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot autonomous assembly control, in particular to a part compliant assembly method and device based on target pose recognition and reinforcement learning, equipment, medium and product. BACKGROUND

[0002] In recent years, with the rapid development of intelligent manufacturing and industrial automation, robot autonomous assembly technology has been widely applied in high-end manufacturing fields such as automobiles, electronics, aerospace, etc. The assembly process often involves precise operations such as part grasping, positioning and insertion, especially in the case of small assembly gap or complex part shape, the traditional assembly method based on fixed trajectory or rigid control often cannot meet the requirements of high precision and high compliance. Therefore, the assembly strategy combining machine vision, intelligent perception fusion and adaptive control has gradually become a research hotspot.

[0003] In existing research, pose estimation technology based on vision has been widely used to obtain the spatial position of parts and assembly holes. Traditional methods rely on single modal feature extraction (such as using only 2D images or 3D point clouds), resulting in insufficient accuracy and robustness of pose estimation in cases of light changes, occlusions and complex backgrounds. At the same time, the traditional method of positioning parts and assembly holes often uses two-dimensional code positioning for precise positioning, which cannot cope with the random position of various parts and lacks generalization. On the other hand, in view of the inevitable fitting error and contact uncertainty in the assembly process, existing research attempts to combine force control to achieve compliant insertion. However, this kind of method usually relies on high-precision positioning in advance, and it is difficult to realize adaptive adjustment in complex environments. In recent years, the application of reinforcement learning in robot control has shown significant advantages. It can realize nonlinear control and strategy optimization in complex tasks through interaction with the environment. However, there are still many challenges in effectively combining reinforcement learning with robot assembly tasks, such as high-dimensional state space, high assembly precision requirements, and difficult strategy migration. At the same time, in recent years, research on reinforcement learning in assembly tasks has focused on building a large action model to complete the whole process of part grasping, handling and insertion. However, this kind of method often lacks modular design, and needs to retrain the entire large model for different working environments or assembly scenarios. Reinforcement learning itself is a difficult training and convergence method, resulting in high training cost, poor adaptability, and difficulty in flexible task migration and reuse. SUMMARY

[0004] The purpose of the present application is to provide a part compliant assembly method, device, equipment, medium and product based on target pose recognition and reinforcement learning, which can improve the precision, efficiency and robustness of autonomous compliant assembly of assembly robots.

[0005] To achieve the above object, the application provides the following scheme. In a first aspect, the application provides a part compliant assembly method based on target pose recognition and reinforcement learning, which is applied to an assembly robot. Obtain image information of the part and the socket in the assembly task; the image information includes an RGB image and a depth image; Input the image information into a part pose recognition model to predict the spatial position and pose of the part and the socket, and obtain pose prediction information; Use a PBVS method to control the motion of the end effector of the robot arm based on the pose prediction information and a coordinate transformation matrix, so as to realize the coarse positioning of the end effector of the robot arm to the part to the socket; the coarse positioning is the reserved tolerance area around the socket position; the reserved tolerance area is determined based on error information; the error information includes the Euclidean distance error of the preset part center point and the socket center point, and the angle deviation of the preset part axis vector and the socket axis vector; Use a reinforcement learning strategy network to adjust the end pose state information of the part after coarse positioning, and determine end pose incremental action information; the end pose state information includes the relative position, pose deviation and contact force information of the part and the socket; Based on the end pose incremental action information, the part is assembled, and a mobility model constructed based on the contact force information of the assembly feedback is used to adjust the motion information of the end effector of the robot arm; the motion information includes velocity and force.

[0006] In a second aspect, the application provides a part compliant assembly device based on target pose recognition and reinforcement learning, which comprises: An image information acquisition module is configured to obtain image information of a part and a socket in an assembly task; the image information includes an RGB image and a depth image; A part pose recognition module is configured to input the image information into a part pose recognition model to predict the spatial position and pose of the part and the socket, and obtain pose prediction information; a visual servo coarse positioning module configured to control movement of an end effector of the assembly robot based on the coordinate transformation matrix according to the pose prediction information by using a PBVS method, so as to achieve coarse positioning of the end effector to the receptacle; the coarse positioning is coarse positioning of the end effector to a reserved tolerance area around the receptacle; the reserved tolerance area is determined based on error information; the error information includes a preset Euclidean distance error between a center point of the part and a center point of the receptacle, and an angle deviation between a preset axis vector of the part and an axis vector of the receptacle; a reinforcement learning fine positioning module configured to adjust end pose state information of the part after coarse positioning by using a reinforcement learning strategy network, and determine end pose incremental action information; the end pose state information includes relative position, attitude deviation and contact force information of the part and the receptacle; a compliance assembly module configured to assemble the part based on the end pose incremental action information, and adjust movement information of the end effector of the assembly robot by using a compliance model constructed based on the contact force information of the assembly feedback; the movement information includes velocity and force.

[0007] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the part compliance assembly method based on target pose recognition and reinforcement learning.

[0008] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the part compliance assembly method based on target pose recognition and reinforcement learning.

[0009] In a fifth aspect, the present application provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the part compliance assembly method based on target pose recognition and reinforcement learning.

[0010] According to the embodiments provided in the present application, the following technical effects are disclosed: The application provides a part compliant assembly method and device based on target pose recognition and reinforcement learning, equipment, medium and product, which determines pose prediction information by using a part pose recognition model based on RGB images and depth images, and realizes accurate recognition and pose acquisition of an assembly task target. The PBVS method is used to realize coarse positioning of the part, and then the reinforcement learning strategy network and the guide model are used to realize fine adjustment and intelligent insertion. The combination of the part pose recognition model, the reinforcement learning strategy network and the guide model can significantly improve the precision, efficiency and robustness of the autonomous assembly of the assembly robot in a complex scene, so that the adaptive adjustment can be realized in a complex environment. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0012] Figure 1 The flowchart of the part compliant assembly method based on target pose recognition and reinforcement learning is shown in the figure. Figure 2 The flowchart of the part compliant assembly method based on target pose recognition and reinforcement learning is shown in the figure. Figure 3 The network diagram of the part pose recognition module is shown in the figure. Figure 4 The schematic diagram of the visual servo coarse positioning method is shown in the figure. Figure 5 The schematic diagram of the reinforcement learning fine positioning based on the reinforcement learning strategy network is shown in the figure. Figure 6 The schematic diagram of the guide control is shown in the figure. Figure 7 The structure diagram of the part compliant assembly device based on target pose recognition and reinforcement learning is shown in the figure. Figure 8 The structure diagram of the computer equipment provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0013] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0014] This application achieves a novel assembly process by combining pose recognition, visual servoing, and reinforcement learning strategies. It achieves coarse positioning of parts through pose estimation and visual feedback, and uses reinforcement learning to achieve fine adjustment and intelligent insertion, thereby significantly improving the accuracy, efficiency, and robustness of autonomous robot assembly in complex scenarios.

[0015] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] In one exemplary embodiment, a compliant assembly method for parts based on target posture recognition and reinforcement learning is provided, which is applied to an assembly robot.

[0017] like Figure 1 As shown, the part compliance assembly method based on target pose recognition and reinforcement learning includes: Step 100: Obtain image information of parts and sockets in the assembly task. Image information includes: RGB images and depth images.

[0018] Step 200: Input the image information into the part pose recognition model to predict the spatial position and pose of the part and the socket, and obtain pose prediction information.

[0019] In one embodiment, the method for determining the part pose recognition model specifically includes: Acquire historical image information of parts and sockets in historical assembly tasks, as well as their corresponding actual pose information.

[0020] Construct an initial part pose recognition network; the initial part pose recognition network includes: ResNet module, PointNet module and Transformer module.

[0021] Historical image information is input into the initial part pose recognition network, and an optimizer is used to train the network with the goal of minimizing the loss function. The trained part pose recognition network is obtained. The loss function is determined based on the output pose information of the initial part pose recognition network and the actual pose information corresponding to the historical image information. The trained part pose recognition network is then used as the part pose recognition model.

[0022] The expression for the loss function is: .

[0023] The loss function; Loss due to translation; For rotational losses; This represents the translation amount in the actual pose information; To output the translation amount in the pose information; For 3D point sets; This is the 3D rotation matrix in the actual pose information; To output the 3D rotation matrix in the pose information; 3D point set K The point on; For separation The nearest point; It is an L2 norm.

[0024] The processing steps for the output pose information of the initial part pose recognition network include: Based on the RGB images contained in historical image information, the parts and sockets are segmented at the pixel level, and the ResNet module is used to extract convolutional features from the segmented RGB images to obtain the feature vector of the segmented image.

[0025] The depth images contained in the historical image information are converted into point cloud information according to the camera intrinsic parameters, and the PointNet module is used to extract features from the point cloud information to obtain geometric structure features.

[0026] The feature vectors and geometric features of the segmented image are aligned and stitched together based on pixel coordinates, and then fused using the Transformer module. Finally, the output pose information is determined based on the fused features.

[0027] Step 300: Using the PBVS method, based on pose prediction information, motion control is performed on the end effector of the assembly robot's arm according to the coordinate transformation matrix to achieve coarse positioning of the end effector gripping the part to the insertion hole. Coarse positioning refers to a pre-defined tolerance area around the position where the part is gripped to the insertion hole; this tolerance area is determined based on error information, including: a preset Euclidean distance error between the part's center point and the insertion hole's center point, and a preset angular deviation between the part's axis vector and the insertion hole's axis vector.

[0028] In one embodiment, the PBVS method is used to perform motion control on the end effector of the assembly robot's robotic arm based on pose prediction information and coordinate transformation matrix, so as to achieve coarse positioning of the end effector to grip the part into the socket. Specifically, it includes: During the part-grabbing stage of the robotic arm's end effector, the relative pose of the end effector with respect to the part is determined based on the pose prediction information and the coordinate transformation matrix. ; .

[0029] relative pose Decompose the relative poses to obtain the decomposed values. .

[0030] in, This is the transformation matrix from the coordinate system of the robotic arm's end effector to the base coordinate system of the assembly robot; This is the transformation matrix from the part coordinate system to the assembly robot base coordinate system; Here is the rotation error matrix; This is the translation error vector.

[0031] Based on the decomposed relative pose The end effector of the robotic arm is positioned and moved along the error direction to generate end-effector speed control command information; the end-effector speed control command information includes end-effector linear velocity and end-effector angular velocity.

[0032] The desired six-dimensional velocity at the end of the joint is determined based on the end-effector velocity control command information; based on the desired six-dimensional velocity at the end of the joint, the joint velocity and joint position increment are calculated by inverting the Jacobian matrix.

[0033] Step 400: Use a reinforcement learning strategy network to adjust the end effector pose state information of the coarsely localized part, and determine the incremental motion information of the end effector pose. The end effector pose state information includes the relative position of the part and the socket, the attitude deviation, and the contact force information.

[0034] The reinforcement learning policy network is trained using the PPO algorithm with a KL divergence penalty term, based on the corresponding loss function, as well as the total reward function and loss function of the value network.

[0035] The expression for the total reward function is: .

[0036] The expression for the loss function corresponding to the reinforcement learning policy network is: .

[0037] The loss function of the value network is: .

[0038] in, For the total reward function; Rewards for positional deviation; The weight corresponding to the position deviation reward; Rewards for posture deviations; The weights corresponding to the attitude deviation reward; Rewards are given to those who engage with the activity. The weight corresponding to the contact force-guided reward; The loss function corresponding to the reinforcement learning policy network; To analyze the time steps in the sampling trajectory Experiential expectations; For parameters The current policy function; For parameters The old strategy function; For the estimation of the advantage function; For action; State; KL penalty coefficient; In the state The following parameters are The probability distribution of actions output by the current policy function; In the state The following parameters are The action probability distribution output by the old policy function; Let be the loss function of the value network; For value network parameters; For time step The theoretical expectation of the distribution of actions in the upper state; For value network state The estimate; The target reward for each state.

[0039] Step 500: Assemble parts based on the end-effector pose incremental motion information, and adjust the motion information of the robotic arm end effector using an admittance model constructed based on the contact force information of the assembly feedback. The motion information includes velocity and force.

[0040] The mathematical expression corresponding to the admittance model is: .

[0041] in, This is a virtual mass parameter matrix; Here is the damping parameter matrix; This is the stiffness parameter matrix; This is the actual position vector of the robotic arm's end effector; Let be the desired position vector of the robotic arm's end effector; The speed of the robotic arm's end effector; Acceleration of the end effector of the robotic arm; External environmental forces.

[0042] This application presents a compliant assembly method for parts based on target pose recognition and reinforcement learning. This method is designed for complex assembly scenarios and modularizes compliant assembly into a novel approach encompassing pose recognition, visual servoing, reinforcement learning strategy optimization, and admittance control insertion. A multimodal perception network integrating RGB and depth information is used to perform six-dimensional pose estimation of the target parts and sockets, achieving accurate target identification and pose acquisition. Visual servoing, based on pose estimation and visual feedback, enables coarse positioning of the parts. Reinforcement learning and admittance control are then used for fine adjustment and intelligent insertion, significantly improving the accuracy, efficiency, and robustness of autonomous robot assembly in complex scenarios, and enabling adaptive adjustment in complex environments.

[0043] The robotic arm determines the posture and coordinate position of the part and the socket using a posture recognition model. It then uses visual servoing to grip the part and coarsely position it to the socket. The next step is to use reinforcement learning to finely position the part and the socket so that the part is perpendicular to the socket and can be directly inserted. Finally, admittance control is used to smoothly insert the part.

[0044] The pose recognition model uses ResNet, PointNet and Transformer modules to process RGB and depth images, and achieves pose estimation by matching the 3D of parts with the sockets.

[0045] In practical applications, the specific design steps include the following: Step S1: The assembly task is divided into four sub-modules: part posture recognition module, which identifies the spatial position and posture of parts and sockets through a vision system; visual servo coarse positioning module, which drives the end effector of the robotic arm to approach the target position based on visual information; reinforcement learning fine positioning module, which uses reinforcement learning strategy to fine-tune the end effector posture to optimize assembly accuracy; and admittance control compliant assembly module, which achieves compliant insertion through force feedback to avoid jamming and damage. The four sub-modules are performed sequentially.

[0046] Step S2: Perform hand-eye calibration on the camera and robotic arm, establish the camera coordinate system and the robot base coordinate system within the workspace, and obtain the transformation matrix from the camera coordinate system to the base coordinate system. Transformation matrix from the robotic arm coordinate system to the base coordinate system .

[0047] Step S3: Use the RGB image and depth image captured by the camera as input to the part pose recognition model, and use the pose recognition model to predict the spatial position and pose of the part and the hole. The part pose output by the model is... Right now The posture of the socket Right now Establish the transformation matrix from the part coordinate system to the base coordinate system. Transformation matrix from jack coordinate system to base coordinate system .

[0048] in, This represents the translational component of the part in the base coordinate system; The quaternion components represent the part's attitude in the base coordinate system; This represents the translation component of the socket in the base coordinate system; The quaternion components represent the hole orientation in the base coordinate system.

[0049] Step S4: Based on the pose information of the part and hole obtained in Step S3, a visual servo control loop is constructed to achieve coarse positioning of the robotic arm's end effector. Using the Position-Based Visual Servoing (PBVS) method, the spatial error between the part and the hole is calculated, and end-effector speed or position adjustment commands are generated based on the error feedback. The control commands are then mapped into joint space motion commands via Jacobian matrix through robot inverse kinematics, enabling the robotic arm to move the part to a predetermined tolerance area near the hole, including the Euclidean distance error between the part's center point and the hole's center point, and the angular deviation between the part's axis vector and the hole's axis vector.

[0050] Step S5: After coarse localization, the policy network in the reinforcement learning module is used to finely adjust the end-effector pose of the part. The relative position of the part and the hole, the pose deviation, and the contact force information are used as state inputs, and the output is a small incremental motion of the end-effector pose. The state is a 12-dimensional vector. The positional deviation is a 3-dimensional vector. The attitude deviation is a 3-dimensional vector. in, The rotation vectors are the x, y, and z axes, respectively, and the contact force is a 6-dimensional vector. The output action is a 6-dimensional vector. ,in For three-axis translation fine adjustment , , For three-axis rotational fine-tuning.

[0051] It is a six-dimensional contact force vector; These are linear forces along the x, y, and z axes of the base coordinate system, respectively. These represent the torques about the base coordinate axes x, y, and z, respectively. These are the three-dimensional position coordinates of the part in the base coordinate system; These are the three-dimensional position coordinates of the socket in the base coordinate system; This is a transpose.

[0052] Setting a reward letter Position deviation reward Posture deviation reward Contact-guided rewards .

[0053] Step S6: After the reinforcement learning localization is completed, the admittance control phase begins. An admittance model is constructed based on the contact force feedback information. By dynamically adjusting the speed and force of the end effector, the part is made compliant when contacting the orifice, avoiding jamming and damage, and achieving smooth insertion. When the part is inserted, the axial force... Achieve the set target assembly force The insertion is complete at that time.

[0054] like Figure 2 The diagram illustrates the flowchart of a compliant assembly method for parts based on target pose recognition and reinforcement learning. Specifically, it includes: S1, calibrating the camera and robotic arm to determine the coordinate system; S2, training the pose recognition model to determine the poses of the parts and holes; S3, constructing a visual servo control loop based on the pose information of the parts and holes; S4, finely adjusting the end-effector pose of the parts using the policy network in the reinforcement learning module; and S5, inserting the parts with constant force along the z-axis of the hole coordinate system using admittance control.

[0055] Step S1: Establish the camera coordinate system and the robot base coordinate system within the workspace, and obtain the transformation matrix from the camera coordinate system to the base coordinate system. The base coordinates are established in the coordinate system of the robot arm's base. A transformation matrix from the robot arm's end effector coordinate system to the base coordinates is created from the URDF file. In this implementation, an Intel RealSense D435 camera is used to acquire RGB and depth images of the workspace.

[0056] Step S2, as follows Figure 3 As shown, a pose recognition model (part pose recognition model) is constructed, with an input RGB image. The corresponding input depth image is .in, For height, For width.

[0057] Step S2.1: Use a pre-trained segmentation model to perform pixel-level segmentation of the RGB image for parts and sockets of interest. This model has powerful segmentation capabilities to extract regions of interest. Use a pre-trained ResNet module, specifically ResNet-101, to perform convolutional feature extraction on the segmented RGB image to obtain the feature vector of the segmented image. Where N is the number of points in the selected region of interest. The characteristic latitude.

[0058] Step S2.2: Convert the depth map into a point cloud using camera intrinsic parameters. Each point The point cloud is input into the PointNet module to extract geometric features. .in, The three-dimensional coordinates corresponding to each point.

[0059] Step S2.3: Divide the feature vector of the segmented image and geometric structural features Based on pixel coordinate alignment, the concatenated features are input into the Transformer module. The Transformer module's encoder models the global dependencies between multimodal features, and then the fused features are input into the regression head MLP (Multilayer Perceptron) to output the 6D pose of the target part and the target socket. The part pose is... Right now The posture of the socket Right now The transformation matrices from the part coordinate system, the socket coordinate system, to the base coordinate system are as follows: and .

[0060] Step S2.4: Training uses a joint loss function to optimize position and pose, with both losses having equal weights. Translation loss The model is trained by calculating the squared Euclidean distance between the true and predicted translations using L2 norm loss, and minimizing this spatial distance between the predicted and true positions. Rotation loss is also used. ShapeMatch-Loss does not directly compare the rotation matrices themselves, but rather compares the degree of matching of the 3D point sets after transformation by the rotation matrices. Therefore, the loss function of this network is: .

[0061] In this embodiment, the optimizer used is Adam, the batch size is set to 64, the initial learning rate is 0.0001 and gradually decays, and the decay factor is 0.1.

[0062] Step S3, as follows Figure 4 As shown, a visual servo control loop is constructed, extracting the transformation matrices from the posture recognition model to the part coordinate system, the socket coordinate system, and the base coordinate system. and Combined with the transformation matrix from the camera coordinate system to the base coordinate system. The transformation matrix from the end-effector coordinate system to the base coordinate system of the robotic arm .

[0063] Step S3.1, during the stage of the robotic arm gripping the part, the relative pose of the end effector with respect to the part is calculated as follows: , It can be decomposed into a translation error vector. Rotation error matrix ,Right now .Will Converted to a rotation vector representation .

[0064] Step S3.2: To move the end effector pose along the error direction, generate end effector velocity control commands, including the end effector linear velocity. terminal angular velocity ,in and The gain is a proportional coefficient used to adjust the convergence speed. The expected velocity in the final six dimensions is... .

[0065] Step S3.3: Use the Jacobian matrix Inverse calculation of joint velocities. Calculate the 6-dimensional velocities of the end effector. Mapped to the joint space of the robotic arm ,in It is a Jacobian matrix False rebellion, The speed commands are given to each joint. This is done over a small time step. Obtain the joint position increment: .

[0066] Step S3.4: Using the methods from steps 3.1 to 3.3, after the robotic arm grips the part, repeat the method, setting the target position as the insertion hole position. Move the robotic arm, which has gripped the part, to the insertion hole position for coarse positioning. The pre-reserved tolerance area around the target position includes the Euclidean distance error between the part's center point and the insertion hole's center point, and the angular deviation between the part's axis vector and the insertion hole's axis vector. Then, stop the vision servoing phase.

[0067] Step S4: After coarse positioning is completed, as follows Figure 5 As shown, the policy network in the reinforcement learning module is used to finely adjust the end effector pose of the part. The relative position, attitude deviation, and contact force information between the part and the hole are used as state inputs, and the output is a small incremental motion of the end effector pose. The state is a 12-dimensional vector, the position deviation is a 3-dimensional vector, the attitude deviation is a 3-dimensional vector, the contact force is a 6-dimensional vector, and the output motion is a 6-dimensional vector.

[0068] Step S4.1: Based on the state in step S4 It is a 13-dimensional vector that includes the positional deviation between the current position of the part and the target position (hole position) in the base coordinate system, i.e. The attitude deviation is a 3D vector. in The rotation vectors are the x, y, and z axes, respectively, and the contact force is a 6-dimensional vector. The output action is a 6-dimensional vector. ,in For three-axis translation fine adjustment , , For three-axis rotational fine-tuning.

[0069] Step S4.2: Set the total reward function as follows .

[0070] Position deviation reward: .

[0071] Posture deviation reward: .

[0072] Contact Force Guidance Reward: .

[0073] Step S4.3: Train the fine localization policy network using the PPO algorithm with a KL divergence penalty term. Policy network loss function: .

[0074] in, The update magnitude is used to dynamically adjust the strategy, preventing overly rapid jumps. The value network optimizes by minimizing the mean squared error between the current state value and the reward; its loss function is: .in, The target reward is set for each state. The policy and value networks are trained alternately.

[0075] Step S4.4: To deploy the policy network from simulation to reality, various perturbations (such as sensor noise, force feedback errors, etc.) are added to the simulation to improve the policy's robustness and enhance its adaptability to real-world uncertainties. After each action output by the policy network, the attitude prediction model and the six-dimensional force sensor output the observation value at that moment, causing the policy network to make the next action output. When the attitude error is less than a preset threshold, that is, when the Euclidean distance and attitude angle error between the predicted part attitude and the socket reference attitude are both lower than the set tolerance value and the output of the six-dimensional force / torque sensor stably approaches zero, it indicates that the contact force has basically disappeared, indicating that the part and the socket are aligned. The reinforcement learning task is then complete, ending the policy network control phase. Set the position error threshold. Attitude error threshold Position error (Euclidean distance) Attitude error (rotation angle) That is, when and The reinforcement learning localization phase ends at this time. Transpose the rotation matrix of the current part; This is the socket rotation matrix.

[0076] Step S5: After the reinforcement learning localization is completed, the admittance control phase begins, such as... Figure 6 End-point control is implemented. An admittance model is constructed based on contact force feedback information. , The environmental forces are acquired from a six-dimensional force / torque sensor. The controller then determines the contact force... Dynamically adjust the speed of the robotic arm's end effector The relative displacement of the end effector can be obtained by integration, and the joint movement is controlled by the Jacobian matrix through the controller of the robotic arm itself. When encountering slight obstacles (such as small deviations or friction), the end effector automatically adjusts its trajectory.

[0077] By dynamically adjusting the speed and force of the end effector, the part exhibits flexibility upon contact with the orifice, preventing jamming and damage, and achieving smooth insertion. When the part is inserted, the axial force... Achieve the set target assembly force At that time Insertion complete.

[0078] In one exemplary embodiment, such as Figure 7 As shown, a compliant assembly device for parts based on target pose recognition and reinforcement learning is provided, comprising: The image information acquisition module is used to acquire image information of parts and sockets in the assembly task; the image information includes: RGB image and depth image.

[0079] The component posture recognition module is used to input the image information into the component posture recognition model to predict the spatial position and posture of the component and the socket, and obtain posture prediction information.

[0080] The visual servo coarse positioning module is used to perform motion control on the end effector of the assembly robot's arm based on the pose prediction information and the coordinate transformation matrix using the PBVS method, so as to achieve coarse positioning of the end effector gripping the part to the insertion hole. The coarse positioning is a reserved tolerance area around the position where the part is gripped to the insertion hole. The reserved tolerance area is determined based on error information. The error information includes: a preset Euclidean distance error between the center point of the part and the center point of the insertion hole, and a preset angular deviation between the axis vector of the part and the axis vector of the insertion hole.

[0081] The reinforcement learning fine localization module is used to adjust the end pose state information of the part that has completed coarse localization using a reinforcement learning strategy network, and to determine the end pose incremental action information; the end pose state information includes the relative position of the part and the socket, the attitude deviation and the contact force information.

[0082] The admittance control compliant assembly module is used to assemble parts based on the end-effector pose increment motion information, and to adjust the motion information of the robotic arm end effector using an admittance model constructed based on the contact force information of the assembly feedback; the motion information includes speed and force.

[0083] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 8 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores part compliant assembly data based on target posture recognition and reinforcement learning. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a part compliant assembly method based on target posture recognition and reinforcement learning.

[0084] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0085] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0086] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0087] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0088] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0089] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0090] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0091] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for compliant assembly of parts based on target pose recognition and reinforcement learning, characterized in that, The part compliant assembly method based on target posture recognition and reinforcement learning is applied to assembly robots; The part compliance assembly method based on target pose recognition and reinforcement learning includes: Acquire image information of parts and sockets in the assembly task; the image information includes: RGB images and depth images; The image information is input into the part pose recognition model to predict the spatial position and pose of the part and the socket, thereby obtaining pose prediction information. Using the PBVS method, based on the pose prediction information, the motion control of the end effector of the assembly robot arm is performed according to the coordinate transformation matrix to achieve coarse positioning of the end effector gripping the part to the insertion hole; the coarse positioning is a reserved tolerance area around the position where the part is gripped to the insertion hole; the reserved tolerance area is determined based on error information; the error information includes: a preset Euclidean distance error between the center point of the part and the center point of the insertion hole, and a preset angular deviation between the axis vector of the part and the axis vector of the insertion hole; A reinforcement learning strategy network is used to adjust the end-effector pose state information of the part after coarse positioning to determine the end-effector pose incremental motion information; the end-effector pose state information includes the relative position of the part and the socket, the attitude deviation and the contact force information. The parts are assembled based on the incremental motion information of the end effector pose, and the motion information of the robotic arm end effector is adjusted using an admittance model constructed based on the contact force information of the assembly feedback; the motion information includes speed and force.

2. The part compliance assembly method based on target pose recognition and reinforcement learning according to claim 1, characterized in that, The method for determining the part posture recognition model specifically includes: Acquire historical image information and corresponding actual pose information of parts and sockets in historical assembly tasks; Construct an initial part pose recognition network; the initial part pose recognition network includes: a ResNet module, a PointNet module, and a Transformer module; Historical image information is input into an initial part pose recognition network, and an optimizer is used to train the network with the goal of minimizing the loss function, resulting in a trained part pose recognition network. The loss function is determined based on the output pose information of the initial part pose recognition network and the actual pose information corresponding to the historical image information. The trained part pose recognition network is defined as the part pose recognition model. The processing steps for the output pose information of the initial part pose recognition network include: Based on the RGB images contained in historical image information, the parts and sockets are segmented at the pixel level, and the ResNet module is used to extract convolutional features from the segmented RGB images to obtain the feature vector of the segmented image. The depth images contained in the historical image information are converted into point cloud information according to the camera intrinsic parameters, and the PointNet module is used to extract features from the point cloud information to obtain geometric structure features. The feature vectors and geometric features of the segmented image are aligned and stitched together based on pixel coordinates, and then fused using the Transformer module. Finally, the output pose information is determined based on the fused features.

3. The part compliance assembly method based on target pose recognition and reinforcement learning according to claim 2, characterized in that, The expression for the loss function is: ； in, The loss function; Loss due to translation; For rotational losses; This represents the translation amount in the actual pose information; To output the translation amount in the pose information; For 3D point sets; This is the 3D rotation matrix in the actual pose information; To output the 3D rotation matrix in the pose information; 3D point set K The point on; For separation The nearest point; It is an L2 norm.

4. The part compliance assembly method based on target pose recognition and reinforcement learning according to claim 1, characterized in that, Using the PBVS method, based on the pose prediction information, motion control is performed on the end effector of the assembly robot's robotic arm according to the coordinate transformation matrix to achieve coarse positioning of the end effector in gripping the part to the insertion hole. Specifically, this includes: During the part-grabbing stage of the robotic arm end effector, the relative pose of the end effector relative to the part is determined based on the pose prediction information and the coordinate transformation matrix. ; ; relative pose Decompose the relative poses to obtain the decomposed values. ; in, This is the transformation matrix from the coordinate system of the robotic arm's end effector to the base coordinate system of the assembly robot; This is the transformation matrix from the part coordinate system to the assembly robot base coordinate system; Here is the rotation error matrix; This is the translation error vector; Based on the decomposed relative pose The end effector of the robotic arm is moved along the error direction to determine the end effector's pose and generate end-effector speed control command information; the end-effector speed control command information includes end-effector linear velocity and end-effector angular velocity. The desired six-dimensional velocity at the end is determined based on the end-vehicle velocity control command information. Based on the expected six-dimensional velocity at the end, the joint velocity and joint position increment are calculated by inverting the Jacobian matrix.

5. The part compliance assembly method based on target pose recognition and reinforcement learning according to claim 1, characterized in that, The reinforcement learning policy network is trained using the PPO algorithm with KL divergence penalty term, based on the corresponding loss function, as well as the total reward function and loss function of the value network. The expression for the total reward function is: ； The expression for the loss function corresponding to the reinforcement learning policy network is: ； The loss function of the value network is: ； in, For the total reward function; Rewards for positional deviation; The weight corresponding to the position deviation reward; Rewards for posture deviations; The weights corresponding to the attitude deviation reward; Rewards are given to those who engage with the activity. The weight corresponding to the contact force-guided reward; The loss function corresponding to the reinforcement learning policy network; To analyze the time steps in the sampling trajectory Experiential expectations; For parameters The current policy function; For parameters The old strategy function; For the estimation of the advantage function; For action; State; KL penalty coefficient; In the state The following parameters are The probability distribution of actions output by the current policy function; In the state The following parameters are The action probability distribution output by the old policy function; Let be the loss function of the value network; For value network parameters; For time step The theoretical expectation of the distribution of actions in the upper state; For value network state The estimate; The target reward for each state.

6. The part compliance assembly method based on target pose recognition and reinforcement learning according to claim 1, characterized in that, The mathematical expression corresponding to the admittance model is: ； in, This is a virtual mass parameter matrix; Here is the damping parameter matrix; This is the stiffness parameter matrix; This is the actual position vector of the robotic arm's end effector; Let be the desired position vector of the robotic arm's end effector; The speed of the robotic arm's end effector; Acceleration of the end effector of the robotic arm; External environmental forces.

7. A compliant assembly device for parts based on target pose recognition and reinforcement learning, characterized in that, include: The image information acquisition module is used to acquire image information of parts and sockets in the assembly task; The image information includes: RGB image and depth image; The component posture recognition module is used to input the image information into the component posture recognition model to predict the spatial position and posture of the component and the socket, and obtain posture prediction information. The visual servo coarse positioning module is used to perform motion control on the end effector of the assembly robot's robotic arm based on the pose prediction information and the coordinate transformation matrix using the PBVS method, so as to achieve coarse positioning of the end effector gripping the part to the insertion hole; the coarse positioning is a reserved tolerance area around the position where the part is gripped to the insertion hole; the reserved tolerance area is determined based on error information; the error information includes: a preset Euclidean distance error between the center point of the part and the center point of the insertion hole, and a preset angular deviation between the axis vector of the part and the axis vector of the insertion hole; The reinforcement learning fine localization module is used to adjust the end pose state information of the part that has completed coarse localization using a reinforcement learning strategy network, and to determine the end pose incremental action information; the end pose state information includes the relative position, attitude deviation and contact force information of the part and the socket. The admittance control compliant assembly module is used to assemble parts based on the end-effector pose increment motion information, and to adjust the motion information of the robotic arm end effector using an admittance model constructed based on the contact force information of the assembly feedback; the motion information includes speed and force.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the part compliant assembly method based on target pose recognition and reinforcement learning as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the part compliant assembly method based on target posture recognition and reinforcement learning as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the part compliant assembly method based on target posture recognition and reinforcement learning as described in any one of claims 1-6.

Images