A complex irregular shape workpiece compliant assembly method based on machine vision

Abstract

The application discloses a kind of complex irregular shape workpiece compliant assembly method based on machine vision, belong to robot assembly technical field.It includes: the 3D point cloud of the assembly workpiece is obtained, and its contour is obtained as the template contour of assembly workpiece;Based on the RGB-D image of assembly base obtained by the depth camera, the contour of assembly base assembly hole is extracted using Canny edge detection, and its centroid position is calculated;Two-stage contour registration method is used, and the contour of base assembly hole is matched with the template contour of assembly workpiece, to realize the prepositioning of assembly workpiece;Assembly robot is according to the posture of prepositioning, and the force signal collected by force sensor is inserted into assembly workpiece according to admittance algorithm until assembly is completed.The application combines visual geometry guidance with force perception model, and realizes compliant assembly of complex shape workpiece according to admittance control algorithm.The method is efficient, safe, and provides reliable technical support for industrial scene application.

Description

Technical Field

[0001] This invention relates to the field of robot assembly technology, and in particular to a compliant assembly method for complex, irregularly shaped workpieces based on machine vision. Background Technology

[0002] Robot assembly technology is widely used in modern industrial automated production and manufacturing, such as Figure 1 As shown in (a), most robotic assembly research focuses on the assembly of circular shafts and holes, a relatively simple process commonly found in the assembly of many mechanical products. Robots can efficiently and accurately complete assembly tasks through standardized mating methods, such as insertion and rotation. However, with the diversification of product shapes, the assembly difficulty of complex-shaped workpieces is constantly increasing, and traditional assembly strategies are no longer sufficient to meet the needs of intelligent industrial production. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention proposes an assembly strategy based on the synergy of visual geometric guidance and precise force perception. Canny edge detection is used to extract the contour information of assembly holes, and a two-stage contour registration algorithm is combined to achieve visual geometric guidance. Furthermore, based on a precise force perception model, the interference of noise, zero drift, gravity, and other factors on the assembly process is effectively reduced, enabling the robot to accurately perceive environmental contact forces and achieve precise force perception. Simultaneously, a compliant control method based on admittance control enables compliant assembly of complex-shaped workpieces.

[0004] To achieve the above objectives, this invention proposes a compliant assembly method for complex irregular-shaped workpieces based on machine vision. The method is used in an industrial robot assembly system, which includes: an assembly robot, a depth camera, a force sensor, an assembly base, and an assembly workpiece.

[0005] The method includes the following steps:

[0006] (1) Obtain the 3D point cloud of the assembled workpiece, and obtain its contour based on normal estimation as the template contour of the assembled workpiece.

[0007] (2) Based on the depth camera, the RGB-D image of the assembly base is acquired, the RGB image is preprocessed, the outline of the assembly hole of the assembly base is extracted using Canny edge detection, and its centroid position is calculated.

[0008] (3) A two-stage contour registration method based on spindle deviation and affine transformation is adopted to match the contour of the assembly hole of the assembly base with the template contour of the assembly workpiece, eliminate rotation error, and realize the pre-positioning of the assembly workpiece by combining the centroid position.

[0009] (4) The assembly robot inserts the workpiece according to the force signal collected by the force sensor and the admittance control algorithm according to the pre-positioned posture until the assembly is completed.

[0010] Furthermore, step (1) specifically includes:

[0011] (1.1) For each point in the 3D point cloud of the assembled workpiece, the K-nearest neighbor algorithm is used to select its neighborhood;

[0012] (1.2) Calculate the local density in the neighborhood of each point in the 3D point cloud of the assembled workpiece;

[0013]

[0014] Where: ρ i Let N be the local density at point i. i Let V be the number of points in the neighborhood of the i-th point, and let V be the volume of the neighborhood of the i-th point.

[0015] (1.3) Compare the local density of each point with the set density threshold T ρ In comparison, local densities lower than T ρ The points are marked as boundary points;

[0016] (1.4) For a boundary point, calculate its normal vector and determine whether its direction is consistent with the normal direction of its neighboring points. If they are consistent, keep it; otherwise, delete the boundary point.

[0017] (1.5) After traversing and marking all points, project the boundary points onto the 2D plane and use the resulting boundary point profile as the template profile for the assembled workpiece.

[0018] Furthermore, step (1.1) specifically includes:

[0019] (1.1.1) Calculate the Euclidean distance between each point in the 3D point cloud of the assembled workpiece and the current point;

[0020]

[0021] Where: (x i y i , z i (x) represents the coordinates of the current point. j y j , z j Let d be the coordinates of any point. i，j The Euclidean distance between the two points;

[0022] (1.1.2) Sort the Euclidean distances between each point and the current point in ascending order;

[0023] (1.1.3) Select the K points with the smallest Euclidean distance from the current point as the neighborhood of the current point.

[0024] Furthermore, step (2) specifically involves:

[0025] (2.1) Obtain an RGB-D image of the assembly base using the depth camera;

[0026] (2.2) Perform grayscale conversion, global binarization, and closing operation preprocessing on the RGB image;

[0027] (2.3) Use the Canny edge detection algorithm to extract the contours in the preprocessed RGB image;

[0028] (2.4) Count the number of pixels inside each closed contour and retain the contour with the largest number of pixels as the contour of the mounting hole of the mounting base.

[0029] (2.5) Calculate the centroid position based on the contour of the mounting hole of the mounting base;

[0030]

[0031] Where: A is the area of ​​the contour, N is the number of points in the contour, (x c y c (x) represents the coordinates of the centroid. i y i () represents the coordinates of the boundary point;

[0032] (2.6) Obtain depth information from the RGB-D image, and then obtain the 3D position of the centroid in the world coordinate system.

[0033] Furthermore, the steps are as follows:

[0034] (3.1) First stage;

[0035] (3.1.1) Calculate the minimum bounding rectangle of the assembly hole contour of the assembly base and the template contour of the assembly workpiece, and obtain their respective main axis directions;

[0036] (3.1.2) Calculate the difference in principal axis angles between the two smallest bounding rectangles, and use it as the registration angle difference for the first stage;

[0037] Δθ=θ2-θ1

[0038] Where: θ2 is the deflection angle between the main axis and the x-axis of the template contour of the assembly workpiece; θ1 is the deflection angle between the main axis and the x-axis of the assembly hole contour of the assembly base.

[0039] (3.2) Second stage;

[0040] (3.2.1) Construct the affine transformation matrix T;

[0041]

[0042] Where: a and d determine the scaling of the contour, b and c determine the shearing of the contour, and t x t y The parameter is the translation parameter;

[0043] x1=a * x+b * y+t x

[0044] y1 = c * x+d * y+t y

[0045] Where (x, y) are the coordinates of the point in the original contour coordinate system, and (x1, y1) are the new coordinates of the point after the affine transformation;

[0046] (3.2.2) Let the outline of the assembly hole of the assembly base be the original outline, and the outline of the assembly workpiece after the first stage adjustment be the outline after affine transformation. Combine the RANSAC method to eliminate the influence of outliers, and solve the affine transformation matrix T with the minimum root mean square error by the following formula.

[0047]

[0048] Where: p i It is the i-th point of the mounting hole contour of the mounting base, q i It is the i-th corresponding point of the template outline of the assembled workpiece, and n is the number of outline points;

[0049] (3.2.3) Based on the affine transformation matrix T obtained in step (3.2.2), calculate the registration angle difference in the second stage;

[0050] Δθ′=arctan2(c,α)

[0051] Where: Δθ′ represents the rotation angle difference in the second stage;

[0052] (3.3) Rotate the assembly workpiece according to the sum of the registration angle differences between the two stages, and adjust the position of the assembly workpiece according to the position of the center of mass to complete the pre-positioning of the assembly workpiece.

[0053] θ=Δθ+Δθ′

[0054] Where: θ is the final calculated actual rotation angle difference.

[0055] Furthermore, step (4) specifically involves:

[0056] (4.1) Real-time acquisition of force signals measured by force sensors and preprocessing based on force perception model;

[0057] (4.2) Determine the force signal f in the z-axis direction z With force threshold f th The relationship, if f z >f th And at this time, the position p of the assembly robot's end effector z With position threshold p th If the difference is within the set range, it means that the assembled workpiece is in contact with the assembly hole of the assembly base, and step (4.3) is executed; otherwise, return to step (4.1).

[0058] (4.3) Insert the workpiece into the assembly based on the admittance control algorithm until the assembly is completed.

[0059] Furthermore, the preprocessing based on the force perception model specifically involves:

[0060] Establish the assembly robot base coordinate system, world coordinate system, and force sensor coordinate system; adjust the assembly robot pose; collect multiple sets of assembly robot pose information under stable conditions and the corresponding force signals measured by the force sensor; and calculate the gravity, center of gravity, and zero drift of the force sensor of the assembly robot end effector.

[0061] Based on the calculated gravity, center of gravity, and zero drift of the force sensor of the assembly robot's end effector, gravity compensation is performed on the force signal measured by the force sensor.

[0062] Extended Kalman filtering is applied to the force signal measured by the compensated force sensor.

[0063] Furthermore, step (4.3) specifically includes:

[0064] (4.3.1) Based on the desired position x of the assembly robot's end effector d The inverse kinematics is used to convert it into the initial angle q of each joint of the assembly robot;

[0065] (4.3.2) Based on the angles of each joint, calculate the position deviation at the current moment using forward kinematics. and speed deviation Combined with the force signal F measured by the preprocessed force sensor ext The deviation between the actual position and the expected position of the assembly robot's end effector at the next moment is calculated.

[0066]

[0067] Where M is the system's mass matrix, B is the damping matrix, and K is the stiffness matrix. This represents the velocity deviation at time t. This represents the deviation between the actual position and the desired position at time t. F represents the acceleration at time t+1. ext The force signal measured by the force sensor after preprocessing;

[0068] (4.3.3) Based on the deviation between the actual position and the expected position of the assembly robot's end effector at the next moment. And the desired position x of the assembly robot end effector d Obtain the actual position x of the assembly robot's end effector at the next moment. u ;

[0069]

[0070] (4.3.4) Using inverse kinematics, determine the actual position x of the assembly robot's end effector at the next moment. u Converted into control quantities q for each joint u To take control;

[0071] (4.3.5) Based on the control amount q of each joint u Once the new joint angle is obtained, repeat steps (4.3.2)-(4.3.5) to achieve continuous compliant control based on force feedback until assembly is complete.

[0072] The beneficial effects of this invention are:

[0073] This invention uses visual geometry to precisely calculate the centroid position of the assembly hole contour and corrects the rotational deviation between the template contour and the actual contour using a two-stage contour registration algorithm, enabling accurate pre-positioning of irregularly shaped workpieces. Simultaneously, through a precise force perception model, the robot can accurately sense the contact forces with the environment and continuously adjust its assembly posture according to an admittance control algorithm, ultimately achieving compliant assembly of complex-shaped workpieces. This method is highly efficient and safe, providing reliable technical support for industrial applications. Attached Figure Description

[0074] Figure 1 This is a schematic diagram of the assembly of a circular workpiece and a workpiece with a complex shape according to an embodiment of the present invention.

[0075] Figure 2 This is a schematic diagram of the process of a compliant assembly method for complex irregular-shaped workpieces based on machine vision, according to an embodiment of the present invention.

[0076] Figure 3 This is a schematic diagram illustrating the template outline fabrication process for assembling the workpiece according to an embodiment of the present invention.

[0077] Figure 4This is a schematic diagram of the process for extracting the contour of the assembly hole of the assembly base according to an embodiment of the present invention.

[0078] Figure 5 This is a schematic diagram of the contour registration method according to an embodiment of the present invention.

[0079] Figure 6 This is a schematic diagram of the first stage of coarse registration in an embodiment of the present invention.

[0080] Figure 7 This is a schematic diagram of the coordinate system of the assembly robot according to an embodiment of the present invention.

[0081] Figure 8 This is a schematic diagram of the gravity of the end effector of the assembly robot in an embodiment of the present invention.

[0082] Figure 9 This is a comparison diagram of the force sensor's three axial data before and after EKF filtering in an embodiment of the present invention.

[0083] Figure 10 This is the equivalent model for admittance control in an embodiment of the present invention.

[0084] Figure 11 This is a block diagram of the admittance control structure according to an embodiment of the present invention. Detailed Implementation

[0085] The present invention will be further explained and described below with reference to the accompanying drawings and embodiments.

[0086] like Figure 2 As shown, this invention provides a compliant assembly method for complex, irregularly shaped workpieces based on machine vision, used in an industrial robot assembly system. The system includes: an assembly robot, a depth camera, a force sensor, an assembly base, and the workpiece to be assembled. The method includes the following steps:

[0087] S101. Obtain the 3D point cloud of the assembled workpiece, and use its contour as the template contour of the assembled workpiece based on normal estimation.

[0088] like Figure 3 As shown, the process of creating the template outline for the assembled workpiece is as follows:

[0089] (1) For each point in the 3D point cloud of the assembled workpiece, the K-nearest neighbor algorithm is used to select its neighborhood.

[0090] You can export the STL model of the workpiece from SolidWorks and convert it to a PLY file to obtain the 3D point cloud of the assembly workpiece. Alternatively, you can obtain the 3D point cloud of the assembly workpiece through laser scanning, depth cameras, or other methods.

[0091] The selection of the neighborhood is as follows:

[0092] (1.1) Calculate the Euclidean distance between each point in the 3D point cloud of the assembled workpiece and the current point.

[0093]

[0094] Where: (x i y i , z i (x) represents the coordinates of the current point. j y j , z j Let d be the coordinates of any point. i，j Let be the Euclidean distance between the two points.

[0095] (1.2) Sort the Euclidean distances between each point and the current point in ascending order of distance.

[0096] (1.3) Select the K points with the smallest Euclidean distance from the current point as the neighborhood of the current point.

[0097] (2) Calculate the local density in the neighborhood of each point in the 3D point cloud of the assembled workpiece.

[0098]

[0099] Where: ρ i Let N be the local density at point i. i Let V be the number of points in the neighborhood of the i-th point, and let V be the volume of the neighborhood of the i-th point.

[0100] (3) Compare the local density of each point with the set density threshold T ρ In comparison, local densities lower than T ρ The points are marked as boundary points.

[0101] (4) For a boundary point, calculate its normal vector and determine whether its direction is consistent with the normal direction of its neighboring points. If they are consistent, keep it; otherwise, delete the boundary point.

[0102] (5) After traversing and marking all points, project the boundary points onto the 2D plane and use the resulting boundary point profile as the template profile for the assembled workpiece.

[0103] S102. Based on the depth camera, acquire the RGB-D image of the assembly base, preprocess the RGB image, use Canny edge detection to extract the contour of the assembly hole of the assembly base, and calculate its centroid position.

[0104] like Figure 4 As shown, the process of extracting the contour of the assembly hole on the assembly base and calculating the centroid position is as follows:

[0105] (1) Obtain RGB-D images of the assembly base using a depth camera.

[0106] (2) Perform grayscale conversion, global binarization and closing operation preprocessing on the RGB image.

[0107] (3) Use the Canny edge detection algorithm to extract the contours in the preprocessed RGB image.

[0108] (4) Count the number of pixels inside each closed contour and retain the contour with the largest number of pixels as the contour of the mounting hole of the mounting base.

[0109] (5) Calculate the centroid position based on the contour of the mounting hole of the mounting base.

[0110]

[0111] Where: A is the area of ​​the contour, N is the number of points in the contour, (x c y c (x) represents the coordinates of the centroid. i y i ) represents the coordinates of the boundary point.

[0112] (6) Obtain depth information from the RGB-D image, and then obtain the 3D position of the centroid in the world coordinate system.

[0113] S103. A two-stage contour registration method based on spindle deviation and affine transformation is adopted to match the contour of the assembly hole of the assembly base with the template contour of the assembly workpiece, eliminate rotation error, and realize the pre-positioning of the assembly workpiece by combining the centroid position.

[0114] like Figure 5 As shown, the pre-positioning of the assembled workpiece is divided into two stages, specifically:

[0115] (1) First stage.

[0116] (1.1) Calculate the minimum bounding rectangle of the assembly hole contour of the assembly base and the contour of the assembly workpiece, and obtain their respective principal axis directions, such as... Figure 6 As shown.

[0117] (1.2) Calculate the difference Δθ between the principal axis directions of the two smallest bounding rectangles.

[0118] Δθ=θ2-θ1 (4)

[0119] Where θ2 is the deflection angle between the main axis of the assembly workpiece contour and the x-axis; θ1 is the deflection angle between the main axis of the base assembly hole contour and the x-axis.

[0120] (2) Second stage.

[0121] (2.1) Construct the affine transformation matrix T.

[0122]

[0123] Where: a and b are parameters that determine the rotation of the contour, c and d are parameters that determine the scaling of the contour, and t x t y The parameters of the translation.

[0124] x1=a * x+b * y+t x

[0125] y1 = c * x+d * y+t y (6)

[0126] Where (x, y) are the coordinates of the point in the original contour coordinate system, and (x1, y1) are the new coordinates of the point after the affine transformation.

[0127] (2.2) Let the outline of the assembly hole of the assembly base be the original outline, and the outline of the assembly workpiece after the first stage adjustment be the outline after affine transformation. Combine the RANSAC method to eliminate the influence of outliers, and solve the transformation matrix T with the minimum root mean square error by the following formula.

[0128]

[0129] Where: p i It is the i-th point of the mounting hole contour of the mounting base, q i It is the i-th corresponding point of the template outline of the assembled workpiece, and n is the number of outline points.

[0130] (2.3) Based on the affine transformation matrix T obtained in step (2.2), calculate the registration angle difference in the second stage.

[0131] Δθ′=arctan2(c,a) (8)

[0132] Where: Δθ′ represents the rotation angle difference in the second stage;

[0133] (3) Rotate the assembly workpiece according to the sum of the registration angle difference between the two stages, and adjust the position of the assembly workpiece according to the position of the center of mass to complete the pre-positioning of the assembly workpiece.

[0134] θ=Δθ+Δθ′ (9)

[0135] Where: θ is the final calculated actual rotation angle difference.

[0136] S104. Force signals are collected by force sensors, and the insertion of the assembly workpiece is completed based on the force signals and the admittance algorithm until the assembly is completed.

[0137] (1) Real-time acquisition of force signals measured by force sensors and preprocessing based on force perception model.

[0138] The force perception model combines a gravity compensation algorithm with an extended Kalman filter to achieve accurate perception of environmental contact forces.

[0139] (1.1) Establish the assembly robot base coordinate system, world coordinate system, and force sensor coordinate system, adjust the assembly robot pose, collect multiple sets of assembly robot pose information under stable conditions and their corresponding end force sensor information, calculate the gravity, center of gravity and zero drift of the assembly robot end effector and the force sensor; perform gravity compensation on the force sensor measurement signal based on the calculated gravity, center of gravity and zero drift of the assembly robot end effector and the force sensor.

[0140] In practical applications, the force measured by force sensors includes not only the actual force generated by contact with the environment, but is also affected by factors such as sensor zero drift, external noise, and the weight of the end effector. The calibration process can eliminate the bias of the sensor itself, such as zero drift, while gravity compensation eliminates the influence of the end effector's gravity, making the measurement results more accurate and able to truly reflect the contact force with the environment.

[0141] like Figure 7 As shown, O w -X w Y w Z w O represents the world coordinate system. b -X b Y b Z b To represent the robot's base coordinate system, when building a platform, the robot's base coordinate system often cannot be guaranteed to be absolutely horizontal. Therefore, let the world coordinate system revolve around the X-axis. w After rotating the axis by θ1, continue to rotate around Y. w The axis rotation θ2 will coincide with the robot's base coordinate system. The rotation matrix for this process is denoted as...

[0142]

[0143] Similarly, O f -X f Y f Z f Let the coordinate system of the force sensor be denoted as . Then the rotation matrix from the base coordinate system to the force sensor coordinate system is denoted as .

[0144]

[0145] The gravity G of the end effector in the world coordinate system can be calculated from the two formulas above.w Relationship with gravity under force sensor:

[0146]

[0147] Where: G0 = [0, 0, -G] T , representing the initial gravity of the end effector in the world coordinate system.

[0148] When the force sensor measures data at the point of contact with the environment, it is affected by zero-point drift and the gravity of the end effector. The actual contact force F0 measured at this time can be expressed as:

[0149]

[0150] Simplifying the expression, we get:

[0151]

[0152] In the formula: I is a 3×3 identity matrix, and:

[0153]

[0154] Next, the robot's pose is adjusted N times, and the real data from the force sensor under different poses is collected. Substituting this data into the calculation formula (14), we can obtain:

[0155]

[0156] Let P = [p] x p y p z f x0 f y0 f z0 ] T Let P represent the expression, the above equation can be simplified to:

[0157] F = RP (17)

[0158] Then P can be expressed by the following formula:

[0159] P = (R T R) -1 R T F (18)

[0160] From this, we can obtain the various quantities in P, and further obtain the gravity G of the robot's end effector and the rotation angles θ1 and θ2:

[0161]

[0162] Figure 8This is a gravity diagram of the end effector in the force sensor coordinate system. Let the coordinates of the center of gravity Q in the force sensor coordinate system be (q...). x q y q z The torque in the initial state is [t]. x0 t y0 t z0 ] T The torque at any pose is [t] x t y t z ] T Then, the torque generated by the gravity G of the end effector in the force sensor coordinate system is:

[0163]

[0164] Combining the above equation with formula (14) and simplifying, we get:

[0165]

[0166] remember:

[0167]

[0168] Substituting equation (22) into equation (21) and rearranging it into matrix form:

[0169]

[0170] Similarly, by adjusting the robot's pose N times and collecting real data from the force sensor under different poses, substituting into equation (23) yields:

[0171]

[0172] [q] x q y q z c x c y c z ] T Using C as the denot, equation (24) can be simplified to:

[0173] M = FC (25)

[0174] Then C can be expressed by the following formula:

[0175] C = (F T F) -1 F T M (26)

[0176] Therefore, the centroid coordinates (q) can be calculated. x q y qz ) and parameter c x c y c z By combining equations (18) and (22), the zero-point drift of the sensor torque can be obtained:

[0177]

[0178] The gravitational component G of the end effector in the force sensor coordinate system can be obtained from equations (12) and (20). t =[g x g y g z ] T and torque component T t =[t gx t gy t gz ] T Therefore, the force F that needs to be compensated can be determined. c With torque T c :

[0179]

[0180] After compensation, the actual contact force F between the robot's end effector and the environment can be directly measured using a force sensor. r and torque T r :

[0181]

[0182] (1.2) Perform extended Kalman filtering on the compensated force sensor measurement signal.

[0183] The signals acquired by force sensors often contain environmental interference, sensor noise, and high-frequency vibrations generated during assembly. These noises can affect the accuracy of the signal and thus the success rate of assembly tasks. To reduce noise interference in this system, this invention introduces an extended Kalman filter (EKF). As an effective nonlinear filtering method, EKF can denoise in real time under dynamic environments by predicting and updating sensor data, providing smooth and accurate force signals.

[0184] The basic idea of ​​EKF is to linearize the nonlinear function by performing a first-order Taylor expansion, thereby applying Kalman filtering at each step. The following equations can be established in the system:

[0185]

[0186] In equation (30), x k Let f(x) be the current state vector. k-1 uk-1 ) is the state transition equation, h(x) k () represents the measurement equation. k and v k This includes process noise and measurement noise.

[0187] For the state transition equation and measurement equation in x k-1 Perform a first-order Taylor expansion at this point:

[0188]

[0189] in,

[0190] During the prediction phase, the state estimate x at time k-1 is used. k-1 and control input u k Derive the state prediction value at time k. and prediction error covariance

[0191]

[0192] During the update phase, combined with the predicted values Calculate the Kalman gain using the measurement equation:

[0193]

[0194] In the final correction phase, the state and covariance are updated:

[0195]

[0196] Real-time correction and prediction of sensor data using extended Kalman filtering can effectively suppress the influence of random noise and improve measurement accuracy. To verify the filtering effect of extended Kalman filtering on a six-dimensional force sensor, a set of data along the x, y, and z axes during robot motion was collected and processed using extended Kalman filtering. The results are shown below. Figure 9 As shown.

[0197] pass Figure 9 As can be seen, the extended Kalman filter has a good filtering effect, which can effectively filter out the noise in the measurement data, enabling the force sensor to obtain stable and accurate environmental contact force, and providing accurate force sensing for the assembly of complex-shaped workpieces.

[0198] (2) Determine the force signal f in the z-axis direction z With force threshold f th The relationship, if f z >f th And at this time, the robot's end effector is at position p. z With position threshold pth If the difference is within the set range, it indicates that the workpiece is in contact with the base hole, and step (3) is executed; otherwise, return to (1).

[0199] (3) Based on the admittance control algorithm, adjust the angle of each joint of the robot to complete the insertion of the workpiece until the assembly is completed.

[0200] Traditional rigid control methods rely on precise trajectory tracking, requiring the robot's end effector to move strictly along a predetermined path. However, this rigid control cannot effectively cope with complex or unpredictable external environments, especially when the robot needs to interact or cooperate with its environment, such as grasping, assembly, or medical surgical procedures. Compliant control, on the other hand, emphasizes controlling the robot's relative motion with its external environment through the coordination of force and position, thereby ensuring that the robot can interact with its environment without damage.

[0201] Admittance control is a type of compliant control. Its basic idea is that the robot uses force / torque sensors to acquire real-time force information between its end effector and the environment. This force is then input into an admittance model described by a second-order linear system. The desired position is calculated based on the admittance model and transmitted to the position controller to drive the robot to the corresponding position. Due to its natural and flexible response to external forces, admittance control is widely used in scenarios requiring human-robot interaction or precision operation.

[0202] The admittance control process of a robot can be described by the following second-order differential equation:

[0203]

[0204] Where M is the system's mass matrix, B is the damping matrix, and K is the stiffness matrix; x e =(xx) d () indicates the deviation between the actual position and the desired position. and These represent the deviations in velocity and acceleration, respectively; F ext This refers to the contact force between the robot and its environment. The admittance model can be equivalent to, for example... Figure 10 The "mass-spring-damping" system shown.

[0205] Admittance control requires an external contact force F. ext To calculate the displacement or velocity required by the robot, equation (35) is rewritten as follows:

[0206]

[0207] The acceleration deviation is further integrated twice to obtain the position deviation of the robot arm's end effector:

[0208]

[0209] The obtained positional deviation x e With expected position x d The actual position x is obtained by adding them together. u Then, using the robot's inverse kinematics, the desired position q of each joint is calculated. u .

[0210] The block diagram of the admittance control system is as follows: Figure 11 As shown.

[0211] (3.1) Based on the desired position x of the assembly robot's end effector d This is converted into the initial angles q of each joint of the assembly robot through inverse kinematics.

[0212] (3.2) Based on the angles of each joint, calculate the current position deviation at the current moment using forward kinematics. and speed deviation Combined with the current pre-processed force sensor measurement signal F ext The deviation between the actual position and the desired position of the assembly robot's end effector at the next moment is calculated.

[0213]

[0214] Where M is the system's mass matrix, B is the damping matrix, and K is the stiffness matrix. This represents the velocity deviation at time t. This represents the deviation between the actual position and the desired position at time t. F represents the acceleration at time t+1. ext This is the preprocessed measurement signal from the force sensor.

[0215] (3.3) Based on the deviation between the actual position and the expected position of the assembly robot's end effector at the next moment. And the desired position x of the assembly robot end effector d Obtain the actual position x of the assembly robot's end effector at the next moment. u ;

[0216]

[0217] (3.4) Using inverse kinematics, the actual position x of the assembly robot's end effector at the current moment is obtained. u Converted into control quantities q for each joint u .

[0218] (3.5) Based on the control amount q of each joint uOnce the new joint angle is obtained, repeat steps (3.2)-(3.5) to achieve continuous compliant control based on force feedback until assembly is complete.

[0219] To verify the effectiveness of the above-mentioned compliant assembly method for complex-shaped workpieces that combines vision and force perception, the present invention conducted workpiece assembly verification. The experiment showed that the registration angle error of the method was about 0.5°, the centroid positioning error was about 1.17mm, the assembly force was maintained below 6N, and the average assembly time was about 37s. The experiment verified the advantages of this method in terms of efficiency and safety, and provided reliable technical support for industrial applications.

[0220] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A compliant assembly method for complex, irregularly shaped workpieces based on machine vision, characterized in that, The method is used in an industrial robot assembly system, which includes: an assembly robot, a depth camera, a force sensor, an assembly base, and an assembly workpiece. The method includes the following steps: (1) Obtain the 3D point cloud of the assembled workpiece and use its contour as the template contour of the assembled workpiece based on normal estimation. (2) Obtain the RGB of the mounting base based on the depth camera. The D image is preprocessed, the RGB image is used to extract the contour of the assembly hole of the assembly base using Canny edge detection, and its centroid position is calculated. (3) A two-stage contour registration method based on spindle deviation and affine transformation is adopted to match the contour of the assembly hole of the assembly base with the template contour of the assembly workpiece, eliminate rotation error, and realize the pre-positioning of the assembly workpiece by combining the centroid position. The specific steps (3) are as follows: (3.1) First stage; (3.1.1) Calculate the minimum bounding rectangle of the assembly hole contour of the assembly base and the template contour of the assembly workpiece, and obtain their respective main axis directions; (3.1.2) Calculate the difference in principal axis angles between the two smallest bounding rectangles, and use it as the registration angle difference for the first stage; in: The main axis for assembling the template contour of the workpiece and The deflection angle of the axis; The spindle with the mounting hole contour for assembling the base The deflection angle of the axis; (3.2) Second stage; (3.2.1) Construct the affine transformation matrix ; in: and Determines the scaling of the outline. and The cut that determines the outline , These are the parameters for translation; in: These are the coordinates of a point in the original contour coordinate system. These are the new coordinates of the point after the affine transformation; (3.2.2) Let the contour coordinates of the assembly hole of the assembly base be the original contour, and the contour of the assembly workpiece adjusted in the first stage be the contour after affine transformation. Combined with the RANSAC method, the influence of outliers is eliminated, and the transformation matrix with the minimum root mean square error is solved by the following formula. ; in: It is the first of the mounting base mounting hole contours One point, It is the first template outline of the assembled workpiece One corresponding point, It is the number of contour points; (3.2.3) The affine transformation matrix obtained from step (3.2.2) Calculate the registration angle difference in the second stage; in: This indicates the difference in rotation angle during the second stage; (3.3) Rotate the assembly workpiece according to the sum of the registration angle differences between the two stages, and adjust the position of the assembly workpiece according to the position of the center of mass to complete the pre-positioning of the assembly workpiece. in: This is the final calculated actual angle difference; (4) The assembly robot inserts the workpiece according to the force signal collected by the force sensor and the admittance control algorithm according to the pre-positioned posture until the assembly is completed.

2. The compliant assembly method for complex irregular-shaped workpieces based on machine vision according to claim 1, characterized in that, The specific steps (1) are as follows: (1.1) For each point in the 3D point cloud of the assembled workpiece, the K-nearest neighbor algorithm is used to select its neighborhood; (1.2) Calculate the local density in the neighborhood of each point in the 3D point cloud of the assembled workpiece; in: Let be the local density at point i. Let be the number of points in the local neighborhood of the i-th point. Let be the volume of the neighborhood of the i-th point; (1.3) Compare the local density of each point with the set density threshold. In comparison, local density is lower than The points are marked as boundary points; (1.4) For a boundary point, calculate its normal vector and determine whether its direction is consistent with the normal direction of its neighboring points. If they are consistent, keep it; if they are not consistent, delete the boundary point. (1.5) After traversing and marking all points, project the boundary points onto the 2D plane and use the resulting boundary point profile as the template profile for the assembled workpiece.

3. The compliant assembly method for complex irregular-shaped workpieces based on machine vision according to claim 2, characterized in that, The specific steps (1.1) are as follows: (1.1.1) Calculate the Euclidean distance between each point in the 3D point cloud of the assembled workpiece and the current point; in: The coordinates of the current point. Let be the coordinates of any point. The Euclidean distance between the two points; (1.1.2) Sort the Euclidean distances between each point and the current point in ascending order; (1.1.3) Select the K points with the smallest Euclidean distance from the current point as the neighborhood of the current point.

4. The compliant assembly method for complex irregular-shaped workpieces based on machine vision according to claim 1, characterized in that, Step (2) specifically involves: (2.1) Obtain the RGB of the mounting base using the depth camera. D-image; (2.2) Perform grayscale conversion, global binarization, and closing operation preprocessing on the RGB image; (2.3) Use the Canny edge detection algorithm to extract the contours in the preprocessed RGB image; (2.4) Count the number of pixels inside each closed contour and retain the contour with the largest number of pixels as the contour of the mounting hole of the mounting base; (2.5) Calculate the centroid position based on the contour of the mounting hole of the mounting base; in: It is the area of ​​the outline. It is the number of points in the outline. Let these be the coordinates of the centroid position. These are the coordinates of the boundary points; (2.6) Obtain depth information from the RGB-D image, and then obtain the 3D position of the centroid in the world coordinate system.

5. The compliant assembly method for complex irregular-shaped workpieces based on machine vision according to claim 1, characterized in that, Step (4) specifically involves: (4.1) Real-time acquisition of force signals measured by force sensors and preprocessing based on force perception model; (4.2) Determine the force signal in the z-axis direction and force threshold If the relationship, And at this time, the position of the assembly robot's end effector With position threshold If the difference is within the set range, it means that the assembled workpiece is in contact with the assembly hole of the assembly base, and step (4.3) is executed; otherwise, return to (4.1). (4.3) Insert the workpiece into the assembly based on the admittance control algorithm until the assembly is completed.

6. The compliant assembly method for complex irregular-shaped workpieces based on machine vision according to claim 5, characterized in that, The preprocessing based on the force perception model specifically involves: Establish the assembly robot base coordinate system, world coordinate system, and force sensor coordinate system; adjust the assembly robot pose; collect multiple sets of assembly robot pose information under stable conditions and the corresponding force signals measured by the force sensor; and calculate the gravity, center of gravity, and zero drift of the force sensor of the assembly robot end effector. Based on the calculated gravity, center of gravity, and zero drift of the force sensor of the assembly robot's end effector, gravity compensation is performed on the signal measured by the force sensor. Extended Kalman filtering is applied to the force signal measured by the compensated force sensor.

7. The compliant assembly method for complex irregular-shaped workpieces based on machine vision according to claim 5, characterized in that, The specific steps (4.3) are as follows: (4.3.1) Based on the desired position of the assembly robot's end effector By using inverse kinematics, this can be converted into the preliminary angles of each joint of the assembly robot. ; (4.3.2) Based on the angles of each joint, calculate the position deviation at the current moment using forward kinematics. and speed deviation Combined with the force signal measured by the preprocessed force sensor The deviation between the actual position and the expected position of the assembly robot's end effector at the next moment is calculated. ; in: It is the system's quality matrix. It is the damping matrix. It is the stiffness matrix. This represents the velocity deviation at time t. This represents the deviation between the actual position and the desired position at time t. Represents the acceleration at time t+1. The force signal measured by the force sensor after preprocessing; (4.3.3) Based on the deviation between the actual position and the expected position of the assembly robot's end effector at the next moment. and the desired position of the assembly robot's end effector Obtain the actual position of the assembly robot's end effector at the next moment. ; (4.3.4) By using inverse kinematics, the actual position of the assembly robot's end effector at the next moment is determined. Converted into control quantities for each joint To take control; (4.3.5) Based on the control amount of each joint Once the new joint angle is obtained, repeat steps (4.3.2)-(4.3.5) to achieve continuous compliant control based on force feedback until assembly is complete.

Images