Registration and estimation method for spatial frame components based on local contour feature information

Abstract

This invention discloses a registration estimation method for spatial frame components based on local contour feature information. The steps are as follows: S1, determine the transformation relationship between the sensor coordinate system and the base coordinate system of the assembly mechanism; S2, construct a mathematical model to realize pose registration between the target workpiece and the spatial frame component; S3, determine the number of local contour features and their positions on the spatial frame component, and obtain multiple contour feature information vectors of the spatial frame component under the theoretical assembly pose in the sensor coordinate system; S4, obtain the transformation matrix between the theoretical assembly pose of the target workpiece and the base coordinate system of the assembly mechanism; S5, solve the registration pose of the target workpiece based on the contour feature information vectors of the spatial frame component under the actual assembly pose and the pose registration mathematical model. The registration effect of this method has both good deviation control range and repeatable positioning accuracy, and the data volume and time consumption during processing are small, and there are no restrictions on the contour shape of the component.

Description

Technical Field

[0001] This invention relates to the field of automated assembly on production lines in the automotive, electronics, and aerospace manufacturing industries, and in particular to a method for registration and estimation of spatial frame components based on local contour feature information. Background Technology

[0002] In modern manufacturing, rapid, contactless, intelligent visual measurement and registration technology is crucial for improving industrial production efficiency and product quality, and is a vital component of future intelligent manufacturing. By collecting feature information of three-dimensional objects, extracting the absolute and relative spatial positions of target components, and forming corresponding decision-making algorithms, it can be widely applied in industrial production fields such as dimensional control, integrity inspection, visual guidance and positioning, surface inspection, safety area monitoring, 3D object recognition, navigation, sealing, and component assembly. Furthermore, using robots as assembly mechanisms to complete visually guided assembly is essential to further achieve highly efficient intelligent and automated operations, improve manufacturing efficiency, facilitate maintenance, and save costs.

[0003] Positioning and registration of spatial frame components is a crucial step in automated assembly in the automotive, electronics, and aerospace industries. Taking the automotive manufacturing industry as an example, automated assembly of hoods, doors, and windshields often involves camera-based positioning of the vehicle body, using methods such as calibration plate-based methods, coordinate measurement methods, and SNK camera-based methods. Among these, calibration plates require high precision, cannot be damaged during use, and occupy storage space. Positioning the calibration plate within the camera's field of view requires high repeatability, resulting in high costs. Coordinate measurement methods require manual operation of the measuring arm to adjust the camera position, while simultaneously using a rotation and tilting device to precisely aim at the object being measured. This process is relatively cumbersome and time-consuming, significantly reducing production efficiency. In SNK camera-based methods, precise calibration is difficult to achieve when multiple SNK cameras are used for measurement, leading to measurement errors and affecting workpiece positioning accuracy.

[0004] With the continuous integration of laser technology and traditional visual inspection methods, point cloud data provides richer and more flexible feature information for the measurement and positioning of feature points of the measured object, which helps to form new detection algorithms and technical solutions. Currently, patent CN119260374A has been published, proposing a 3D vision-guided automotive door follow-up assembly system. This technology uses a hinge hole pose matrix to determine the real-time installation pose of the hinge holes during real-time assembly, rather than using contour feature information to determine the installation pose. Therefore, it has certain limitations and cannot assemble components without hinge holes. Patent CN119359775A has been published, proposing a low-overlap-rate projectile point cloud registration method based on line feature detection. This technology performs coarse registration by fitting the central axis and then uses the Hough transform method to extract linear bullet mark features to complete the fine registration of the source point cloud and the target point cloud. This technology is only applicable to projectile registration and not... This method is applicable to the registration of components of other shapes and does not involve assembly work. The published patent CN119313713A proposes a method and device for aircraft assembly point cloud registration based on LiDAR scan data. This technology integrates the density, attitude, geometry and point matching consistency loss of the global predicted rigid body transformation matrix and the real rigid body transformation matrix, and obtains the target rigid body transformation matrix with the goal of minimizing the comprehensive loss. This technology uses a large amount of 3D point cloud data, and deep learning is required when extracting features, which is time-consuming. Moreover, after the aircraft model changes, feature extraction and learning need to be re-performed, which is not conducive to the application of assembly in real-time operation scenarios. Summary of the Invention

[0005] The purpose of this invention is to provide a registration and estimation method for spatial frame components based on local contour feature information, so as to solve the problems of low positioning accuracy and high cost of traditional spatial frame components in the prior art.

[0006] Therefore, the technical solution of the present invention is as follows:

[0007] A spatial frame component registration and estimation method based on local contour feature information, comprising the following steps:

[0008] S1. Using the hand-eye calibration method, determine the transformation relationship between the sensor coordinate system and the assembly mechanism base coordinate system in the assembly mechanism;

[0009] S2. Construct a mathematical model to achieve pose registration between the target workpiece and spatial frame components;

[0010] S3. Based on the length characteristics of the spatial frame components, determine the number of local contour features and their positions on the spatial frame components; obtain multiple contour feature information vectors of the spatial frame components in the theoretical assembly pose under the sensor coordinate system through the vision sensor in the assembly mechanism.

[0011] S4. Based on the correspondence between the spatial frame components and the target workpiece under their theoretical assembly poses, obtain the transformation matrix between the theoretical assembly pose of the target workpiece and the coordinate system of the assembly mechanism base.

[0012] S5. Using the same method as in step S3, obtain multiple contour feature information vectors of the spatial frame component under the actual assembly pose. Based on the mathematical model of pose registration and the multiple contour feature information vectors of the spatial frame component under the theoretical assembly pose, use a nonlinear optimization algorithm to solve for the registration pose estimation vector of the target workpiece and the spatial frame component under the actual pose, so as to obtain the registration pose of the target workpiece.

[0013] Furthermore, the specific implementation steps of step S1 are as follows:

[0014] S101. Using the same installation method, replace the vision sensor in the assembly mechanism with a line laser profile sensor, and place a standard ball below the line laser profile sensor.

[0015] S102. Change the posture of the assembly mechanism to perform multiple measurements on a standard ball set at a fixed position using a line laser profile sensor, and obtain the relationship between the coordinates of the ball's center in the coordinate system of the assembly mechanism base and the coordinates in the sensor coordinate system during each measurement.

[0016] S103. Based on the relationship obtained in step S102, the least squares method is used to solve for the position of the center of the sphere by fitting the cross section circle, and the transformation relationship between the sensor coordinate system and the end flange coordinate system of the assembly mechanism is obtained.

[0017] Furthermore, in step S102, the coordinates of the sphere center in the coordinate system of the assembly mechanism base are X. b The X coordinates in the sensor coordinate system during each measurement sj The relational expression satisfies:

[0018]

[0019] In the formula, r ej Let t be the rotation matrix of the end flange coordinate system of the assembly mechanism relative to the base coordinate system of the assembly mechanism. ej Let be the translation matrix of the end flange coordinate system of the assembly mechanism relative to the base coordinate system of the assembly mechanism. The two constitute the transformation relationship between the end flange coordinate system of the assembly mechanism and the base coordinate system of the assembly mechanism. r s Let t be the rotation matrix of the sensor coordinate system relative to the coordinate system of the end flange of the assembly mechanism. s Let be the translation matrix of the sensor coordinate system relative to the coordinate system of the end flange of the assembly mechanism. The two constitute the transformation relationship between the sensor coordinate system and the coordinate system of the end flange of the assembly mechanism.

[0020] Furthermore, the solution steps for step S103 are as follows:

[0021] Let r s = (r1, r2, r3), then:

[0022]

[0023] In the formula, [] T This indicates the transpose of a column vector or matrix;

[0024] Solve for y = (A) T A) -1 A T b, obtain y1, y2, ..., y 12 The value is used to calculate the transformation relationship between the sensor coordinate system and the end flange coordinate system of the assembly mechanism.

[0025]

[0026] Furthermore, the transformation relationship between the sensor coordinate system and the assembly mechanism base coordinate system is obtained.

[0027] Furthermore, a mathematical model F is used to achieve pose registration between the target workpiece and the spatial frame-like components. obj The expression is:

[0028]

[0029] In the formula, n is the number of local contour features, U is the projection matrix, F(x) is the registration pose transformation matrix between the target workpiece and the spatial frame-like component, and x is the registration pose estimation vector in the assembly mechanism coordinate system; ||·|| F q represents the Frobenius norm; i (i = 1, 2, ..., n) is the sensor coordinate system Q Si The spatial frame component contour feature information vector obtained from the theoretical assembly pose, q i (i = 1, 2, ..., n) is the sensor coordinate system Q Si The vector of spatial frame component contour features obtained under the actual assembly pose; For the sensor coordinate system Q Si The transformation matrix between the theoretical assembly pose of the target workpiece and the target workpiece. The actual assembly pose of the target workpiece and the sensor coordinate system Q Si The transformation matrix between them.

[0030] Furthermore, in step S3, for spatial frame components with a maximum length of less than 2m, the number of local contour features ranges from 6 to 8; for spatial frame components with a maximum length of 2m to 5m, the number of local contour features n ranges from 8 to 15.

[0031] Furthermore, the specific implementation steps of step S5 are as follows:

[0032] S501. Obtain multiple local contour feature information vectors of the spatial frame component in the theoretical assembly posture under the coordinate system of the assembly mechanism base, and multiple local contour feature information vectors of the spatial frame component in the actual assembly posture under the coordinate system of the assembly mechanism base.

[0033] S502. Set the initial registration pose estimation vector x0 in the coordinate system of the assembly mechanism base, and its corresponding registration pose transformation matrix is ​​T. R F(x0); T R Let R(x) be the transformation matrix between the actual registration pose of the target workpiece and the coordinate system of the assembly mechanism base, which includes the spatial rotation matrix R(x) and the spatial translation matrix T(x), and its expression is:

[0034]

[0035] In the formula, The transformation matrix between the theoretical assembly pose of the target workpiece and the coordinate system of the assembly mechanism base;

[0036] Step S503: The pose registration mathematical model F constructed in step S2 obj As the objective function, and setting a convergence tolerance ε for the objective function, a nonlinear least squares optimization algorithm is used to iteratively solve the problem, obtaining the registration pose estimation vector x between the theoretical assembly pose of the target workpiece and the actual assembly pose of the space frame-like component. s ;

[0037] Step S504: From the registration pose estimation value vector x s Determine the registration pose T in the coordinate system of the assembly mechanism base. R F(x s ).

[0038] Furthermore, in step S503, the iterative solution formula is:

[0039]

[0040] In the formula, k is the number of iterations; J k Let F be the objective function at step k. obj Jacobian matrix; h k f is the iteration step size vector for the k-th step; kf(x) is the overall matching vector of all target contour features at step k; f(x) is the overall matching vector of all target contour features, containing elements This refers to the matching vectors for the target contour features of the spatial frame-like components under the actual assembly pose.

[0041] Furthermore, the spatial frame component registration estimation method based on local contour feature information also includes step S6: the registration pose obtained in step S5 is sent to the assembly mechanism so that the assembly pose of the target workpiece grasped by the assembly mechanism is adjusted so that the assembly pose of the target workpiece matches the actual assembly pose of the spatial frame component.

[0042] Compared with existing technologies, the beneficial effects of this spatial frame component registration and estimation method based on local contour feature information include:

[0043] 1) This method is suitable for registration between large-sized target workpieces and spatial frame components. It has the advantages of small data volume (usually less than 1 Mbit) and fast processing speed. For spatial frame components with a length of less than 2m, the registration result can be obtained within 1 second. For spatial frame components with a length of 2m to 5m, the registration result can be obtained within 1.5 seconds.

[0044] 2) The registration results obtained by this method can ensure that the deviation range of the large target workpiece relative to the large spatial frame component after pose adjustment is ±2cm, and the repeatability of positioning accuracy is ≤±0.2mm, which proves that the method has both the accuracy and effectiveness of registration, as well as good robustness and consistency.

[0045] 3) In the process of estimating the registration results, the objective function F of this method is... obj The convergence tolerance ε can be freely set according to actual needs, so as to flexibly evaluate the performance of the registration pose estimation results. Under the condition of meeting the convergence tolerance ε, the assembly between the target workpiece and the space frame type component can be realized.

[0046] 4) This method is based on collecting local contour feature information vectors of several parts distributed on spatial frame-like components, so that the method has no restrictions or constraints on the specific contour shape of spatial frame-like components and has strong universality. Attached Figure Description

[0047] Figure 1 This is a flowchart of the spatial frame component registration and estimation method based on local contour feature information of the present invention;

[0048] Figure 2 This is a schematic diagram of the assembly mechanism during the specific assembly process of an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram showing the positional arrangement of the assembly mechanism, the line laser profile sensor, and the standard ball in the hand-eye calibration method of an embodiment of the present invention.

[0050] Figure 4 This is a schematic diagram illustrating the registration process between a target workpiece and a spatial frame-like component based on local contour feature information in an embodiment of the present invention.

[0051] Figure 5 This is a schematic diagram illustrating the sequence of data acquisition of various contour features of the component by the assembly mechanism in an embodiment of the present invention. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention.

[0053] like Figure 2 As shown, the assembly mechanism includes a multi-degree-of-freedom articulated robot with a vision sensor fixed on its end flange. The vision sensor is specifically a two-dimensional or three-dimensional laser vision sensor. By collecting two-dimensional pixel information and spatial point cloud information from the laser vision sensor, multiple local contour feature information vectors of the spatial frame-like components are extracted. This allows for the adjustment of the pose of the target workpiece grasped by the end of the multi-degree-of-freedom articulated robot, so that the pose of the target workpiece is adapted to the pose of the spatial frame-like components, thereby successfully completing the assembly. Figure 2 In the diagram, B represents the robot base, E represents the robot end effector flange, and S represents the vision sensor.

[0054] See Figure 1 The specific implementation steps of the spatial frame component registration and estimation method based on local contour feature information are described below.

[0055] S1. Using the hand-eye calibration method, determine the transformation relationship between the sensor coordinate system and the robot base coordinate system.

[0056] The specific implementation process of step S1 is as follows:

[0057] S101, such as Figure 3 As shown, in the calibration process, the vision sensor is first replaced with a line laser profile sensor, and the line laser profile sensor is installed on the end flange of the multi-degree-of-freedom articulated robot in the same way. A standard sphere fixed on the top of the support is placed below the line laser profile sensor, so that the line laser profile sensor can acquire the profile of the standard sphere by emitting laser light. Then, the multi-degree-of-freedom articulated robot is controlled to acquire the profile of the standard sphere in different poses, and the position of the sphere center is solved by fitting the cross-sectional circle. Since the line laser profile sensor can only directly read the coordinates of the standard sphere center on the x-axis and z-axis, the corresponding y-axis coordinate is obtained by calculation.

[0058] Based on this, the coordinates of the center of the standard sphere are X... s The expression in the sensor coordinate system is:

[0059]

[0060] In the formula, x s y s , z s These are the coordinates of the sphere's center on the x-axis, y-axis, and z-axis of the sensor coordinate system, respectively; R is the radius of the standard sphere; L is the radius of the section circle; X is the x-axis coordinate of the sphere's center read by the line laser profile sensor in the sensor coordinate system; and Z is the z-axis coordinate of the sphere's center read by the line laser profile sensor in the sensor coordinate system.

[0061] S102. Change the posture of the multi-degree-of-freedom robot to perform m measurements on a standard ball set at a fixed position using a line laser profile sensor; where the value of m is generally 12 to 16.

[0062] Specifically, let the coordinates of the center of the sphere in the robot's base coordinate system be X. b Then, in m measurements, its coordinates X in the sensor coordinate system in each measurement are... sj The relational expression satisfies:

[0063]

[0064] In the formula, r ej Let t be the rotation matrix of the robot end-effector coordinate system relative to the robot base coordinate system. ej Here, r represents the translation matrix between the robot's end-effector coordinate system and the robot's base coordinate system; both are provided by the robot system. s Let t be the rotation matrix of the sensor coordinate system relative to the robot end effector flange coordinate system. s Let be the translation matrix of the sensor coordinate system relative to the robot end flange coordinate system, and the two are unknown hand-eye relationships to be solved.

[0065] S103. Using the least squares method, the transformation relationship between the sensor coordinate system and the robot end flange coordinate system is obtained by fitting the position of the center of the sphere to the cross section circle.

[0066] Specifically, let r s = (r1, r2, r3), then we have:

[0067]

[0068] In the formula, [] T This indicates the transpose of a column vector or matrix;

[0069] Solve for y = (A) using the least squares method. T A) -1 A T b, obtain y1, y2, ..., y 12 The value is used to calculate the transformation relationship between the sensor coordinate system and the robot end flange coordinate system.

[0070]

[0071] Furthermore, the transformation relationship between the sensor coordinate system and the assembly mechanism base coordinate system is obtained.

[0072] For any measurement point P in space, its representation in the sensor coordinate system is P s Its representation in the robot base coordinate system is P. b The transformation relationship between the two is as follows:

[0073]

[0074] In the formula, The transformation relationship between the robot end flange coordinate system and the robot base coordinate system (i.e., r in step S101) is as follows. ej and t ej ); The transformation relationship between the sensor coordinate system and the robot end flange coordinate system (i.e., r in step S101) s and t s ).

[0075] In step S1, since the line laser contour sensor is mounted on the robot end flange in the same way as the vision sensor, the sensor coordinate system and its transformation relationship determined in this step are the sensor coordinate system of the vision sensor in the subsequent actual assembly process.

[0076] S2. Construct a mathematical model to achieve pose registration between the target workpiece and the spatial frame-like components, so as to describe the transformation relationship between local contour feature information and assembly pose through the model.

[0077] like Figure 4The diagram shows the registration of the spatial positions of a spatial frame component in its theoretical assembly state (solid box) and actual assembly state (dashed box). Ideally, the target workpiece and the spatial frame component are aligned. However, when the spatial frame component is transported to the assembly position, there is a spatial positional deviation between its actual assembly pose and its theoretical assembly pose. Therefore, only by obtaining the spatial positional deviation of the spatial frame component in its actual assembly pose relative to the spatial frame component in its theoretical assembly state can the pose of the target workpiece be adjusted to be aligned with the spatial frame component in its actual assembly pose, thereby achieving successful assembly.

[0078] In this embodiment, seven measurement locations are set for spatial frame-like components, such as... Figure 4 As shown in the figure, only the change of the contour feature information vector corresponding to the same measurement position of the spatial frame component under different assembly states is drawn as an illustration. That is, the transformation relationship between the contour feature information vector q1 corresponding to the measurement position S1 of the spatial frame component in the theoretical assembly state and the contour feature information vector q1' corresponding to the actual assembly position S1 of the spatial frame component. The transformation method of other measurement positions S2 to S7 is the same.

[0079] In step S2, the specific construction process of the mathematical model for achieving pose registration between the target workpiece and the spatial frame-like components is as follows:

[0080] definition:

[0081] q i (i = 1, 2, ..., n) is the sensor coordinate system Q Si The contour feature information vector of the spatial frame-like component obtained under the theoretical assembly pose; q i (i = 1, 2, ..., n) is the sensor coordinate system Q Si The contour feature information vector of the spatial frame component under the actual assembly pose is obtained; wherein, the contour feature information vector includes, but is not limited to, various measured feature points, feature vectors, feature distances, angles, curvatures, and any other comprehensive feature information that satisfies a certain relationship; in practical applications, in order to obtain better estimation results, the contour feature information vector type is multi-faceted.

[0082] For the sensor coordinate system Q Si The transformation matrix between the theoretical assembly pose of the target workpiece and the target workpiece. The actual assembly pose of the target workpiece and the sensor coordinate system Q Si The transformation matrix between them;

[0083] U is the projection matrix, whose dimensions match the contour feature information vector, so as to perform weighted calculations on each component of the vector; in practical applications, the projection matrix U is defined according to the actual situation.

[0084] Therefore, the mathematical model expression for pose registration between the target workpiece and the spatial frame-like components is:

[0085]

[0086] In the formula, n is the number of local contour features, U is the projection matrix, and ||·|| F F(x) represents the Frobenius norm; F(x) is the registration pose transformation matrix between the target workpiece and the spatial frame-like component; x is the registration pose estimation vector in the robot base coordinate system, which includes the spatial translation position vector and the spatial rotation attitude vector. The specific description form can be, but is not limited to, homogeneous transformation matrix, Euler angle, RPY angle and quaternion form.

[0087] S3. Based on the length characteristics of the spatial frame components, determine the number n of local contour features and their positions on the spatial frame components; based on the acquisition positions of the local contour features, determine the measurement pose of the vision sensor, plan the robot's motion trajectory, and obtain the sensor coordinate system Q under the theoretical assembly pose by acquiring n local contour images of the spatial frame components respectively. Si The spatial frame-like component has n contour feature information vectors q. i (i = 1, 2, ..., n).

[0088] In practical applications, the value of the number n of local contour features is determined according to the spatial frame component. Generally, for spatial frame components with a maximum length of less than 2m, the value of the number n of local contour features ranges from 6 to 8. For spatial frame components with a maximum length of 2m to 5m, the value of the number n of local contour features ranges from 8 to 15. Furthermore, the specific acquisition locations of the n local contour features are evenly distributed on the spatial frame component.

[0089] In this embodiment, as Figure 5 As shown, the maximum length of the spatial frame component is 1.5m, and the number of local contour features n is 7. Specifically, a visual sensor is used to acquire the local contour feature vectors at seven locations on the spatial frame component: positions ① and ② are located on the left border of the spatial frame component and maintain a certain distance; positions ③, ④, and ⑤ are located on the top border of the spatial frame component and maintain a certain distance; and positions ⑥ and ⑦ are located on the right border of the spatial frame component and maintain a certain distance. Furthermore, based on the seven acquisition positions determined above for obtaining the local feature information vectors, the multi-degree-of-freedom robot proceeds according to... Figure 5The measurement sequence shown is used for trajectory planning to operate a multi-degree-of-freedom robot to move to a specified position and collect two-dimensional pixel information or spatial point cloud information of the local contour of spatial frame-like components.

[0090] In this embodiment, although the measurement positions in the figure are marked as points for illustration, the visual sensor actually uses a three-dimensional laser, and the information it collects is spatial point cloud information. For the spatial point cloud information collected at each acquisition position, the spatial point cloud information is processed using existing conventional point cloud processing methods to obtain seven sets of contour feature information vectors that can be used to describe local contour features. Each set of contour feature information vectors consists of point cloud coordinates, feature vectors, feature distances, angles, and curvatures.

[0091] S4. Based on the hand-eye calibration results obtained in step S1, and the spatial frame component contour feature information vector q obtained in step S3 under the theoretical assembly pose. i (i = 1, 2, ..., 7), and based on the correspondence between the spatial frame components and the target workpiece in the theoretical assembly pose, obtain the theoretical assembly pose of the target workpiece and the coordinate system Q of the robot base. b Transformation matrix between

[0092] S5. Using the same method as in step S3, obtain multiple contour feature information vectors of the spatial frame component under the actual assembly pose. Based on the mathematical model of pose registration and the multiple contour feature information vectors of the spatial frame component under the theoretical assembly pose, use a nonlinear optimization algorithm to solve for the registration pose estimation vector of the target workpiece and the spatial frame component under the actual pose, so as to obtain the registration pose of the target workpiece.

[0093] See Figure 5 In this embodiment, the specific implementation steps of step S5 are as follows:

[0094] S501. Based on the hand-eye calibration matrix obtained in step S1 and the sensor coordinate system Q obtained from step S3 Si The contour feature information vector q of the spatial frame-like components obtained under the theoretical assembly pose. i The robot base coordinate system Q is obtained. b Local contour feature information vector of spatial frame-like components under theoretical assembly pose

[0095]

[0096] Using the same method as in step S3, obtain the contour feature information vector q of the spatial frame-like component in the actual assembly pose. i(i = 1, 2, ..., 7), and based on the hand-eye calibration matrix obtained in step S1 Obtain the robot base coordinate system Q b Vector of local contour feature information of spatial frame components under actual assembly pose

[0097] In the actual processing, only the point cloud coordinates in the contour feature information vector change during this step; the other vector information that does not change due to the change in coordinate system remains unchanged.

[0098] S502, Set the robot base coordinate system Q b The initial registration pose estimation vector x0 has a corresponding registration pose transformation matrix T. R F(x0); where T R The transformation matrix between the actual registration pose of the target workpiece and the robot base coordinate system includes the spatial rotation matrix R(x) and the spatial translation matrix T(x), and its expression is:

[0099]

[0100] Step S503: The pose registration mathematical model F constructed in step S2 obj As the objective function, and setting a convergence tolerance ε for the objective function, a nonlinear least squares optimization algorithm is used to iteratively solve the problem, obtaining the registration pose estimation vector x between the theoretical assembly pose of the target workpiece and the actual assembly pose of the space frame-like component. s ;

[0101] The iterative solution formula is as follows:

[0102]

[0103] In the formula, k is the number of iterations; J k Let F be the objective function at step k. obj Jacobian matrix; h k f is the iteration step size vector for the k-th step; k f(x) is the overall matching vector of all target contour features at step k; f(x) is the overall matching vector of all target contour features, containing elements This refers to the matching vectors for the target contour features of the spatial frame-like components under the actual assembly pose. Specifically, the input obtained from step S501 is substituted into... and

[0104] Step S504: Based on the registration pose estimation vector x that satisfies the allowable value ε obtained in step S503. s The robot base coordinate system Q is obtained.b Registration pose T R F(x s ).

[0105] S6. The registration pose obtained in step S5 is sent to the robot for the target workpiece so that the multi-degree-of-freedom robot can grasp and adjust the pose of the target workpiece so that the pose of the target workpiece is adapted to the assembly pose of the current spatial frame component, thereby enabling the multi-degree-of-freedom robot to successfully complete the assembly task of the target workpiece and the spatial frame component.

[0106] In actual operation, the target workpiece is placed on the rack, and the multi-degree-of-freedom articulated robot grasps the target workpiece using a gripper or suction cup mounted on the end flange, and then performs the registration pose T obtained by this method. R F(x s The target workpiece is adjusted to the actual assembly pose that matches the actual spatial frame component pose, and then subsequent assembly actions are performed to complete the assembly between the target workpiece and the spatial frame component.

[0107] Furthermore, in order to demonstrate the effectiveness and accuracy of the spatial frame component registration estimation method based on local contour feature information in actual assembly operations, the registration estimation method of this embodiment is applied to the assembly of actual target workpieces and spatial frame components to test its practical application effect.

[0108] In practical application testing, the spatial frame component is a car windshield mounting frame, and the target workpiece is the windshield. The maximum length of the car windshield mounting frame is 1.5m and the maximum width is 0.9m, while the maximum length of the windshield is 1.4m and the maximum width is 0.8m. Based on the actual length of the car windshield mounting frame, seven measurement positions are set on it, and seven sets of local contour feature information vectors are obtained using the method of this application.

[0109] Based on the actual assembly situation, the method of this application ensures that the center position deviation of the windshield mounting frame relative to the theoretical assembly pose is within ±2cm. This means that, under the premise that the vision sensor can effectively acquire the seven sets of local contour feature information vectors on the windshield mounting frame, the windshield, after pose adjustment, can achieve an assembly deviation of ±3mm from the current windshield mounting frame. Compared to existing large-size workpiece registration methods, this method demonstrates better registration performance, proving the accuracy of the result estimation in actual pose registration applications. Furthermore, with the windshield mounting frame assembly pose unchanged, the repeatability accuracy is ≤±0.2mm, exhibiting good robustness and consistency. This demonstrates that the method of this application effectively ensures the stability and consistency of the measurement results through the acquisition of multi-dimensional feature information. As a comparison, when the types of local feature information vectors in each group are gradually reduced, especially when a single local feature information vector containing point cloud information is used, the final estimation result is affected by surrounding environmental factors or system noise, and the registration estimation effect gradually deteriorates. Not only does the deviation range between the windshield and the car windshield mounting frame increase after the pose adjustment, but the repeatable positioning accuracy also decreases by multiples to ±0.5mm.

[0110] The applicant also needs to emphasize that the point cloud data volume is relatively small during the data processing using the method of this application, and is generally less than 1 Mbit. At the same time, the method of this application does not require deep learning when extracting features. Therefore, in this embodiment, the registration pose calculation process can be completed within 1 second, which is time-saving and efficient. In addition, the method is based on collecting local contour feature information vectors of several parts distributed on spatial frame-like components. There are no restrictions or constraints on the specific contour shape of the spatial frame-like components. It has strong universality and good prospects for industrial application and promotion.

[0111] In summary, the above-disclosed embodiments are merely illustrative of the present invention. The embodiments do not describe all details in detail, nor do they limit the invention to the specific implementations described. Any modifications and substitutions made by those skilled in the art within the scope of this invention should be included within the protection scope of this invention.

Claims

1. A spatial frame class component registration estimation method based on local contour feature information, characterized in that, The steps are as follows: S1, a hand-eye calibration method is used to determine the transformation relationship between the sensor coordinate system in the assembly mechanism and the base coordinate system of the assembly mechanism; S2, construct a mathematical model for realizing pose registration between the target workpiece and the space frame class component; the mathematical model F for realizing pose registration between the target workpiece and the space frame class component obj The expression is: ， In the formula, n is the number of local contour features, U is the projection matrix, F(x) is the registration pose transformation matrix between the target workpiece and the spatial frame-like component, and x is the registration pose estimation vector in the assembly mechanism coordinate system; ||·|| F q represents the Frobenius norm; i (i=1, 2, …, n) is the sensor coordinate system Q Si The spatial frame component contour feature information vector obtained from the theoretical assembly pose, q' i (i=1, 2, …, n) is the sensor coordinate system Q Si The vector of spatial frame component contour features obtained under the actual assembly pose; For the sensor coordinate system Q Si The transformation matrix between the theoretical assembly pose of the target workpiece and the target workpiece. The actual assembly pose of the target workpiece and the sensor coordinate system Q Si The transformation matrix between them; S3, according to the length characteristics of the space frame type component, the number of local contour features and their positions on the space frame type component are determined; through the visual sensor in the assembly mechanism, a plurality of contour feature information vectors of the space frame type component in the theoretical assembly pose in the sensor coordinate system are obtained; S4, according to the correspondence between the space frame type component and the target workpiece in the theoretical assembly pose, a transformation matrix between the theoretical assembly pose of the target workpiece and the base coordinate system of the assembly mechanism is obtained; S5, the same method as step S3 is used to obtain a plurality of contour feature information vectors of the space frame type component in the actual assembly pose, and based on the mathematical model of pose registration and the plurality of contour feature information vectors of the space frame type component in the theoretical assembly pose, a nonlinear optimization algorithm is used to solve the registration pose estimation vector of the target workpiece and the space frame type component in the actual pose, so as to obtain the registration pose of the target workpiece.

2. The method of claim 1, wherein, The specific implementation steps of step S1 are as follows: S101, the visual sensor in the assembly mechanism is replaced by a line laser profile sensor in the same installation mode, and a standard ball is placed below the line laser profile sensor; S102, the attitude of the assembly mechanism is changed to measure the standard ball arranged at a fixed position multiple times through the line laser profile sensor, and a relationship between the coordinates of the ball center in the base coordinate system of the assembly mechanism and the coordinates of the sensor in each measurement is obtained; S103, based on the relationship obtained in step S102, the least square method is used to solve the ball center position by fitting the cross-sectional circle, and the conversion relationship of the sensor coordinate system relative to the end flange coordinate system of the assembly mechanism is obtained.

3. The method of claim 2, wherein, In step S102, the coordinates of the ball center in the coordinate system of the assembly mechanism base are X b The relationship between the coordinates X sj in the coordinate system of the sensor in each measurement satisfies: ， wherein r ej is a rotation matrix of the end flange coordinate system of the assembly mechanism relative to the base coordinate system of the assembly mechanism, t ej is a translation matrix of the end flange coordinate system of the assembly mechanism relative to the base coordinate system of the assembly mechanism, both constituting a transformation relationship of the end flange coordinate system of the assembly mechanism relative to the base coordinate system of the assembly mechanism ; r s is a rotation matrix of the sensor coordinate system relative to the end flange coordinate system of the assembly mechanism, t s is a translation matrix of the sensor coordinate system relative to the end flange coordinate system of the assembly mechanism, both constituting a transformation relationship of the sensor coordinate system relative to the end flange coordinate system of the assembly mechanism .

4. The method of claim 2, wherein, The solving steps of step S103 are as follows: Let r s = (r1, r2, r3), then: ， wherein [] T denotes the transpose of a column vector or matrix; Solve y = (A T A) -1 A T b, get y1, y2, …, y 12 The value is calculated to obtain the conversion relationship of the sensor coordinate system relative to the assembly mechanism end flange coordinate system : ， Further, a transformation relationship between the sensor coordinate system and the base coordinate system of the assembly mechanism is obtained .

5. The method of claim 1, wherein, In step S3, for the space frame type component with a maximum length less than 2m, the number of local contour features is in the range of 6-8; for the space frame type component with a maximum length of 2m to 5m, the number n of local contour features is in the range of 8-15.

6. The method of claim 1, wherein, The specific implementation steps of step S5 are as follows: S501, a plurality of local contour feature information vectors of the space frame type component in the theoretical assembly pose in the base coordinate system of the assembly mechanism are obtained, and a plurality of local contour feature information vectors of the space frame type component in the actual assembly pose in the base coordinate system of the assembly mechanism are obtained; S502, set the initial registration pose estimation value vector x0 in the assembly mechanism base coordinate system, and the corresponding registration pose transformation matrix is T R F(x0);T R is the transformation matrix between the actual registration pose of the target workpiece and the assembly mechanism base coordinate system, which includes a spatial rotation matrix R(x) and a spatial translation matrix T(x), and its expression is: ， In the formula, is the transformation matrix between the theoretical assembly pose of the target workpiece and the base coordinate system of the assembly mechanism; Step S503, constructing the pose registration mathematical model F with step S2 obj As the objective function, and setting the convergence tolerance value ε of the objective function, using the nonlinear least squares optimization algorithm to solve iteratively, get the registration pose estimation value vector x between the target workpiece theoretical assembly pose and the actual assembly pose of the space frame component s ; Step S504, determining the registration pose T s in the coordinate system of the assembly mechanism base from the registration pose estimation value vector x R F(x s ).

7. The method of claim 6, wherein, In step S503, the iterative solving formula is: ， where k is the iteration step number; J k is the Jacobian matrix of the objective function F obj at the kth step; h k is the iteration step length vector at the kth step; is the overall matching vector of all target profile features at the kth step; f(x) is the overall matching vector of all target profile features, containing elements (i = 1, 2, …, 7) is the matching vector of each target profile feature of the spatial frame-like member under the actual assembly pose, .​ 8. The method of claim 1, wherein, It also includes step S6: the registration pose obtained by step S5 is transmitted to the assembly mechanism to adjust the assembly pose of the target workpiece grasped by the assembly mechanism, so that the assembly pose of the target workpiece matches the actual assembly pose of the space frame type component.

Images