Target pose recognition method, device and medium of matching butt joint assembly system

Abstract

This application relates to a target pose recognition method, equipment, and medium for docking and assembly systems, specifically within the field of docking system control. The aim of this application is to address the problems of insufficient stability, high computational and data preparation requirements, and weak engineering interpretability in existing docking alignment methods. The application involves acquiring raw point clouds containing target and environmental information during the docking state of the docking and assembly system, processing these raw point clouds into target candidate point clouds, transforming these candidate point clouds to the docking and assembly system coordinate system, using these candidate point clouds in the docking and assembly system coordinate system to select high-resolution templates from a set of template point clouds, performing ICP-type iterative registration on the high-resolution templates to obtain the optimal rigid body transformation matrix, and superimposing the optimal rigid body transformation matrix with the pose of a reference template in the set of template point clouds to obtain the final pose of the target in the docking and assembly system coordinate system.

Description

Technical Field

[0001] This application belongs to the field of docking system control, and in particular relates to target pose recognition. Background Technology

[0002] In large-scale equipment assembly, module docking, and engine docking, a composite motion architecture of "coarse adjustment + fine adjustment" is often used to achieve component docking. Among these, a six-degree-of-freedom attitude adjustment platform (such as a parallel attitude adjustment mechanism) undertakes the fine-tuning of the pose during the fine docking phase. Traditional docking alignment methods often rely on manual experience or contact measurement, resulting in low efficiency, poor repeatability, and strong dependence on operators. Two-dimensional vision lacks stability under conditions of occlusion, strong reflection, or weak texture; deep learning methods have high requirements for computing power and data preparation and weak engineering interpretability. Therefore, a target pose recognition method is needed that is suitable for industrial docking scenarios, can be implemented on an industrial control computer platform, has a low hardware and algorithm burden, and is robust. Summary of the Invention

[0003] This application aims to address the problems of insufficient stability, high requirements for computing power and data preparation, and weak engineering interpretability of existing docking alignment methods. It provides a target pose recognition method, equipment, and medium for matching a six-degree-of-freedom flexible docking system, so as to improve the stability and repeatability of target pose measurement in docking assembly and reduce the dependence on manual operation and high-computing AI models.

[0004] The first aspect of this application provides a target pose recognition method for a matching and docking assembly system, including:

[0005] In the docking state of the docking assembly system, the original point cloud containing target and environmental information is collected, and the original point cloud is processed into target candidate point cloud;

[0006] Transform the target candidate point cloud into the coordinate system of the docking and assembly system;

[0007] High-scoring templates are selected from the template point cloud set using the target candidate point cloud in the coordinate system of the docking assembly system.

[0008] ICP-type iterative registration is performed on the high-resolution template to obtain the optimal rigid body transformation matrix;

[0009] The optimal rigid body transformation matrix is ​​superimposed with the reference template pose in the template point cloud set to obtain the final pose of the target in the coordinate system of the docking assembly system.

[0010] In one possible design, the target pose recognition method for the above-mentioned matching and docking assembly system also includes:

[0011] After obtaining the final pose, the corrected motion of the matching docking assembly system is calculated based on the difference between the target docking pose and the final pose. The docking of the docking assembly system is completed based on the corrected motion. When the matching quality is lower than its threshold, the target pose recognition is performed again.

[0012] The matching quality includes at least one or more of the root mean square error and the proportion of interior points.

[0013] In one possible design, processing the original point cloud into a target candidate point cloud includes:

[0014] The original point cloud is subjected to ROI cropping, filtering and denoising, voxel downsampling and outlier removal to obtain the target candidate point cloud.

[0015] In one possible design, the ROI trimming includes:

[0016] Based on the geometric constraints of the docking assembly system and the prior position of the target, a three-dimensional space window or polyhedral clipping area is set, and clipping is performed using the three-dimensional space window or polyhedral clipping area;

[0017] The three-dimensional space window or polyhedral clipping area retains only the point cloud of the spatial region within the constraints of the docking assembly system.

[0018] In one possible design, the outlier removal includes:

[0019] Calculate the distance from each point in the point cloud to its corresponding point. The average distance of each nearest neighbor is calculated, and the mean of all average distances is calculated. Points with an average distance greater than the mean are then removed.

[0020] In one possible design, the step of using the target candidate point cloud in the coordinate system of the docking assembly system to filter high-scoring templates from the template point cloud set includes:

[0021] The initial pose is obtained using the kinematic priors of the docking and assembly system.

[0022] Based on the target candidate point cloud in the coordinate system of the docking assembly system, a finite step size attitude search is performed on the template point cloud set to obtain high-scoring templates.

[0023] In one possible design, the method for obtaining the template point cloud set includes: generating the template point cloud set offline based on a target CAD model or scanned point cloud;

[0024] The template point cloud set can cover all attitude ranges in the docking state, and each template records its reference pose and scale information in the target coordinate system.

[0025] In one possible design, the optimal rigid body transformation matrix is ​​superimposed on the reference template pose in the template point cloud set by the following formula:

[0026] ,

[0027] in, For the final pose, The optimal rigid body transformation matrix. The pose is the reference template.

[0028] The second aspect of this application provides a target pose recognition device for a matching and docking assembly system. The target pose recognition device for the matching and docking assembly system includes a processor and a memory. The memory stores at least one instruction, which is loaded and executed by the processor to implement the target pose recognition method for the matching and docking assembly system as described above.

[0029] A third aspect of this application provides a computer storage medium storing at least one instruction, which is loaded and executed by a processor to implement the target pose recognition method of the matching and docking assembly system described above.

[0030] The beneficial effects of this application are:

[0031] (1) Based on 3D point cloud and template matching, more stable pose estimation can be obtained in reflective, dark or weak texture workpiece scenes;

[0032] (2) It does not rely on manual application of markers and is suitable for rapid deployment on engineering sites;

[0033] (3) It does not involve deep learning training, the computing chain can run on a conventional industrial control computer platform, and the engineering maintainability is good;

[0034] (4) After the coordinate system is aligned with the six-degree-of-freedom attitude adjustment platform, the pose matrix can be directly output for docking closed-loop control. Attached Figure Description

[0035] Figure 1 A schematic diagram of the overall structure of a six-degree-of-freedom flexible docking assembly system;

[0036] Figure 2 A schematic diagram illustrating the relationship and coordinate transformation between the camera coordinate system, the target coordinate system, and the docking reference coordinate system;

[0037] Figure 3 A flowchart for a target pose recognition method for a six-degree-of-freedom flexible docking system. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0039] Specific Implementation Method 1: The target pose recognition method for the matching and docking assembly system described in this implementation method includes:

[0040] In the docking state of the docking assembly system, the original point cloud containing target and environmental information is collected, and the original point cloud is processed into target candidate point cloud;

[0041] Transform the target candidate point cloud into the coordinate system of the docking and assembly system;

[0042] High-scoring templates are selected from the template point cloud set using the target candidate point cloud in the coordinate system of the docking assembly system.

[0043] ICP-type iterative registration is performed on the high-resolution template to obtain the optimal rigid body transformation matrix;

[0044] The optimal rigid body transformation matrix is ​​superimposed with the reference template pose in the template point cloud set to obtain the final pose of the target in the coordinate system of the docking assembly system.

[0045] In one embodiment, the target pose recognition method of the matching and docking assembly system further includes:

[0046] After obtaining the final pose, the corrected motion of the matching docking assembly system is calculated based on the difference between the target docking pose and the final pose. The docking of the docking assembly system is completed based on the corrected motion. When the matching quality is lower than its threshold, the target pose recognition is performed again.

[0047] The matching quality includes at least one or more of the root mean square error and the proportion of interior points.

[0048] In one implementation, processing the original point cloud into a target candidate point cloud includes:

[0049] The original point cloud is subjected to ROI cropping, filtering and denoising, voxel downsampling and outlier removal to obtain the target candidate point cloud.

[0050] In one implementation, the ROI trimming includes:

[0051] Based on the geometric constraints of the docking assembly system and the prior position of the target, a three-dimensional space window or polyhedral clipping area is set, and clipping is performed using the three-dimensional space window or polyhedral clipping area;

[0052] The three-dimensional space window or polyhedral clipping area retains only the point cloud of the spatial region within the constraints of the docking assembly system.

[0053] In one implementation, the outlier removal includes:

[0054] Calculate the distance from each point in the point cloud to its corresponding point. The average distance of each nearest neighbor is calculated, and the mean of all average distances is calculated. Points with an average distance greater than the mean are then removed.

[0055] In one embodiment, the step of selecting high-scoring templates from the template point cloud set using the target candidate point cloud in the coordinate system of the docking assembly system includes:

[0056] The initial pose is obtained using the kinematic priors of the docking and assembly system.

[0057] Based on the target candidate point cloud in the coordinate system of the docking assembly system, a finite step size attitude search is performed on the template point cloud set to obtain high-scoring templates.

[0058] In one embodiment, the method for obtaining the template point cloud set includes: generating the template point cloud set offline based on a target CAD model or scanned point cloud;

[0059] The template point cloud set can cover all attitude ranges in the docking state, and each template records its reference pose and scale information in the target coordinate system.

[0060] In one implementation, the optimal rigid body transformation matrix is ​​superimposed on the reference template pose in the template point cloud set using the following formula:

[0061] ,

[0062] in, For the final pose, The optimal rigid body transformation matrix. The pose is the reference template.

[0063] To further illustrate the implementation scheme of this application, Figure 3 A target pose recognition method for a six-DOF flexible docking system is provided, comprising steps 1 to 8. The numbering of these steps does not necessarily restrict their execution order. Each step is described in detail below:

[0064] This embodiment takes a manual / automatic compatible six-degree-of-freedom flexible docking assembly system as the object. The assembly system includes: an omnidirectional moving chassis (1), two sets of identical two-degree-of-freedom lifting and gripping units (first unit 2-1 and second unit 2-2), and a six-degree-of-freedom attitude adjustment platform unit (3). Among them, the omnidirectional moving chassis (1) includes: a steering wheel, a frame, a control cabinet, guide rails and racks, limit sensors and mechanical limit structures; the omnidirectional moving chassis (1) is used to realize large-stroke omnidirectional movement and positioning during the rough docking stage. The two-degree-of-freedom lifting and gripping unit includes: an X-axis adjustment device, a Z-axis adjustment device, a rigid-flexible conversion device and a handwheel device; the two-degree-of-freedom lifting and gripping unit is used for differential adjustment of the Z-axis height and pitch of the product section, and can switch between rigid connection and flexible connection through the rigid-flexible conversion device. The six-degree-of-freedom attitude adjustment platform unit (3) includes: product support bracket, parallel attitude adjustment mechanism, control cabinet and track transfer vehicle. The parallel attitude adjustment mechanism may include: top adapter plate, upper hinge seat, force sensor, servo electric cylinder, lower hinge seat and bottom platform and other components, which are used for multi-degree-of-freedom precision attitude adjustment and long-term load support during the precision docking stage. It can also move along the track via the track transfer vehicle to achieve six-degree-of-freedom precision attitude adjustment.

[0065] S1. Camera Installation and Connection: Install the industrial 3D vision sensor on the fixed bracket and motion platform of the docking system, and establish a data connection with the industrial control computer.

[0066] The visual recognition unit in this embodiment includes: a 3D camera, a mounting bracket, an industrial control computer (including an industrial gigabit network card), and power supply and cable assemblies. The 3D camera preferably uses an industrial 3D vision sensor, which projects a laser beam and acquires structured light images via a binocular camera, then calculates a 3D contour point cloud using algorithms. The 3D camera can be fixedly installed (grounded via a bracket) or mobile installed (grounded via a robotic arm / platform), and communicates with the industrial control computer via a data interface to transmit images, measurements, and 3D information, and receive external control commands to adjust acquisition parameters. The industrial control computer configuration should meet the real-time requirements of point cloud acquisition and processing. A gigabit Ethernet connection is used between the 3D camera and the industrial control computer; an industrial gigabit network card is used to obtain more stable data transmission. A giant frame mode is enabled on the network side to reduce the system overhead of high-bandwidth point cloud transmission. The installation and wiring of the 3D camera must comply with electromagnetic interference protection and grounding requirements to avoid strong interference sources and excessive cable bending that could lead to data instability.

[0067] S2. Camera Calibration: Collect calibration data and obtain camera measurement model parameters and the external parameter transformation matrix of the camera relative to the docking system reference coordinate system.

[0068] The purpose of camera calibration and coordinate system establishment is to use the recognition results for docking control; therefore, a unified coordinate system needs to be established. For example... Figure 2 As shown, the camera coordinate system is defined as follows: Docking system reference coordinate system (Use the six-degree-of-freedom attitude adjustment platform base or vehicle frame as a reference), target coordinate system (The origin is the target design reference point / feature center).

[0069] Camera calibration includes two types of parameters:

[0070] (1) Camera measurement model parameters, used to convert sensor observations into point clouds;

[0071] (2) Camera extrinsic transformation matrix It is a 4×4 homogeneous transformation matrix, including rotations. With translation Used to establish the camera coordinate system Point cloud transformation to docking system reference coordinate system This transformation enables the conversion of target candidate point clouds. It is consistent with the control coordinates of the six-degree-of-freedom platform, providing a consistent reference for subsequent pose output and motion control.

[0072] During implementation, the manufacturer's calibration function and calibration fixtures or equivalent geometric calibration parts are used to collect multi-pose calibration data within the camera's field of view, and the calibration solution is obtained using the accompanying 3D vision application software. .

[0073] S3, Point Cloud Acquisition: Acquire the original point cloud containing the target.

[0074] During or before docking, the camera is triggered to acquire raw point cloud data containing target and environmental information. (coordinate system) ).

[0075] S4. Point cloud preprocessing: preprocessing the raw point cloud. Perform ROI cropping, filtering and denoising, voxel downsampling, and outlier removal to obtain the target candidate point cloud. .

[0076] Let the original point cloud be... any point in the middle In camera coordinate system The three-dimensional coordinates below.

[0077] (a) ROI clipping: Based on the geometric constraints of the docking system and the prior position of the target, a three-dimensional spatial window or polyhedron clipping area is set to retain only the point cloud of the spatial region within the constraints of the docking system to suppress background interference.

[0078] Given the axis-aligned bounding box of the docking region: .

[0079] The cropped point cloud is as follows: .

[0080] (b) Voxel downsampling: The point cloud is downsampled according to the voxel size, and the voxel representative points are retained to reduce the number of points and suppress local noise, thereby improving the matching efficiency.

[0081] Given voxel side length The space is divided into voxel grids; voxel indexing .

[0082] For each non-empty voxel Take the centroid as the representative point: , The downsampled point cloud was obtained. .

[0083] (c) Outlier removal: Outliers are removed using statistical filtering or radius filtering.

[0084] downsampled point cloud For each point in the array, calculate its distance to... Average distance of the nearest neighbors Statistics of all mean with standard deviation Set threshold and ( (Empirical coefficient), eliminating those that meet the criteria. The points are used to suppress scattered noise and measurement burrs.

[0085] (d) Optional smoothing: Smooth the point cloud to reduce measurement jitter.

[0086] The above processing yields a target candidate point cloud for matching. Its reduced number of points and more stable geometry are beneficial for improving the convergence and repeatability of template matching / registration.

[0087] S5. Coordinate Transformation: Transform the target candidate point cloud From the camera coordinate system Transform to docking system reference coordinate system .

[0088] For the camera coordinate system points within Converted to the docking system reference coordinate system points within .

[0089] S6. Create and load the target point cloud template library.

[0090] Based on target CAD models or high-quality scanned point clouds, generate template point cloud sets offline. Templates can be generated by discrete sampling at different observation distances and azimuth angles to cover the possible attitude range during docking. Each template records its position in the target coordinate system. The system provides baseline pose and scale information. The template library can be generated and stored once during system deployment, and candidate template sets can be loaded on demand during online recognition to reduce computational load.

[0091] S7. Template matching and registration for pose determination: Solving the pose of the point cloud after coordinate transformation. With template point cloud collection Perform coarse and fine registration, using matching error or overlap score as the optimization objective, solve for the optimal rigid body transformation matrix, and obtain the target pose. .

[0092] (1) Coarse registration: The initial pose is provided by utilizing the kinematic priors of the docking fixture (current readings of the six-DOF platform and geometric constraints of the docking slot); the template point cloud set is then used for registration. Perform a pose search with a finite step size to quickly filter out several high-scoring templates.

[0093] (2) Fine registration: Perform ICP (Iterative Closest Point Algorithm) type iterative registration on the selected high-scoring templates to minimize the point-to-point or point-to-surface distance error and output the optimal rigid body transformation matrix. Using a reference template pose from the template library. Starting from this point, the optimal rigid body transformation matrix is... pose of the reference template By superimposing the coordinates, the target in the docking system reference coordinate system can be obtained. Final position .

[0094] The final pose can be further converted into Euler angles. or quaternion This is for use by the docking controller. It also outputs matching quality metrics (such as root mean square error (RMSE), interior point ratio, etc.) to assess the reliability of the docking control.

[0095] S8, Pose Output and Dock Control Interface: Final Pose And matching quality indicators for closed-loop docking control of the six-degree-of-freedom attitude adjustment platform.

[0096] Final pose Decompose into translation vectors The attitude (Euler angles or quaternions) is output to the docking controller. The controller outputs the target docking pose. and final pose The differences are used to calculate the corrected motion of the six-degree-of-freedom platform, driving the parallel attitude adjustment mechanism and product support unit to complete the fine docking adjustment. When the matching quality is below the threshold, re-acquisition, expansion of ROI, or switching of candidate template set can be triggered to improve recognition robustness. Matching quality indicators include root mean square error (RMSE) and / or interior point ratio, used to determine whether the recognition results can be used for docking closed-loop control.

[0097] This embodiment discloses a target pose recognition method for a six-degree-of-freedom flexible docking assembly system, addressing the spatial six-degree-of-freedom pose measurement requirements of target components in docking assembly. The method employs an industrial 3D vision sensor to acquire a 3D point cloud of the docking target. On an industrial computer, point cloud cropping, filtering and denoising, coordinate system transformation, and matching and registration based on a point cloud template library are performed. The output is the pose matrix of the target relative to the docking system's reference coordinate system, which is then provided to a six-degree-of-freedom pose adjustment platform for closed-loop docking control. This method does not involve deep learning training and is suitable for achieving stable recognition under complex working conditions.

[0098] Specific Implementation Method Two: The target pose recognition device of the matching docking assembly system described in this embodiment includes a processor and a memory. The memory stores at least one instruction, which is loaded and executed by the processor to implement the target pose recognition method of the matching docking assembly system as described in Specific Implementation Method One.

[0099] Specific Implementation Method 3: A computer storage medium as described in this embodiment stores at least one instruction, which is loaded and executed by a processor to implement the target pose recognition method of the matching and docking assembly system as described in Specific Implementation Method 1.

[0100] While specific embodiments of this application have been described herein with reference to them, it should be understood that these embodiments are merely examples of the principles and applications of this application. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of this application as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A target pose recognition method for a matching and docking assembly system, characterized in that, include: In the docking state of the docking assembly system, the original point cloud containing target and environmental information is collected, and the original point cloud is processed into target candidate point cloud; Transform the target candidate point cloud into the coordinate system of the docking and assembly system; High-scoring templates are selected from the template point cloud set using the target candidate point cloud in the coordinate system of the docking assembly system. ICP-type iterative registration is performed on the high-resolution template to obtain the optimal rigid body transformation matrix; The optimal rigid body transformation matrix is ​​superimposed with the reference template pose in the template point cloud set to obtain the final pose of the target in the coordinate system of the docking assembly system.

2. The target pose recognition method for the matching and docking assembly system according to claim 1, characterized in that, Also includes: After obtaining the final pose, the corrected motion of the matching docking assembly system is calculated based on the difference between the target docking pose and the final pose. The docking of the docking assembly system is completed based on the corrected motion. When the matching quality is lower than its threshold, the target pose recognition is re-performed. The matching quality includes at least one or more of the root mean square error and the proportion of interior points.

3. The target pose recognition method for the matching and docking assembly system according to claim 1 or 2, characterized in that, The step of processing the original point cloud into a target candidate point cloud includes: The original point cloud is subjected to ROI cropping, filtering and denoising, voxel downsampling and outlier removal to obtain the target candidate point cloud.

4. The target pose recognition method for the matching and docking assembly system according to claim 3, characterized in that, The ROI clipping includes: Based on the geometric constraints of the docking assembly system and the prior position of the target, a three-dimensional space window or polyhedral clipping area is set, and clipping is performed using the three-dimensional space window or polyhedral clipping area; The three-dimensional space window or polyhedral clipping area retains only the point cloud of the spatial region within the constraints of the docking assembly system.

5. The target pose recognition method for the matching and docking assembly system according to claim 3, characterized in that, The outlier removal includes: Calculate the distance from each point in the point cloud to its corresponding point. The average distance of each nearest neighbor is calculated, and the mean of all average distances is calculated. Points with an average distance greater than the mean are then removed.

6. The target pose recognition method for the matching and docking assembly system according to claim 1 or 2, characterized in that, The step of using the target candidate point cloud in the coordinate system of the docking assembly system to filter high-scoring templates from the template point cloud set includes: The initial pose is obtained using the kinematic priors of the docking and assembly system. Based on the target candidate point cloud in the coordinate system of the docking assembly system, a finite step size attitude search is performed on the template point cloud set to obtain high-scoring templates.

7. The target pose recognition method for the matching and docking assembly system according to claim 4, characterized in that, The method for obtaining the template point cloud set includes: generating the template point cloud set offline based on a target CAD model or scanned point cloud; The template point cloud set can cover all attitude ranges in the docking state, and each template records its reference pose and scale information in the target coordinate system.

8. The target pose recognition method for the matching and docking assembly system according to claim 1 or 2, characterized in that, The optimal rigid body transformation matrix is ​​superimposed on the reference template pose in the template point cloud set using the following formula: ， in, For the final pose, The optimal rigid body transformation matrix. The pose is the reference template.

9. A target pose recognition device for a matching and docking assembly system, characterized in that, The target pose recognition device of the matching and docking assembly system includes a processor and a memory. The memory stores at least one instruction, which is loaded and executed by the processor to implement the target pose recognition method of the matching and docking assembly system as described in any one of claims 1 to 8.

10. A computer storage medium, characterized in that, The computer storage medium stores at least one instruction, which is loaded and executed by a processor to implement the target pose recognition method of the matching docking assembly system as described in any one of claims 1 to 8.

Images