Collaborative assembly methods, media, and equipment based on point cloud registration

Abstract

This invention provides a collaborative assembly method, medium, and device based on point cloud registration. It determines the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration grasping pose of the component to be installed, and the calibration bottom point cloud and calibration installation pose of the component to be installed. A first installation correction pose is obtained based on the calibration point cloud and the current point cloud of the mounting plate. A corrected actual grasping pose is obtained based on the calibration surface point cloud and the current point cloud of the component to be installed. The component to be installed is grasped based on the actual grasping pose, and the bottom scene point cloud of the component to be installed is acquired in the grasping scenario. The calibration installation pose is corrected based on the bottom scene point cloud and the calibration bottom point cloud to obtain a second installation correction pose. The first and second installation correction poses are combined to obtain the final installation pose of the component to be installed. This solves the problems of low assembly accuracy and poor adaptability in existing technologies.

Description

Technical Field

[0001] This invention relates to the field of automated production and assembly technology, and in particular to a collaborative assembly method, medium, and equipment based on point cloud registration. Background Technology

[0002] In traditional industrial assembly processes, robotics is primarily used to perform single or limited assembly tasks. These single-robot systems are typically designed to handle standardized, repetitive tasks, such as simple part picking and placing. In these applications, single-robot systems offer high operational efficiency and low labor costs. However, as industrial products become increasingly complex, the demands for assembly precision and flexibility also rise. In high-precision assembly tasks involving multiple parts and multiple steps, single-robot systems often struggle to meet production needs. Due to their singular operational nature and limitations in path planning, they often require significantly more time to complete the entire assembly process when handling multi-part, multi-step assembly tasks. This not only limits the overall efficiency of the production line but also increases production costs. In high-precision assembly tasks, a single robot system may struggle to control multiple operation points and paths simultaneously, leading to decreased assembly accuracy and affecting the quality of the final product. Furthermore, in the face of changing production demands and product upgrades, single robot systems often require reprogramming and adjustments, which is not only time-consuming and labor-intensive but also limits the rapid response capability of the production line. In scenarios where multiple robots need to collaborate to complete complex assembly tasks, single robot systems lack effective collaboration mechanisms, making it difficult to achieve effective communication and coordination among multiple robots, thus failing to fully realize the potential of multi-robot systems.

[0003] Therefore, how to improve it is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0004] Based on this, the purpose of this application is to provide a collaborative assembly method, medium, and device based on point cloud registration to solve at least one of the technical problems mentioned in the background art.

[0005] Firstly, this application provides a collaborative assembly method based on point cloud registration, comprising:

[0006] Determine the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration gripping pose of the part to be installed, and the calibration bottom point cloud and calibration installation pose of the part to be installed.

[0007] Obtain the current point cloud of the mounting plate, and correct the calibration placement pose based on the calibration point cloud and the current point cloud of the mounting plate to obtain the first installation correction pose;

[0008] Obtain the current point cloud of the part to be installed, and correct the calibrated grasping pose based on the calibration surface point cloud of the part to be installed and the current point cloud of the part to be installed to obtain the corrected actual grasping pose.

[0009] Based on the actual grasping pose, grasp the component to be installed and obtain the bottom scene point cloud of the component to be installed in the grasping scene;

[0010] The calibration installation pose is corrected based on the bottom scene point cloud and the calibration bottom point cloud to obtain the second installation correction pose.

[0011] By combining the first and second installation correction poses, the final installation pose of the component to be installed is obtained.

[0012] Furthermore, the steps for determining the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration gripping pose of the component to be mounted, and the calibration bottom point cloud and calibration mounting pose of the component to be mounted include:

[0013] Place the mounting plate in the first preset position, scan the surface of the mounting plate, and obtain the calibration point cloud and calibration placement pose of the mounting plate;

[0014] Place the part to be installed in the second preset position, scan the surface of the part to be installed, and obtain the calibration surface point cloud of the part to be installed;

[0015] Control the robot to grasp the part to be installed, and record the current pose of the part to be installed for calibrating the grasping pose;

[0016] Place the component to be installed on the third preset position on the mounting plate, and obtain the current point cloud and current pose of the bottom of the component to be installed, so as to calibrate the bottom point cloud and calibrate the mounting pose.

[0017] Furthermore, the step of obtaining the first installation correction pose includes:

[0018] Obtain the current point cloud of the mounting plate, perform point cloud registration with the calibration point cloud of the mounting plate, and determine the first transformation matrix between the two;

[0019] The calibration placement pose is corrected based on the first transformation matrix to obtain the first installation correction pose.

[0020] Further, the steps of acquiring the current point cloud of the mounting board, performing point cloud registration between it and the calibration point cloud of the mounting board, and determining the first transformation matrix between the two include:

[0021] Calculate the feature descriptors of each point in the calibration point cloud of the mounting plate and the current point cloud of the mounting plate to construct several feature point pairs;

[0022] The first transformation matrix is ​​obtained by constructing a least-squares equation based on the feature point pairs.

[0023] The first transformation matrix is ​​finely registered to obtain the optimized first transformation matrix.

[0024] Further steps for obtaining the grasping and correction pose include:

[0025] Acquire the current grasped point cloud of the part to be installed, and transform the current grasped point cloud to the robot base coordinate system according to the prior transformation matrix between the acquisition device and the robot;

[0026] Perform point cloud registration between the calibration surface point cloud of the component to be installed and the currently captured point cloud, and determine the second transformation matrix between the two;

[0027] The calibrated grasping pose is corrected based on the second transformation matrix to obtain the corrected grasping pose.

[0028] Further steps for obtaining the scene point cloud of the component to be installed include:

[0029] The robot is controlled to grasp the part to be installed according to the grasping correction posture and move it to the fourth preset position;

[0030] Scan the bottom surface of the component to be installed to obtain the point cloud of the component in the capture scene, which is called the scene point cloud.

[0031] Furthermore, the step of obtaining the second installation correction pose includes:

[0032] Perform point cloud registration between the scene point cloud and the calibration bottom point cloud, and determine the third transformation matrix between the two.

[0033] The calibration installation pose is corrected based on the third transformation matrix to obtain the second installation correction pose.

[0034] Furthermore, the step of correcting the calibration installation pose based on the third transformation matrix to obtain the second installation correction pose includes:

[0035]

[0036] Among them, F rec For the second installation correction pose of the part to be installed, C p_m To obtain the robot's pose when the data acquisition device acquires the calibration bottom point cloud of the part to be installed, T jp This is the third transformation matrix. For camera extrinsic parameters, C p_c The robot's pose when acquiring the bottom scene point cloud of the component to be installed.

[0037] Secondly, this application also provides a computer storage medium storing executable program code; the executable program code is used to execute the point cloud registration-based collaborative assembly method described in any one of the first aspects.

[0038] Thirdly, this application also provides a terminal device, including a memory and a processor; the memory stores program code executable by the processor; the program code is used to execute the point cloud registration-based collaborative assembly method described in any one of the first aspects.

[0039] This invention provides a collaborative assembly method, medium, and device based on point cloud registration. It determines the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration gripping pose of the component to be installed, and the calibration bottom point cloud and calibration installation pose of the component to be installed. Then, it acquires the current point cloud of the mounting plate and corrects the calibration placement pose based on the calibration point cloud and the current point cloud of the mounting plate to obtain a first corrected installation pose. Since the mounting plate may shift during installation, the offset of the mounting plate is calculated first to improve the installation accuracy of subsequent installation steps. Next, it acquires the current point cloud of the component to be installed and corrects the calibration gripping pose based on the calibration surface point cloud and the current point cloud of the component to be installed to obtain the corrected actual gripping pose, avoiding errors caused by the placement position of the component to be installed. Misalignment can cause errors when the robot grasps the part to be installed, affecting assembly efficiency. To address this, the robot grasps the part based on its actual grasping pose and acquires the bottom scene point cloud of the part in the grasping scenario. By acquiring the point cloud of the contact surface between the part and the mounting plate, the accuracy of the calculated final installation pose is improved, eliminating influencing factors and reducing errors. Then, the calibrated installation pose is corrected based on the bottom scene point cloud and the calibrated bottom point cloud to obtain a second corrected installation pose. Finally, the first and second corrected installation poses are combined to obtain the final installation pose of the part. By simultaneously integrating the pose changes between the mounting plate and the part, assembly accuracy is improved, while avoiding the need for system reprogramming and adjustments due to production demands and product upgrades. This solves the problems of low assembly accuracy and poor adaptability in existing technologies. Attached Figure Description

[0040] Figure 1 This is a flowchart of a collaborative assembly method based on point cloud registration according to an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram showing the placement of the compressor according to an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the mounting base position according to an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the bottom position of the compressor according to an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0045] It should be noted that if the embodiments of the present invention involve directional indications, such as up, down, left, right, front, back, etc., these directional indications are only used to explain the relative positional relationship and movement of the components in a specific orientation. If the specific orientation changes, the directional indications will also change accordingly. Furthermore, if the embodiments of the present invention involve descriptions such as "first," "second," "S1," "S2," "step one," "step two," etc., these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance, or implicitly specifying the number of technical features indicated or the execution order of the method. Those skilled in the art will understand that anything that does not violate the inventive concept should be included within the scope of protection of the present invention.

[0046] like Figure 1 As shown, this invention provides a collaborative assembly method based on point cloud registration:

[0047] S1: Determine the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration gripping pose of the part to be installed, and the calibration bottom point cloud and calibration mounting pose of the part to be installed.

[0048] Specifically, the acquisition method can be, but is not limited to, acquiring point clouds and poses at a preset ideal position on the mounting plate using an acquisition device to obtain calibration point clouds and calibration placement poses for the mounting plate; acquiring point clouds and poses at a preset ideal position on the part to be installed, where it can be accurately grasped, to obtain calibration surface point clouds and calibration grasping poses for the part to be installed; and acquiring information at the bottom of the part to be installed, where it is accurately placed on the mounting plate, to obtain calibration installation poses. The acquisition device can be, optionally, a digital acquisition device, a camera, a mobile phone, or other acquisition device capable of capturing still images or continuous video. The teaching motion trajectory can be, optionally, manually taught by a person skilled in the art or preset.

[0049] Preferably, the steps of determining the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration gripping pose of the part to be mounted, and the calibration bottom point cloud and calibration mounting pose of the part to be mounted may optionally include, but are not limited to:

[0050] S11: Place the mounting plate in the first preset position, scan the surface of the mounting plate, and obtain the calibration point cloud and calibration placement pose of the mounting plate;

[0051] For example, taking a compressor as the component to be installed, the mounting plate is first conveyed to a predetermined position by a conveyor belt. Assuming that the mounting plate is in the most ideal preset position, i.e. the first preset position, the surface of the mounting plate can be scanned to collect the point cloud and pose of the mounting plate at this time, which are used to calibrate the point cloud and calibrate the placement pose of the mounting plate.

[0052] S12: Place the part to be installed in the second preset position, scan the surface of the part to be installed and record the current pose of the part to be installed, and obtain the calibration surface point cloud and calibration grasping pose of the part to be installed.

[0053] For example, taking a compressor as the component to be installed, the compressor is placed in a location such as... Figure 2 In the material trough shown, assuming that the compressor is at the preset ideal position, i.e. the second preset position, the surface of the part to be installed is scanned to collect the point cloud of the compressor at this time, which is the calibration surface point cloud of the compressor. Then, the robot controlled by those skilled in the art can grasp the compressor and record the current pose of the compressor as the calibration grasping pose.

[0054] S13: Control the robot to grab the part to be installed, scan the bottom of the part to be installed, and obtain the bottom point cloud of the part to be installed;

[0055] Specifically, but not limited to, controlling the robot to grab the part to be installed into the air, so that the acquisition device can scan the bottom of the part to be installed, and thus obtain the bottom point cloud of the part to be installed.

[0056] S14: Place the component to be installed on the third preset position on the mounting plate, and obtain the current point cloud and current pose of the component to be installed, so as to calibrate the bottom point cloud and calibrate the mounting pose.

[0057] For example, taking a compressor as the component to be installed, it is optional, but not limited to, according to step S13, that a robot controlled by a person skilled in the art can grasp the compressor to correctly install the compressor onto the correct position on the mounting plate. At this time, the point cloud of the compressor on the mounting plate is the calibration bottom point cloud, and its corresponding pose is the calibration installation pose. At the same time, the robot's motion trajectory is recorded as the robot's teaching motion trajectory.

[0058] S2: Obtain the current point cloud of the mounting plate, and correct the calibration placement pose based on the calibration point cloud and the current point cloud of the mounting plate to obtain the first installation correction pose;

[0059] Specifically, the method can be optional, but not limited to, scanning the surface of the mounting plate in the current scene using a data acquisition device to obtain the current point cloud of the mounting plate in the coordinate system of the data acquisition device. Then, a global feature point cloud registration algorithm is used to calculate the rotation and translation matrix between the calibration point cloud of the mounting plate and the current point cloud of the mounting plate, which is the first transformation matrix. The calibration placement pose is then corrected based on the first transformation matrix to obtain the first installation correction pose. The global feature point cloud registration algorithm can include commonly used algorithms such as the Teaser++ algorithm, the global optimization iterative nearest point algorithm, the fast global registration algorithm, and the super four-point coplanar algorithm. More specifically, the Teaser++ algorithm is preferred for global point cloud registration.

[0060] Preferably, the step of acquiring the current point cloud of the mounting plate, correcting the calibration placement pose based on the calibration point cloud of the mounting plate and the current point cloud of the mounting plate, and obtaining the first installation correction pose may optionally include, but is not limited to, the following:

[0061] S21: Obtain the current point cloud of the mounting plate, perform point cloud registration with the calibration point cloud of the mounting plate, and determine the first transformation matrix between the two;

[0062] Preferably, the steps of acquiring the current point cloud of the mounting plate, performing point cloud registration between it and the calibration point cloud of the mounting plate, and determining the first transformation matrix between the two may include:

[0063] S211: Calculate the feature descriptors of each point in the calibration point cloud of the mounting plate and the current point cloud of the mounting plate to construct several feature point pairs;

[0064] S212: Construct a least-squares equation based on the feature point pairs to obtain the first transformation matrix;

[0065] Specifically, the method can be optional, but not limited to, calculating the feature descriptors of each point in the calibration point cloud and the current point cloud of the mounting board using a point cloud feature extraction algorithm. Then, the points whose feature descriptors in the calibration point cloud and the current point cloud of the mounting board are closest to each other form a feature point pair. By traversing the calibration point cloud and the current point cloud of the mounting board, several feature point pairs can be obtained. Based on the feature point pairs, a least squares expression is constructed to obtain the first transformation matrix. The point cloud feature extraction algorithm can include common algorithms such as the fast point feature histogram algorithm, the normal vector calculation method, the superpoint graph algorithm, and the curvature descriptor algorithm.

[0066] Preferably, since the global feature point cloud registration algorithm is mainly based on the global features of the point cloud, it may fail to capture local details. Therefore, the initial transformation matrix may have large errors in some areas, potentially leading to local optima rather than global optima, especially when the point cloud has complex structures or repetitive features. Furthermore, it may be affected by noise and outliers, resulting in an inaccurate transformation matrix. Therefore, after determining the first transformation matrix between the calibration point cloud of the mounting plate and the current point cloud, the algorithm may optionally include:

[0067] S213: Perform fine registration on the first transformation matrix to obtain the optimized first transformation matrix.

[0068] Specifically, the Iterative Closest Point (IPC) algorithm can be used, but is not limited to, to perform fine registration of the first transformation matrix based on the calibration point cloud of the mounting board and the current point cloud, thereby obtaining the optimized first transformation matrix T. 12 .

[0069] S22: Correct the calibration placement pose according to the first transformation matrix to obtain the first installation correction pose.

[0070] Specifically, the first installation correction pose corresponding to the current installation plate can be calculated by substituting the first transformation matrix obtained in step S21, the calibration placement pose of the installation plate in step S1, and the rotation and translation matrix between the acquisition device and the robot, i.e., the eye-to-hand extrinsic parameter (E), into the pose calculation formula.

[0071] Preferably, but not limited to, the first mounting correction pose corresponding to the current mounting plate is calculated according to pose calculation formula 1-1:

[0072]

[0073] Among them, F cur The first mounting and correction pose corresponds to the mounting plate, E is the eye-to-hand extrinsic parameter, i.e., the relationship between the camera and the robot's base coordinate system, and T... 12 G1 is the transformation matrix between the point cloud of the mounting plate template and the current point cloud of the mounting plate, and G1 is the calibration placement pose of the mounting plate.

[0074] Preferably, since the current point cloud of the mounting board may contain excessively redundant data, it needs to be optimized to reduce the data processing volume in subsequent steps, thereby saving resources and improving efficiency. Therefore, before step S21, the following steps may also be included:

[0075] S20: Filter and / or downsample the current point cloud of the mounting plate to obtain the optimized current point cloud of the mounting plate;

[0076] Specifically, since the current point cloud of the mounting plate acquired by the acquisition device contains a large amount of data, in order to reduce the amount of data processing in subsequent steps and improve efficiency, after obtaining the current point cloud of the mounting plate, the current point cloud of the mounting plate can be filtered according to the pass-through filtering method to remove outliers and noise in the point cloud, and then the point cloud is uniformly downsampled to simplify the point cloud data structure and obtain the optimized point cloud data of the calibration workpiece.

[0077] It is worth noting that parameters such as the filtering range in the pass-through filtering method and the downsampling rate in the uniform downsampling method can be arbitrarily set by those skilled in the art.

[0078] S3: Obtain the current point cloud of the part to be installed, and correct the calibrated grasping pose based on the calibration surface point cloud of the part to be installed and the current point cloud of the part to be installed, so as to obtain the corrected actual grasping pose.

[0079] Specifically, optional but not limited to scanning the surface of the part to be installed in the current scene using a data acquisition device to obtain the current grasping point cloud of the part to be installed, then calculating the rotation and translation matrix between the calibration surface point cloud and the current grasping point cloud using a global feature point cloud registration algorithm, which is the second transformation matrix, and correcting the calibration grasping pose according to the second transformation matrix to obtain the grasping corrected pose.

[0080] Preferably, the specific steps for obtaining the current point cloud of the part to be installed, and correcting the calibration grasping pose based on the calibration surface point cloud of the part to be installed and the current point cloud of the part to be installed, to obtain the grasping correction pose, may include, but are not limited to:

[0081] S31: Obtain the current grasped point cloud of the part to be installed, and transform the current grasped point cloud to the robot base coordinate system according to the prior transformation matrix between the acquisition device and the robot;

[0082] For example, you can optionally convert the currently captured point cloud to the robot's base coordinate system according to point cloud conversion formula 3-1:

[0083]

[0084] Among them, P b This represents the current grasped point cloud of the part to be installed in the robot's base coordinate system. To acquire the device pose, i.e., the transformation matrix from the robot end effector to the robot base coordinate system. Let P be the prior transformation matrix between the data acquisition device and the robot. c This is the current point cloud captured in the coordinate system of the acquisition device.

[0085] S32: Perform point cloud registration between the calibration surface point cloud of the part to be installed and the currently captured point cloud, and determine the second transformation matrix between the two;

[0086] Specifically, the rotation and translation matrix between the calibration surface point cloud and the current gripping point cloud of the part to be installed in the robot base coordinate system can be calculated by a global feature point cloud registration algorithm, and this matrix serves as the second transformation matrix.

[0087] Preferably, since the global feature point cloud registration algorithm is mainly based on the global features of the point cloud, it may not be able to capture local details. Therefore, it is preferable to use the Iterative Closest Point (ICP) algorithm to perform fine registration on the second transformation matrix to obtain the optimized second transformation matrix.

[0088] S33: Correct the calibrated grasping pose according to the second transformation matrix to obtain the corrected grasping pose.

[0089] Preferably, but not limited to, the calibration surface point cloud is calibrated according to the grasping pose correction formula 3-2:

[0090] G zq =T 12 ·G s 3-2

[0091] Among them, G zq To capture and correct the pose, T 12 G is the second transformation matrix. s To calibrate the grasping pose.

[0092] Preferably, since the current point cloud of the component to be installed may contain excessively redundant data, it needs to be optimized to reduce the data processing volume in subsequent steps, thereby saving resources and improving efficiency. Therefore, before step S31, the following steps may also be included:

[0093] S30: Obtain the current point cloud of the component to be installed, and perform filtering and / or downsampling processing on it to obtain the optimized current point cloud;

[0094] Specifically, optional but not limited to performing filtering and / or downsampling processing on the current captured point cloud of the component to be installed according to step S21, to obtain an optimized current captured point cloud.

[0095] S4: Based on the actual grasping pose, grasp the component to be installed and obtain the bottom scene point cloud of the component to be installed in the grasping scene;

[0096] Specifically, taking a compressor as an example, since... Figure 3 The surface of the mounting plate shown and as Figure 4The bottom of the compressor shown is connected to each other by mounting studs and mounting holes. Therefore, in order to improve the assembly accuracy, it is possible, but not limited to, to control the robot to grasp the part to be installed according to the grasping correction posture obtained in step S3, and grasp it into the air so that the acquisition device can scan its bottom to obtain the surface point cloud of the bottom of the part to be installed under the grasping scene at this time, which is the bottom scene point cloud.

[0097] Preferably, the steps of grasping the object to be installed based on the grasping and correcting pose, and obtaining the bottom scene point cloud of the object to be installed in the grasping scene, may include, but are not limited to:

[0098] S41: Control the robot to grasp the part to be installed according to the grasping correction posture and move it to the fourth preset position;

[0099] S42: Scan the bottom surface of the component to be installed to obtain the point cloud of the component to be installed in the grasping scene, which is the bottom scene point cloud.

[0100] For example, taking a compressor as the component to be installed, once the gripping correction pose of the compressor is calculated, the robot can be controlled to grip the compressor according to the gripping correction pose, so as to grip it to the most ideal position preset in mid-air, namely the fourth preset position. This allows the acquisition device to scan the bottom surface of the compressor and obtain the scene point cloud of the compressor.

[0101] Preferably, after acquiring the scene point cloud of the component to be installed in the grasping scenario, it can be filtered and / or downsampled to reduce the amount of data processing in subsequent steps, thereby saving resources and improving efficiency.

[0102] S5: Correct the calibration installation pose based on the bottom scene point cloud and the calibration bottom point cloud to obtain the second installation correction pose;

[0103] Specifically, optional but not limited to using the bottom scene point cloud of the component to be installed obtained in step S4 and the calibration bottom point cloud of the component to be installed in step S1, a transformation matrix between the two can be calculated using a global feature point cloud registration algorithm. This is the third transformation matrix. The calibration installation pose can be corrected based on the third transformation matrix to obtain the pose of the component to be installed on the mounting plate, which is the second installation correction pose.

[0104] Preferably, the step of correcting the calibration installation pose based on the bottom scene point cloud and the calibration bottom point cloud to obtain the second installation correction pose may include, but is not limited to, the following:

[0105] S51: Perform point cloud registration between the bottom scene point cloud and the calibration bottom point cloud, and determine the third transformation matrix between the two;

[0106] Specifically, the rotation and translation matrix between the bottom scene point cloud and the calibrated bottom point cloud can be calculated using a global feature point cloud registration algorithm, and this matrix serves as the third transformation matrix.

[0107] Preferably, since the global feature point cloud registration algorithm is mainly based on the global features of the point cloud, it may not be able to capture local details. Therefore, it is preferable to use the Iterative Closest Point (IPC) algorithm to perform fine registration on the third transformation matrix, so as to obtain the optimized third transformation matrix T. jp .

[0108] S52: Correct the calibration installation pose according to the third transformation matrix to obtain the second installation correction pose.

[0109] For example, optional but not limited to correcting the calibration installation pose according to the correction pose calculation formula 5-1, the second installation correction pose is obtained:

[0110]

[0111] Among them, F rec For the second installation correction pose of the part to be installed, C p_m The image capture point for the correction model refers to the robot's pose when the acquisition device obtains the calibration bottom point cloud of the part to be installed. T jp This is the third transformation matrix. For camera extrinsic parameters, C p_c The current shooting point is the robot's pose when acquiring the bottom scene point cloud of the component to be installed.

[0112] S6: Combine the first installation correction pose and the second installation correction pose to obtain the final installation pose of the part to be installed.

[0113] Specifically, since both the mounting plate and the component to be installed will shift to a certain extent during the installation process, when calculating the final installation pose of the component, it is necessary to consider the influence of the displacement of the mounting plate and the displacement of the component itself on the final installation pose. This can be achieved by selecting, but not limited to, the first installation correction pose of the mounting plate obtained in step S2 after the change of the mounting plate, and the second installation correction pose of the component after the change of the calibration gripping pose of the component in step S5. By combining the first and second installation correction poses, the final installation pose of the component can be obtained.

[0114] For example, the formula for calculating the final installation pose of the component to be installed can be, but is not limited to, expressed as Equation 6-1:

[0115] F = F cur ·F rec 6-1

[0116] Where F represents the final installation position of the component to be installed. cur For the first installation correction pose of the mounting plate, F rec This is the second installation and calibration position for the component to be installed.

[0117] In this embodiment, a collaborative assembly method based on point cloud registration according to the present invention is presented. This method determines the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration gripping pose of the component to be installed, and the calibration bottom point cloud and calibration installation pose of the component to be installed. Then, the current point cloud of the mounting plate is obtained. The calibration placement pose is corrected based on the calibration point cloud and the current point cloud of the mounting plate to obtain a first corrected installation pose. Since the mounting plate may shift during installation, the offset of the mounting plate is calculated first to improve the installation accuracy of subsequent installation steps. Then, the current point cloud of the component to be installed is obtained. The calibration gripping pose is corrected based on the calibration surface point cloud and the current point cloud of the component to be installed to obtain the corrected actual gripping pose, avoiding errors caused by the placement of the component to be installed. Errors caused by positional deviations can lead to inaccuracies when the robot grasps the part to be installed, affecting assembly efficiency. This problem can be addressed by grasping the part based on its actual grasping pose and acquiring the bottom scene point cloud of the part in the grasping scenario. By acquiring the point cloud of the contact surface between the part and the mounting plate, the accuracy of calculating the final installation pose of the part can be improved, eliminating influencing factors and reducing errors. Then, the calibrated installation pose is corrected based on the bottom scene point cloud and the calibrated bottom point cloud to obtain a second corrected installation pose. Finally, the first and second corrected installation poses are combined to obtain the final installation pose of the part. By simultaneously integrating the pose changes between the mounting plate and the part, assembly accuracy is improved, while avoiding the need for system reprogramming and adjustment due to production demands and product upgrades. This solves the problems of low assembly accuracy and poor adaptability in existing technologies.

[0118] On the other hand, the present invention also provides a computer storage medium storing executable program code; the executable program code is used to execute any of the above-mentioned point cloud registration-based collaborative assembly methods.

[0119] On the other hand, the present invention also provides a terminal device, including a memory and a processor; the memory stores program code that can be executed by the processor; the program code is used to execute any of the above-mentioned point cloud registration-based collaborative assembly methods.

[0120] For example, the program code can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the program code in the terminal device.

[0121] The terminal device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the terminal device may also include input / output devices, network access devices, buses, etc.

[0122] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0123] The memory can be an internal storage unit of the terminal device, such as a hard drive or RAM. The memory can also be an external storage device of the terminal device, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory can include both internal and external storage units of the terminal device. The memory is used to store the program code and other programs and data required by the terminal device. The memory can also be used to temporarily store data that has been output or will be output.

[0124] The aforementioned computer storage medium and terminal device are created based on the aforementioned collaborative assembly method based on point cloud registration. Their technical functions and beneficial effects will not be elaborated here. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A collaborative assembly method based on point cloud registration, characterized in that, include: Determine the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration gripping pose of the part to be installed, and the calibration bottom point cloud and calibration installation pose of the part to be installed. Obtain the current point cloud of the mounting plate, and correct the calibration placement pose based on the calibration point cloud and the current point cloud of the mounting plate to obtain the first installation correction pose; Obtain the current point cloud of the part to be installed, and correct the calibrated grasping pose based on the calibration surface point cloud of the part to be installed and the current point cloud of the part to be installed to obtain the corrected actual grasping pose. Based on the actual grasping pose, grasp the component to be installed and obtain the bottom scene point cloud of the component to be installed in the grasping scene; The calibration installation pose is corrected based on the bottom scene point cloud and the calibration bottom point cloud to obtain the second installation correction pose. By combining the first and second installation correction poses, the final installation pose of the component to be installed is obtained.

2. The method according to claim 1, characterized in that, The steps for determining the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration gripping pose of the part to be mounted, and the calibration bottom point cloud and calibration mounting pose of the part to be mounted include: Place the mounting plate in the first preset position, scan the surface of the mounting plate, and obtain the calibration point cloud and calibration placement pose of the mounting plate; Place the part to be installed in the second preset position, scan the surface of the part to be installed, and obtain the calibration surface point cloud of the part to be installed; Control the robot to grasp the part to be installed, and record the current pose of the part to be installed for calibrating the grasping pose; Place the component to be installed on the third preset position on the mounting plate, and obtain the current point cloud and current pose of the bottom of the component to be installed, so as to calibrate the bottom point cloud and calibrate the mounting pose.

3. The method according to claim 1, characterized in that, The steps for obtaining the first installation and calibration pose include: Obtain the current point cloud of the mounting plate, perform point cloud registration with the calibration point cloud of the mounting plate, and determine the first transformation matrix between the two; The calibration placement pose is corrected based on the first transformation matrix to obtain the first installation correction pose.

4. The method according to claim 3, characterized in that, The steps of acquiring the current point cloud of the mounting board, registering it with the calibration point cloud of the mounting board, and determining the first transformation matrix between the two include: Calculate the feature descriptors of each point in the calibration point cloud of the mounting plate and the current point cloud of the mounting plate to construct several feature point pairs; The first transformation matrix is ​​obtained by constructing a least-squares equation based on the feature point pairs. The first transformation matrix is ​​finely registered to obtain the optimized first transformation matrix.

5. The method according to claim 1, characterized in that, The steps to obtain the capture and correction pose include: Acquire the current grasped point cloud of the part to be installed, and transform the current grasped point cloud to the robot base coordinate system according to the prior transformation matrix between the acquisition device and the robot; Perform point cloud registration between the calibration surface point cloud of the component to be installed and the currently captured point cloud, and determine the second transformation matrix between the two; The calibrated grasping pose is corrected based on the second transformation matrix to obtain the corrected grasping pose.

6. The method according to claim 1, characterized in that, The steps to obtain the scene point cloud of the component to be installed include: The robot is controlled to grasp the part to be installed according to the grasping correction posture and move it to the fourth preset position; Scan the bottom surface of the component to be installed to obtain the point cloud of the component in the capture scene, which is called the scene point cloud.

7. The method according to claim 1, characterized in that, The steps for obtaining the second installation and calibration pose include: Perform point cloud registration between the scene point cloud and the calibration bottom point cloud, and determine the third transformation matrix between the two. The calibration installation pose is corrected based on the third transformation matrix to obtain the second installation correction pose.

8. The method according to claim 7, characterized in that, The steps for correcting the calibration installation pose based on the third transformation matrix to obtain the second installation correction pose include: Among them, F rec For the second installation correction pose of the part to be installed, C p_m To obtain the robot's pose when the data acquisition device acquires the calibration bottom point cloud of the part to be installed, T jp This is the third transformation matrix. For camera extrinsic parameters, C p_c The robot's pose when acquiring the bottom scene point cloud of the component to be installed.

9. A computer storage medium, characterized in that, It stores executable program code; the executable program code is used to execute the collaborative assembly method based on point cloud registration as described in any one of claims 1-8.

10. A terminal device, characterized in that, It includes a memory and a processor; the memory stores program code that can be executed by the processor; the program code is used to execute the collaborative assembly method based on point cloud registration as described in any one of claims 1-8.

Images