Collaborative assembly method based on point cloud registration, medium and equipment

Abstract

The invention provides a collaborative assembly method based on point cloud registration, a medium and equipment, and the method comprises the steps: determining a calibration point cloud and a calibration placement pose of a mounting plate, a calibration surface point cloud and a calibration grabbing pose of a to-be-mounted part, and a calibration bottom point cloud and a calibration mounting pose of the to-be-mounted part; obtaining a first installation correction pose according to the calibration point cloud of the installation plate and the current point cloud of the installation plate, obtaining a corrected actual grabbing pose according to the calibration surface point cloud of the to-be-installed part and the current point cloud of the to-be-installed part, grabbing the to-be-installed part according to the actual grabbing pose, and obtaining a bottom scene point cloud of the to-be-installed part in a grabbing scene, and correcting the calibrated installation pose according to the bottom scene point cloud and the calibrated bottom point cloud to obtain a second installation correction pose, and integrating the first installation correction pose and the second installation correction pose to obtain a final installation pose of the to-be-installed part. The problems of low assembly precision, poor adaptability and the like in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the field of automated production and assembly technology, and in particular to a collaborative assembly method, medium and equipment based on point cloud registration. Background Art

[0002] In traditional industrial assembly processes, robotics is mainly used to perform single or limited assembly tasks. These single-robot systems are usually designed to handle standardized, highly repetitive tasks, such as simple component picking and placement. In these applications, single-robot systems can provide high operating efficiency and low labor costs. However, as the design of industrial products becomes increasingly complex, the requirements for assembly accuracy and flexibility are also increasing. In high-precision assembly tasks involving multiple components and multiple steps, single-robot systems often find it difficult to meet production needs. When handling multi-component, multi-step assembly tasks, due to the singleness of their operations and the limitations of path planning, they often take more time to complete the entire assembly process. This not only limits the overall efficiency of the production line, but also increases production costs. At the same time, in high-precision assembly tasks, a single robot system may find it difficult to control multiple operation points and paths at the same time, resulting in a decrease in assembly accuracy and affecting the quality of the final product. In the face of changing production needs and product upgrades, single robot systems often need to be reprogrammed and adjusted, which is not only time-consuming and labor-intensive, but also limits the rapid response capability of the production line. In scenarios where multiple robots are required to collaborate to complete complex assembly tasks, single robot systems lack an effective collaboration mechanism, making it difficult to achieve effective communication and coordination between multiple robots, thereby failing to fully tap the potential of the multi-robot system.

[0003] Therefore, how to improve it is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0004] Based on this, the purpose of this application is to provide a collaborative assembly method, medium and device based on point cloud registration to solve at least one technical problem mentioned in the above background technology.

[0005] In a first aspect, the present application provides a collaborative assembly method based on point cloud registration, comprising:

[0006] Determine the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration grasping pose of the part to be mounted, and the calibration bottom point cloud and calibration mounting pose of the part to be mounted;

[0007] Acquire the current point cloud of the mounting plate, and calibrate the placement posture according to the calibration point cloud of the mounting plate and the current point cloud of the mounting plate to obtain a first installation correction posture;

[0008] Acquire the current point cloud of the part to be installed, and correct the calibrated grasping posture according to the calibrated surface point cloud of the part to be installed and the current point cloud of the part to be installed to obtain the corrected actual grasping posture;

[0009] According to the actual grasping posture, grasp the part to be installed, and obtain the bottom scene point cloud of the part to be installed in the grasping scene;

[0010] Correcting the calibrated installation posture according to the bottom scene point cloud and the calibrated bottom point cloud to obtain a second installation correction posture;

[0011] The final installation posture of the component to be installed is obtained by combining the first installation correction posture and the second installation correction posture.

[0012] Further, the steps of determining the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration grasping pose of the part to be mounted, and the calibration bottom point cloud and calibration mounting pose of the part to be mounted include:

[0013] The mounting plate is placed at a first preset position, and the surface of the mounting plate is scanned to obtain a calibration point cloud and a calibration placement pose of the mounting plate;

[0014] Placing the part to be installed at a second preset position, scanning the surface of the part to be installed, and obtaining a calibrated surface point cloud of the part to be installed;

[0015] Control the robot to grab the part to be installed and record the current position and posture of the part to be installed to calibrate the grabbing position and posture;

[0016] The part to be installed is placed at a third preset position on the installation plate, and the current point cloud and current posture of the bottom of the part to be installed are obtained to calibrate the bottom point cloud and the installation posture.

[0017] Furthermore, the step of obtaining the first installation correction posture includes:

[0018] Obtain the current point cloud of the mounting plate, perform point cloud registration on the current point cloud and the calibration point cloud of the mounting plate, and determine a first transformation matrix between the two;

[0019] The calibrated placement posture is corrected according to the first transformation matrix to obtain a first installation correction posture.

[0020] Furthermore, the step of obtaining the current point cloud of the mounting plate, performing point cloud registration on the current point cloud and the calibration point cloud of the mounting plate, and determining a first transformation matrix therebetween includes:

[0021] Calculate the feature descriptors of each point in the calibration point cloud of the mounting plate and the current point cloud of the mounting plate to construct a number of feature point pairs;

[0022] Construct a least squares equation based on the feature point pairs to obtain a first transformation matrix;

[0023] The first transformation matrix is ​​precisely aligned to obtain an optimized first transformation matrix.

[0024] Furthermore, the step of obtaining the grasping correction posture includes:

[0025] Obtain the current grasped point cloud of the part to be installed, and transform the current grasped point cloud into the robot base coordinate system according to the prior transformation matrix between the acquisition device and the robot;

[0026] Performing point cloud registration on the calibrated surface point cloud of the installation part and the currently captured point cloud to determine a second transformation matrix between the two;

[0027] The calibrated grasping posture is corrected according to the second transformation matrix to obtain the grasping correction posture.

[0028] Furthermore, the step of obtaining the scene point cloud of the part to be installed includes:

[0029] Controlling the robot to grasp the part to be installed according to the grasping correction posture, and moving it to a fourth preset position;

[0030] Scan the bottom surface of the part to be installed to obtain the point cloud of the part to be installed in the capture scene, which is the scene point cloud.

[0031] Furthermore, the step of obtaining a second installation correction posture includes:

[0032] Perform point cloud registration on the scene point cloud and the calibration bottom point cloud to determine the third transformation matrix between the two;

[0033] The calibrated installation posture is corrected according to the third transformation matrix to obtain a second installation correction posture.

[0034] Further, the step of correcting the calibrated installation posture according to the third transformation matrix to obtain a second installation correction posture includes:

[0035]

[0036] Among them, F rec is the second installation correction posture of the part to be installed, C p_m T is the position and posture of the robot when the acquisition device obtains the calibrated bottom point cloud of the part to be installed. jp is the third transformation matrix, is the camera external parameter, C p_c The robot's position and posture when obtaining the bottom scene point cloud of the part to be installed.

[0037] In a second aspect, the present application further provides a computer storage medium storing an executable program code; the executable program code is used to execute the collaborative assembly method based on point cloud registration as described in any one of the first aspects.

[0038] In a third aspect, the present application also provides a terminal device, comprising a memory and a processor; the memory stores program code executable by the processor; the program code is used to execute the collaborative assembly method based on point cloud registration as described in any one of the first aspects.

[0039] The present invention provides a collaborative assembly method, medium and equipment based on point cloud registration, which determines the calibrated point cloud and calibrated placement posture of the mounting plate, the calibrated surface point cloud and calibrated grasping posture of the part to be mounted, and the calibrated bottom point cloud and calibrated installation posture of the part to be mounted, and then obtains the current point cloud of the mounting plate, corrects the calibrated placement posture according to the calibrated point cloud of the mounting plate and the current point cloud of the mounting plate, and obtains the first installation correction posture. Since the mounting plate may be offset during installation, the offset of the mounting plate is first calculated to improve the installation accuracy of subsequent installation steps, and then obtains the current point cloud of the part to be mounted, and corrects the calibrated grasping posture according to the calibrated surface point cloud of the part to be mounted and the current point cloud of the part to be mounted, to obtain the corrected actual grasping posture, so as to avoid the placement position of the part to be mounted. The offset causes errors when the robot grabs the parts to be installed, affecting the assembly efficiency. The parts to be installed can be grabbed according to the actual grasping posture, and the bottom scene point cloud of the parts to be installed in the grasping scene can be obtained. By obtaining the point cloud of the contact surface between the parts to be installed and the mounting plate, the accuracy of calculating the final installation posture of the parts to be installed can be improved, the influencing factors can be eliminated, and the error can be reduced. Then, the calibrated installation posture is corrected according to the bottom scene point cloud and the calibrated bottom point cloud to obtain the second installation correction posture. Finally, the first installation correction posture and the second installation correction posture are combined to obtain the final installation posture of the parts to be installed. By simultaneously combining the posture changes between the mounting plate and the parts to be installed, the assembly accuracy is improved, and the problem of reprogramming and adjustment of the system due to production needs and product upgrades is avoided. The problems of low assembly accuracy and poor adaptability of the existing technology are solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of a collaborative assembly method based on point cloud registration according to an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the placement of a compressor according to an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of the position of the mounting base according to an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the bottom position of the compressor according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] It should be noted that if the embodiments of the present invention involve directional indications, such as up, down, left, right, front, back, etc., then the directional indication is only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly. In addition, if the embodiments of the present invention involve descriptions of "first, second", "S1, S2", "step one, step two", etc., then such descriptions are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of indicated technical features or indicating the execution order of the method, etc. Those skilled in the art can understand that anything that does not violate the gist of the invention under the technical concept of the invention should be included in the protection scope of the present invention.

[0046] like Figure 1 As shown, the present invention provides a collaborative assembly method based on point cloud registration:

[0047] S1: Determine the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration grasping pose of the part to be mounted, and the calibration bottom point cloud and calibration mounting pose of the part to be mounted;

[0048] Specifically, it is optional but not limited to collecting the point cloud and posture in the preset ideal position of the mounting plate by the acquisition device, as the calibration point cloud and calibration placement posture of the mounting plate; at the preset ideal position of the part to be installed, the point cloud and posture in the state that can be accurately grasped are the calibration surface point cloud and calibration grasping posture of the part to be installed; when the part to be installed is accurately placed on the mounting plate, the collected information is the calibration bottom point cloud and the calibration installation posture; the acquisition device may optionally include digital acquisition devices, cameras, mobile phones and other acquisition devices that can be used to capture static images or continuous videos; the teaching motion trajectory may be optionally taught manually or preset by a technician in this field.

[0049] Preferably, the steps of determining the calibration point cloud and calibration placement posture of the mounting plate, the calibration surface point cloud and calibration gripping posture of the part to be mounted, and the calibration bottom point cloud and calibration mounting posture of the part to be mounted may optionally include but are not limited to:

[0050] S11: placing the mounting plate at a first preset position, scanning the surface of the mounting plate, and obtaining a calibrated point cloud and a calibrated placement posture of the mounting plate;

[0051] For example, taking the compressor as the part to be installed, the mounting plate is first conveyed to the predetermined position by the conveyor belt. Assuming that the mounting plate is in the preset most ideal position, that is, the first preset position, the surface of the mounting plate can be scanned to collect the point cloud and posture of the mounting plate at this time, which are the calibrated point cloud and calibrated placement posture of the mounting plate.

[0052] S12: placing the part to be installed at a second preset position, scanning the surface of the part to be installed and recording the current posture of the part to be installed, and obtaining a calibrated surface point cloud and a calibrated grasping posture of the part to be installed;

[0053] For example, taking the compressor as an example, the compressor is placed in a position such as Figure 2 In the material trough shown, assuming that the compressor is in the preset most ideal position, that is, the second preset position, the surface of the part to be installed is scanned to collect the point cloud of the compressor at this time, which is the calibration surface point cloud of the compressor. Then, a technician in this field can choose to control the robot to grasp the compressor and record the current posture of the compressor, which is the calibrated grasping posture.

[0054] S13: Control the robot to grab the part to be installed, scan the bottom of the part to be installed, and obtain a calibrated bottom point cloud of the part to be installed;

[0055] Specifically, it is optional but not limited to controlling the robot to grab the part to be installed in mid-air, so that the acquisition device can scan the bottom of the part to be installed, and the calibrated bottom point cloud of the part to be installed can be obtained.

[0056] S14: placing the part to be installed at a third preset position on the installation plate, and obtaining the current point cloud and current posture of the part to be installed, which are used to calibrate the bottom point cloud and the installation posture.

[0057] For example, taking the compressor as the part to be installed, it is optional but not limited to that a technician in this field controls the robot to grab the compressor according to step S13 to correctly install the compressor to the correct position on the mounting plate. At this time, the point cloud of the compressor on the mounting plate is the calibrated bottom point cloud, and its corresponding posture is the calibrated installation posture. At the same time, the robot's motion trajectory is recorded to teach the robot the motion trajectory.

[0058] S2: Acquire the current point cloud of the mounting plate, and calibrate the placement posture according to the calibration point cloud of the mounting plate and the current point cloud of the mounting plate to obtain a first installation correction posture;

[0059] Specifically, it is optional but not limited to scanning the surface of the mounting plate in the current scene by the acquisition device to obtain the current point cloud of the mounting plate in the acquisition device coordinate system, and then calculating the rotation and translation matrix between the calibration point cloud of the mounting plate and the current point cloud of the mounting plate by the global feature point cloud registration algorithm as the first transformation matrix, and correcting the calibration placement posture according to the first transformation matrix to obtain the first installation correction posture; the global feature point cloud registration algorithm may optionally include the Teaser++ algorithm, the global optimization iterative closest point algorithm, the fast global registration algorithm, the super four-point coplanar algorithm and other commonly used algorithms. More specifically, the global point cloud registration algorithm preferably adopts the Teaser++ algorithm.

[0060] Preferably, the step of obtaining the current point cloud of the mounting plate, and correcting the calibration placement posture according to the calibration point cloud of the mounting plate and the current point cloud of the mounting plate to obtain the first installation correction posture may optionally include but is not limited to:

[0061] S21: Acquire the current point cloud of the mounting plate, perform point cloud registration on the current point cloud and the calibration point cloud of the mounting plate, and determine a first transformation matrix between the two.

[0062] Preferably, the step of obtaining the current point cloud of the mounting plate, performing point cloud registration on the current point cloud and the calibration point cloud of the mounting plate, and determining a first transformation matrix therebetween may optionally include:

[0063] S211: Calculate the feature descriptors of each point in the calibration point cloud of the mounting plate and the current point cloud of the mounting plate to construct a plurality of feature point pairs;

[0064] S212: constructing a least squares equation according to the feature point pairs to obtain a first transformation matrix;

[0065] Specifically, it is optional but not limited to calculating the feature descriptors of each point in the calibration point cloud of the mounting plate and the current point cloud of the mounting plate through a point cloud feature extraction algorithm, and then forming a pair of feature point pairs with the points in the calibration point cloud of the mounting plate and the current point cloud of the mounting plate that are closest to the feature descriptors. By traversing the calibration point cloud of the mounting plate and the current point cloud of the mounting plate, a number of feature point pairs can be obtained, and a least squares formula is constructed based on the feature point pairs to obtain a first transformation matrix; the point cloud feature extraction algorithm may optionally include common algorithms such as fast point feature histogram algorithm, normal vector calculation method, super point graph algorithm, curvature descriptor algorithm, etc.

[0066] Preferably, since the global feature point cloud registration algorithm is mainly based on the global features of the point cloud, it may not be able to capture local details. Therefore, the preliminary transformation matrix may have large errors in some areas, which may lead to local optimality rather than global optimality, especially when the point cloud has complex structures or repeated features. At the same time, it may be affected by noise and outliers, resulting in inaccurate transformation matrix. Therefore, after determining the first transformation matrix between the calibration point cloud of the mounting plate and the current point cloud, it may also optionally include:

[0067] S213: Perform precise registration on the first transformation matrix to obtain an optimized first transformation matrix.

[0068] Specifically, it is optional but not limited to using an iterative closest point algorithm (IPC) to precisely align the first transformation matrix according to the calibration point cloud of the mounting plate and the current point cloud to obtain an optimized first transformation matrix T 12 .

[0069] S22: Correcting the calibration placement posture according to the first transformation matrix to obtain a first installation correction posture.

[0070] Specifically, it is optional but not limited to substituting the first transformation matrix obtained in step S21 and the calibrated placement posture of the mounting plate in step S1, as well as the rotation and translation matrix between the acquisition device and the robot, that is, the eye-in-hand external parameter (E), into the posture calculation formula to calculate the first installation correction posture corresponding to the current mounting plate.

[0071] Preferably, it is optional but not limited to calculating the first installation correction posture corresponding to the current installation plate according to the posture calculation formula 1-1:

[0072]

[0073] Among them, F cur is the first installation correction pose corresponding to the installation plate, E is the eye-in-hand external parameter, that is, the relationship between the camera and the robot base coordinate system, T 12 is the transformation matrix between the mounting plate template point cloud and the current point cloud of the mounting plate, and G1 is the calibrated placement pose of the mounting plate.

[0074] Preferably, since the amount of data redundancy in the current point cloud of the installation plate may be too large, it needs to be optimized to reduce the amount of data processing in subsequent steps, thereby saving resources and improving efficiency. Therefore, before step S21, the following may be optionally included:

[0075] S20: filtering and / or downsampling the current point cloud of the mounting plate to obtain an optimized current point cloud of the mounting plate;

[0076] Specifically, since the amount of data contained in the current point cloud of the mounting plate acquired by the acquisition device is too large, in order to reduce the amount of data processing in subsequent steps and improve efficiency, after obtaining the current point cloud of the mounting plate, the current point cloud of the mounting plate can be filtered according to the straight-through filtering method to remove outliers and noise points in the point cloud, and then the point cloud is uniformly down-sampled to simplify the point cloud data structure and obtain optimized point cloud data of the calibration workpiece.

[0077] It is worth noting that parameters such as the filtering range in the straight-through filtering method and the downsampling rate in the uniform downsampling method are arbitrarily set by those skilled in the art.

[0078] S3: obtaining the current point cloud of the part to be installed, and correcting the calibrated grasping posture according to the calibrated surface point cloud of the part to be installed and the current point cloud of the part to be installed to obtain the corrected actual grasping posture;

[0079] Specifically, it is optional but not limited to scanning the surface of the part to be installed in the current scene through an acquisition device to obtain the current grasping point cloud of the part to be installed, and then calculating the rotation and translation matrix between the calibrated surface point cloud and the current grasping point cloud through a global feature point cloud alignment algorithm as the second transformation matrix, and correcting the calibrated grasping posture according to the second transformation matrix to obtain the grasping correction posture.

[0080] Preferably, the current point cloud of the part to be installed is obtained, and the calibrated grasping posture is corrected according to the calibrated surface point cloud of the part to be installed and the current point cloud of the part to be installed, and the specific steps of obtaining the grasping correction posture are optional but not limited to include:

[0081] S31: Acquire the current grasping point cloud of the part to be installed, and transform the current grasping point cloud into the robot base coordinate system according to the prior transformation matrix between the acquisition device and the robot;

[0082] For example, you can choose to convert the current captured point cloud to the robot base coordinate system according to the point cloud conversion formula 3-1:

[0083]

[0084] Among them, P b is the current grasping point cloud of the part to be installed in the robot base coordinate system, To collect the device pose, that is, the transformation matrix from the robot end to the robot base coordinate system, is the prior transformation matrix between the acquisition device and the robot, P c It is the currently captured point cloud in the acquisition device coordinate system.

[0085] S32: performing point cloud registration on the calibrated surface point cloud of the installation part and the currently captured point cloud to determine a second conversion matrix between the two;

[0086] Specifically, it is optional but not limited to calculating the rotation and translation matrix between the calibration surface point cloud and the current grasping point cloud of the to-be-installed part in the robot base coordinate system through a global feature point cloud registration algorithm as the second transformation matrix.

[0087] Preferably, since the global feature point cloud registration algorithm is mainly based on the global features of the point cloud and may not be able to capture local details, it is preferred to use the iterative closest point algorithm (ICP) to accurately align the second transformation matrix to obtain an optimized second transformation matrix.

[0088] S33: Correcting the calibrated grasping posture according to the second transformation matrix to obtain a grasping correction posture.

[0089] Preferably, it is optional but not limited to correcting the calibration surface point cloud according to the grasping posture correction formula 3-2:

[0090] G zq =T 12 ·G s 3-2

[0091] Among them, G zq To grasp the correct posture, T 12 is the second transformation matrix, G s To calibrate the grasping pose.

[0092] Preferably, since the amount of data redundancy in the current captured point cloud of the part to be installed may be too large, it needs to be optimized to reduce the amount of data processing in subsequent steps, thereby saving resources and improving efficiency. Therefore, before step S31, the following may be optionally included:

[0093] S30: Acquire the current grasped point cloud of the part to be installed, and perform filtering and / or downsampling processing on it to obtain an optimized current grasped point cloud;

[0094] Specifically, it is optional but not limited to performing filtering and / or down-sampling processing operations on the current captured point cloud of the installation component according to step S21 to obtain an optimized current captured point cloud.

[0095] S4: Grasp the part to be installed according to the actual grasping posture, and obtain the bottom scene point cloud of the part to be installed in the grasping scene;

[0096] Specifically, taking the compressor as an example, due to the following Figure 3 The mounting plate surface shown and Figure 4The bottom of the compressor shown is connected to each other through mounting studs and mounting holes. Therefore, in order to improve the assembly accuracy, it is optional but not limited to controlling the robot to grasp the part to be installed according to the grasping correction posture obtained in step S3, grasping it in mid-air, so that the acquisition device scans its bottom, and obtains the surface point cloud of the bottom of the part to be installed in the grasping scene at this time, which is the bottom scene point cloud.

[0097] Preferably, the steps of grasping the part to be installed according to the grasping correction posture and obtaining the bottom scene point cloud of the part to be installed in the grasping scene may optionally include but are not limited to:

[0098] S41: Control the robot to grasp the part to be installed according to the grasping correction posture, and move it to a fourth preset position;

[0099] S42: Scan the bottom surface of the part to be installed to obtain a point cloud of the part to be installed in the captured scene, which is the bottom scene point cloud.

[0100] For example, taking the compressor as the part to be installed, after the grasping correction posture of the compressor is calculated, the robot can be controlled to grasp the compressor according to the grasping correction posture, so as to grasp it to the most ideal position preset in mid-air, that is, the fourth preset position, so that the acquisition device can scan the bottom surface of the compressor to obtain the scene point cloud of the compressor.

[0101] Preferably, after acquiring the scene point cloud of the part to be installed in the grabbing scene, it can be optionally filtered and / or downsampled to reduce the amount of data processing in subsequent steps, thereby saving resources and improving efficiency.

[0102] S5: Correcting the calibrated installation posture according to the bottom scene point cloud and the calibrated bottom point cloud to obtain a second installation correction posture;

[0103] Specifically, it is optional but not limited to the bottom scene point cloud of the part to be installed obtained in step S4 and the calibrated bottom point cloud of the part to be installed in step S1, and the transformation matrix between the two is calculated by the global feature point cloud registration algorithm, which is the third transformation matrix, and the calibrated installation posture is corrected according to the third transformation matrix to obtain the posture of the part to be installed when it is installed on the mounting plate, which is the second installation correction posture.

[0104] Preferably, the step of correcting the calibrated installation posture according to the bottom scene point cloud and the calibrated bottom point cloud to obtain a second installation correction posture may optionally include but is not limited to:

[0105] S51: performing point cloud registration on the bottom scene point cloud and the calibrated bottom point cloud to determine a third transformation matrix between the two;

[0106] Specifically, it is optional but not limited to calculating the rotation and translation matrix between the bottom scene point cloud and the calibrated bottom point cloud through a global feature point cloud registration algorithm as the third transformation matrix.

[0107] Preferably, since the global feature point cloud registration algorithm is mainly based on the global features of the point cloud and may not be able to capture local details, it is preferred to use an iterative closest point algorithm (IPC) to perform precise registration on the third transformation matrix to obtain an optimized third transformation matrix T jp .

[0108] S52: Correct the calibrated installation posture according to the third transformation matrix to obtain a second installation correction posture.

[0109] For example, it is optional but not limited to correcting the calibrated installation posture according to the correction posture calculation formula 5-1, correcting the calibrated installation posture, and obtaining the second installation correction posture:

[0110]

[0111] Among them, F rec is the second installation correction posture of the part to be installed, C p_m is the photo point of the correction model, that is, the position and posture of the robot when the acquisition device obtains the calibration bottom point cloud of the part to be installed, T jp is the third transformation matrix, is the camera external parameter, C p_c The current photo point is the robot's position when acquiring the bottom scene point cloud of the part to be installed.

[0112] S6: The first installation correction posture and the second installation correction posture are combined to obtain a final installation posture of the component to be installed.

[0113] Specifically, since the mounting plate and the part to be mounted will be offset to a certain extent during the process of installing the part to be mounted on the mounting plate, it is necessary to consider the influence of the displacement of the mounting plate and the displacement of the part to be mounted itself on the final installation posture when calculating the final installation posture, that is, it is optional but not limited to obtaining the first installation correction posture of the mounting plate after the mounting plate changes according to step S2, and the second installation correction posture of the part to be mounted after the calibrated grasping posture of the part to be mounted changes in step S5, and combining the first installation correction posture and the second installation correction posture to obtain the final installation posture of the part to be mounted.

[0114] For example, the calculation formula for the final installation posture of the part to be installed can be optionally but not limited to the formula 6-1:

[0115] F=F cur ·F rec 6-1

[0116] Among them, F is the final installation posture of the part to be installed, F cur is the first installation correction posture of the mounting plate, F rec Correct the posture of the second installation of the part to be installed.

[0117] In this embodiment, a collaborative assembly method based on point cloud registration of the present invention is provided, by determining the calibrated point cloud and calibrated placement posture of the mounting plate, the calibrated surface point cloud and calibrated grasping posture of the part to be mounted, and the calibrated bottom point cloud and calibrated installation posture of the part to be mounted, and then obtaining the current point cloud of the mounting plate, and correcting the calibrated placement posture according to the calibrated point cloud of the mounting plate and the current point cloud of the mounting plate to obtain a first installation correction posture. Since the mounting plate may be offset during installation, the offset of the mounting plate is first calculated to improve the installation accuracy of subsequent installation steps, and then the current point cloud of the part to be mounted is obtained, and the calibrated grasping posture is corrected according to the calibrated surface point cloud of the part to be mounted and the current point cloud of the part to be mounted to obtain the corrected actual grasping posture to avoid the placement position of the part to be mounted. In order to prevent the robot from making errors when grasping the parts to be installed due to the offset of the position, thus affecting the assembly efficiency, the robot can grasp the parts to be installed according to the actual grasping posture, and obtain the bottom scene point cloud of the parts to be installed in the grasping scene, and obtain the point cloud of the contact surface between the parts to be installed and the mounting plate to improve the accuracy of calculating the final installation posture of the parts to be installed, eliminate the influencing factors, and reduce the error. Then, the calibrated installation posture is corrected according to the bottom scene point cloud and the calibrated bottom point cloud to obtain the second installation correction posture. Finally, the first installation correction posture and the second installation correction posture are combined to obtain the final installation posture of the parts to be installed. By simultaneously combining the posture changes between the mounting plate and the parts to be installed, the assembly accuracy is improved, and the problem of reprogramming and adjustment of the system due to production needs and product upgrades is avoided. The problems of low assembly accuracy and poor adaptability of the prior art are solved.

[0118] On the other hand, the present invention further provides a computer storage medium storing executable program code; the executable program code is used to execute any of the above-mentioned collaborative assembly methods based on point cloud registration.

[0119] On the other hand, the present invention also provides a terminal device, including a memory and a processor; the memory stores program codes that can be executed by the processor; the program codes are used to execute any of the above-mentioned collaborative assembly methods based on point cloud registration.

[0120] Exemplarily, the program code may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the program code in the terminal device.

[0121] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the terminal device may also include an input / output device, a network access device, a bus, etc.

[0122] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0123] The memory may be an internal storage unit of the terminal device, such as a hard disk or a memory. The memory may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device. Further, the memory may also include both an internal storage unit of the terminal device and an external storage device. The memory is used to store the program code and other programs and data required by the terminal device. The memory may also be used to temporarily store data that has been output or is to be output.

[0124] The above-mentioned computer storage medium and terminal device are created based on the above-mentioned collaborative assembly method based on point cloud registration. Their technical functions and beneficial effects are no longer repeated here. The various technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the various technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A collaborative assembly method based on point cloud registration, characterized in that: include: Determine the calibration point cloud and calibration placement pose of the mounting plate, the calibration surface point cloud and calibration grasping pose of the part to be mounted, and the calibration bottom point cloud and calibration mounting pose of the part to be mounted; Acquire the current point cloud of the mounting plate, and calibrate the placement posture according to the calibration point cloud of the mounting plate and the current point cloud of the mounting plate to obtain a first installation correction posture; Acquire the current point cloud of the part to be installed, and correct the calibrated grasping posture according to the calibrated surface point cloud of the part to be installed and the current point cloud of the part to be installed to obtain the corrected actual grasping posture; Grasp the part to be installed according to the actual grasping posture, and obtain the bottom scene point cloud of the part to be installed in the grasping scene; Correcting the calibrated installation posture according to the bottom scene point cloud and the calibrated bottom point cloud to obtain a second installation correction posture; The final installation posture of the component to be installed is obtained by combining the first installation correction posture and the second installation correction posture.

2. The method according to claim 1, characterized in that: The steps of determining the calibration point cloud and the calibration placement pose of the mounting plate, the calibration surface point cloud and the calibration grasping pose of the part to be mounted, and the calibration bottom point cloud and the calibration mounting pose of the part to be mounted include: The mounting plate is placed at a first preset position, and the surface of the mounting plate is scanned to obtain a calibration point cloud and a calibration placement pose of the mounting plate; Placing the part to be installed at a second preset position, scanning the surface of the part to be installed, and obtaining a calibrated surface point cloud of the part to be installed; Control the robot to grab the part to be installed and record the current position and posture of the part to be installed to calibrate the grabbing position and posture; The part to be installed is placed at a third preset position on the installation plate, and the current point cloud and current posture of the bottom of the part to be installed are obtained to calibrate the bottom point cloud and the installation posture.

3. The method according to claim 1, characterized in that The step of obtaining a first installation correction posture includes: Obtain the current point cloud of the mounting plate, perform point cloud registration on the current point cloud and the calibration point cloud of the mounting plate, and determine a first transformation matrix between the two; The calibrated placement posture is corrected according to the first transformation matrix to obtain a first installation correction posture.

4. The method according to claim 3, characterized in that The step of obtaining the current point cloud of the mounting plate, performing point cloud registration on the current point cloud and the calibration point cloud of the mounting plate, and determining a first transformation matrix therebetween comprises: Calculate the feature descriptors of each point in the calibration point cloud of the mounting plate and the current point cloud of the mounting plate to construct a number of feature point pairs; Construct a least squares equation based on the feature point pairs to obtain a first transformation matrix; The first transformation matrix is ​​precisely aligned to obtain an optimized first transformation matrix.

5. The method according to claim 1, characterized in that The steps to obtain the grasping correction posture include: Obtain the current grasped point cloud of the part to be installed, and transform the current grasped point cloud into the robot base coordinate system according to the prior transformation matrix between the acquisition device and the robot; Performing point cloud registration on the calibrated surface point cloud of the installation part and the currently captured point cloud to determine a second transformation matrix between the two; The calibrated grasping posture is corrected according to the second transformation matrix to obtain the grasping correction posture.

6. The method according to claim 1, characterized in that The steps of obtaining the scene point cloud of the part to be installed include: Controlling the robot to grasp the part to be installed according to the grasping correction posture, and moving it to a fourth preset position; Scan the bottom surface of the part to be installed to obtain the point cloud of the part to be installed in the capture scene, which is the scene point cloud.

7. The method according to claim 1, characterized in that The step of obtaining a second installation correction posture includes: Perform point cloud registration on the scene point cloud and the calibration bottom point cloud to determine the third transformation matrix between the two; The calibrated installation posture is corrected according to the third transformation matrix to obtain a second installation correction posture.

8. The method according to claim 7, characterized in that The step of correcting the calibrated installation posture according to the third transformation matrix to obtain a second installation correction posture includes: Among them, F rec is the second installation correction posture of the part to be installed, C p_m T is the position and posture of the robot when the acquisition device obtains the calibrated bottom point cloud of the part to be installed. jp is the third transformation matrix, is the camera external parameter, C p_c The robot's position and posture when obtaining the bottom scene point cloud of the part to be installed.

9. A computer storage medium, characterized in that: An executable program code is stored; the executable program code is used to execute the collaborative assembly method based on point cloud registration as described in any one of claims 1 to 8.

10. A terminal device, characterized in that: It comprises a memory and a processor; the memory stores a program code executable by the processor; the program code is used to execute the collaborative assembly method based on point cloud registration as described in any one of claims 1-8.

Images