Calibration method, device and tool for multiple radars of material grabbing machine

Abstract

The invention relates to a calibration method, device and tool for multiple radars of a material grabbing machine, and belongs to the technical field of robots. The objective of the invention is to solve the problems of low precision and poor applicability of an existing multi-radar calibration method for the material grabbing machine. According to the invention, material grabbing machine pose information and laser radar point cloud data are fused, point cloud splicing, point cloud segmentation, rough matching and fine matching are carried out, and a transformation matrix from all radars to a reference radar is obtained, so that material grabbing machine multi-radar calibration is completed. According to the invention, the radar overlapping region limitation caused by a large material grabbing machine body can be effectively broken, and the calibration precision and reliability are improved; effective calibration can be carried out under complex terrains, and dependence on flat roads is avoided; any two radars can be directly calibrated, accumulative errors caused by cascade calibration are avoided, and therefore the overall calibration precision and accuracy are improved. The method is suitable for unmanned projects of the grabbing machine, can improve the overall performance of a sensing system, and has important practical value.

Description

Technical Field

[0001] The invention belongs to the technical field of robots and relates to a calibration method, device and tool for a multi-radar of a material grabber. Background Art

[0002] With the overall advancement of the unmanned project of the grabber, the perception technology and data quality are constantly improving, especially the higher requirements for the multi-radar calibration on the grabber. The accuracy of the calibration is crucial to the target recognition ability and stability of the subsequent perception algorithm.

[0003] However, current multi-radar calibration methods are mostly designed for unmanned vehicles, and have problems such as limited radar overlap area, measurement errors caused by terrain, and cumulative errors caused by cascade calibration. They cannot be directly used for large-scale engineering machinery such as grabbers.

[0004] Therefore, a technical solution for multi-radar calibration of the grabber is urgently needed to meet the data quality requirements of the subsequent perception algorithm and improve the overall performance of the perception system.

[0005] Multi-radar calibration: refers to the calibration and adjustment of the measurement results or data of multiple radars, which usually involves accurately locating the position and direction of each radar to ensure that they accurately measure and track targets in the same coordinate system to achieve the purpose of collaborative work.

[0006] Cascade Calibration: A complex calibration method in which one radar is calibrated to a third radar via another intermediate radar, involving multiple steps, each of which is used to transfer and adjust calibration parameters to ensure the accuracy of the final calibration.

[0007] CloudCompare: is a 3D point cloud processing software based on the GPL open source protocol, which can run on Windows, MacOS and Linux. It was originally designed for direct comparison of dense 3D point clouds, relying on a specific octree structure, and has excellent performance in tasks such as point cloud comparison. Summary of the invention

[0008] In view of this, the purpose of the present invention is to provide a method, device and tool for calibrating multiple radars of a material grabber. The material grabber posture information + laser radar point cloud data are integrated, and point cloud splicing, point cloud segmentation, coarse matching and fine matching are performed to obtain the transformation matrix from all radars to the reference radar, and complete the calibration of the multiple radars of the material grabber.

[0009] In order to achieve the above object, the present invention provides the following technical solutions:

[0010] A method for calibrating a multi-radar of a material grabber, the method comprising the following steps:

[0011] S1: Equipment installation: install the laser radar, rotary encoder and inclination sensor on the same material grabber. The laser radar is installed around the body of the material grabber, the rotary encoder is installed on the turntable, and the inclination sensor is installed on the chassis. Ensure that all sensors communicate with the industrial computer normally and the sensor time is aligned.

[0012] S2: Data collection: The calibration plate is lifted off the ground and placed still 10 meters away from the material grabber. The crawler of the material grabber does not move and rotates slowly and evenly at a speed of 1st gear for one circle. At the same time, the laser radar, rotary encoder, and inclination sensor collect data and save them on the industrial computer. The industrial computer obtains the position information of the material grabber obtained by the rotary encoder and inclination sensor, and obtains point cloud data by the laser radar.

[0013] S3: Data processing, integrating the gripper posture information and point cloud data, and performing point cloud stitching and point cloud segmentation to obtain point cloud data containing calibration plate objects in multiple radar local coordinate systems, where the radar local coordinate system is the coordinate system of the first frame of the radar when the rotation starts;

[0014] S4: Rough matching: select one of the multiple radars as the reference radar, use CloudCompare software for rough matching, prepare two frames of point cloud data of the reference radar and the radar to be calibrated, which only contain the calibration plate object, set them in different colors for easy observation, click the Align (point pairs picking) function button, select the reference radar as the reference variable, the radar to be calibrated as the align variable, select the four vertices of the calibration plate as the reference points, and calculate the transformation matrix, which contains the rotation matrix and the translation vector;

[0015] S5: Fine matching: take the rough matching result as the initial value, use the ICP algorithm for fine matching, calculate the transformation relationship between the radar to be calibrated and the reference radar coordinate system, and finally transform all radar data to the reference radar coordinate system according to the selected calibration parameters to realize multi-radar calibration.

[0016] Furthermore, in S3, the gripper position information includes rotation angle data collected by the rotary encoder and angle data between the gripper and the horizontal plane collected by the inclination sensor.

[0017] Furthermore, in S3, point cloud stitching refers to stitching point cloud data collected by multiple radars to obtain complete point cloud data.

[0018] Furthermore, in S3, point cloud segmentation refers to setting segmentation range parameters to obtain point cloud data containing only the calibration plate object to avoid noise interference caused by other dynamic objects in the scene.

[0019] Furthermore, in S4, rough matching refers to using CloudCompare software to perform rough matching of point clouds, selecting point cloud data containing only calibration plate objects of the reference radar and the radar to be calibrated, setting them to different colors for easy observation, selecting four vertices of the calibration plate as reference points, and calculating to obtain a transformation matrix, which includes a rotation matrix and a translation vector.

[0020] Furthermore, in S5, the precise matching refers to taking the rough matching result as the initial value, using the ICP algorithm to perform precise matching of the point cloud, calculating the transformation relationship from the radar to be calibrated to the reference radar coordinate system, and finally converting all radar data to the reference radar coordinate system according to the selected calibration parameters to realize multi-radar calibration.

[0021] Furthermore, in S5, the ICP algorithm refers to an iterative closest point algorithm, which is used to iteratively minimize the difference between point pairs, starting from the transformation matrix of the rough match, and gradually optimizing until the convergence condition is reached.

[0022] A multi-radar calibration device for a material grabber, the device comprising:

[0023] LiDAR is installed around the body of the grabber to collect environmental point cloud data;

[0024] The rotary encoder is installed on the turntable to collect the rotation angle data of the grabber;

[0025] The inclination sensor is installed on the chassis to collect the angle data between the grabber and the horizontal plane;

[0026] The industrial computer is connected to all sensors to receive and process the data collected by the above sensors; the industrial computer receives the data collected by the laser radar, rotary encoder and inclination sensor in real time; the industrial computer integrates the gripper's posture information and point cloud data, performs point cloud stitching, point cloud segmentation, coarse matching and fine matching operations, and finally obtains the transformation matrix from all radars to the reference radar to complete the multi-radar calibration.

[0027] Furthermore, the industrial computer also includes a GPS module for collecting position data of the gripper.

[0028] A calibration tool for a multi-radar of a material grabber, the tool comprising:

[0029] CloudCompare software performs rough matching of point clouds, specifically: CloudCompare software receives point cloud data of two frames of reference radar and radar to be calibrated that only contain calibration plate objects; CloudCompare software calculates the transformation matrix between the two frames of point clouds, including rotation matrix and translation vector, by letting the user manually select four vertices of the calibration plate as reference points; CloudCompare software outputs the rough matching result, i.e., the transformation matrix between the two frames of point clouds;

[0030] The ICP algorithm performs precise point cloud matching, specifically: the ICP algorithm receives the rough matching results output by the CloudCompare software as the initial transformation matrix, as well as the point cloud data of two frames of the reference radar and the radar to be calibrated; based on the initial transformation matrix, the ICP algorithm minimizes the difference between point pairs through the iterative closest point algorithm, and gradually optimizes the transformation matrix until the convergence condition is reached; the ICP algorithm calculates the final transformation matrix, which includes the rotation matrix and the translation vector, representing the transformation relationship between the radar to be calibrated and the reference radar coordinate system; the ICP algorithm outputs the final transformation matrix, that is, the transformation relationship between the radar to be calibrated and the reference radar coordinate system.

[0031] The beneficial effects of the present invention are:

[0032] (1) It can effectively overcome the limitation of the existing radar calibration method due to the small radar overlap area caused by the large body of the grabber, and significantly improve the accuracy and reliability of the multi-radar calibration of the grabber.

[0033] (2) It can effectively calibrate on complex terrains such as bumpy dirt roads or stone roads, avoiding dependence on flat roads. The multi-radar calibration of the grabber is no longer restricted by terrain, significantly improving the applicability and accuracy of the application in actual working environments.

[0034] (3) It is possible to directly calibrate any two of the multiple radars, avoiding the cumulative error caused by cascade calibration, thereby improving the overall calibration precision and accuracy.

[0035] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0037] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0038] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and the features in the embodiments can be combined with each other without conflict.

[0039] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0040] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0041] See also Figure 1 , a calibration method for a multi-radar of a material grabber, the method steps are:

[0042] S1: Equipment installation: Install the laser radar, rotary encoder, and inclination sensor on the same material grabber. The laser radar is installed around the body of the material grabber, the rotary encoder is installed on the turntable, and the inclination sensor is installed on the chassis. Ensure that all sensors communicate with the industrial computer normally and the sensor time is aligned;

[0043] S2: Data collection: The calibration plate is lifted off the ground and placed still at a distance of 10 meters from the material grabber. The crawler of the material grabber does not move and rotates slowly and evenly at a speed of 1st gear for one circle. At the same time, the laser radar, rotary encoder, and inclination sensor collect data and save them on the industrial computer. The industrial computer obtains the position information of the material grabber obtained by the rotary encoder and inclination sensor, and obtains point cloud data by the laser radar.

[0044] S3: Data processing: Fuse the gripper posture information and point cloud data and perform point cloud stitching and point cloud segmentation to obtain point cloud data containing calibration plate objects in multiple radar local coordinate systems, where the radar local coordinate system is the coordinate system of the first frame of the radar when the rotation starts;

[0045] S31: All radars are rigidly connected to the body of the grabber. When the radar rotates one circle around the center of the grabber, the distance from the radar to the center of the grabber can be measured as r through the CAD drawing. The rotary encoder can be used to know θ i =2π / N·i is the rotation angle corresponding to the uth frame, where N represents the number of frames. The angle between the gripper and the horizontal plane obtained by the inclination sensor is Then the translation of each axis of the radar at the i-th frame is:

[0046] T x = r cos(θ i )

[0047]

[0048]

[0049] For point P in the i-th frame point cloud data i =(x i ,y i ,z i ), first apply the rotation matrix R z (θ i ) is rotated, and then the translation vector T = (T x ,T y ,T z ) is translated. The transformed point P′ i It can be expressed as:

[0050] P′ i =R z (θ i )·P i +T

[0051] The rotation matrix R z (θ i ) is expressed as:

[0052]

[0053] S32: Through the above steps, point cloud data in multiple radar local coordinate systems can be obtained, and segmentation range parameters are further set to obtain point cloud data containing only calibration plate objects to avoid noise interference caused by other dynamic objects in the scene;

[0054] S4: Rough matching: Select one of the multiple radars as the reference radar, use CloudCompare software for rough matching, prepare two frames of point cloud data of the reference radar and the radar to be calibrated that only contain the calibration plate object, set them in different colors for easy observation, click the Align (point pairs picking) function button, select the reference radar as the reference variable, the radar to be calibrated as the align variable, select the four vertices of the calibration plate as the reference points, and calculate to obtain the transformation matrix H, which contains the rotation matrix and the translation vector;

[0055] S5: Fine matching: Use the rough matching result as the initial value, use the ICP algorithm for fine matching, calculate the conversion relationship between the radar to be calibrated and the reference radar coordinate system, and finally, according to the selected calibration parameters, convert all radar data to the reference radar coordinate system to achieve multi-radar calibration.

[0056] S51: Use the ICP algorithm for precise matching. The core of the algorithm is to iteratively minimize the difference between point pairs. Starting from the transformation matrix of the rough matching, the algorithm is gradually optimized until the convergence condition is reached. Assume that the point cloud data of the radar to be calibrated and the reference radar, including the calibration plate, are denoted as {P A} and {P B}, for each point cloud {P A} In the point cloud {P B Find the nearest point in The nearest point The set of

[0057] S52: Let {P′ A} is {P A The new position set of the midpoint after the rough matching transformation matrix H is transformed, and the covariance matrix of the corresponding points is calculated as

[0058]

[0059] in and They are {P′ A} and {P B}. Perform SVD decomposition on H and get H = U·S·V T , calculate the rotation matrix R = U·D·V T , where D is a diagonal matrix whose diagonal elements are [1,1,det(U·V T )], ensure that R is an orthogonal matrix and the determinant is 1;

[0060] S53: Solving for the translation vector Based on the obtained R and T, update the transformation matrix H = [R|P], and repeat the above steps until the number of iterations reaches the upper limit;

[0061] This method utilizes the rotatable feature of the grabber itself to extract the grabber's posture information and environmental point cloud data from the lidar, rotary encoder, and inclination sensor, and performs operations such as point cloud stitching, point cloud segmentation, coarse matching, and fine matching to obtain the transformation matrix from all radars to the benchmark radar, and complete the multi-radar calibration of the grabber.

[0062] Technical effects achieved:

[0063] It can effectively break the limitation of the existing radar calibration method due to the small radar overlapping area caused by the large body of the grabber, and significantly improve the accuracy and reliability of the multi-radar calibration of the grabber.

[0064] It can perform effective calibration on complex terrains such as bumpy dirt roads or stone roads, avoiding dependence on flat roads. It makes the multi-radar calibration of the grabber no longer restricted by terrain, significantly improving the application applicability and accuracy in actual working environments.

[0065] Any two of the multiple radars can be calibrated directly to avoid the cumulative error caused by cascade calibration, thereby improving the overall calibration precision and accuracy.

[0066] Example:

[0067] Equipment: Grabber equipped with 4 laser radars, 1 rotary encoder, 1 inclination sensor and 1 industrial computer.

[0068] Install four laser radars around the grabber body, the rotary encoder on the turntable, and the inclination sensor on the chassis to ensure that all sensors communicate with the industrial computer normally and the sensor time is aligned.

[0069] The calibration plate is lifted off the ground and placed still 10 meters away from the grabber. The crawler of the grabber does not move and rotates slowly and evenly at a speed of 1st gear for one circle. At the same time, the laser radar, rotary encoder and inclination sensor collect data and save it on the industrial computer.

[0070] The industrial computer integrates the gripper's posture information and point cloud data, performs point cloud stitching and segmentation, and obtains point cloud data containing calibration plate objects in multiple radar local coordinate systems.

[0071] CloudCompare software is used for rough matching. The four vertices of the calibration plate are selected as reference points to calculate the transformation matrix from the radar to be calibrated to the reference radar.

[0072] The ICP algorithm is used for precise matching, and the rough matching result is used as the initial value to calculate the final transformation matrix to achieve multi-radar calibration.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for calibrating a multi-radar of a material grabber, characterized in that: The method comprises the following steps: S1: Equipment installation: install the laser radar, rotary encoder and inclination sensor on the same material grabber. The laser radar is installed around the body of the material grabber, the rotary encoder is installed on the turntable, and the inclination sensor is installed on the chassis. Ensure that all sensors communicate with the industrial computer normally and the sensor time is aligned. S2: Data collection: The calibration plate is lifted off the ground and placed still 10 meters away from the material grabber. The crawler of the material grabber does not move and rotates slowly and evenly at a speed of 1st gear for one circle. At the same time, the laser radar, rotary encoder, and inclination sensor collect data and save them on the industrial computer. The industrial computer obtains the position information of the material grabber obtained by the rotary encoder and inclination sensor, and obtains point cloud data by the laser radar. S3: Data processing, integrating the gripper posture information and point cloud data, and performing point cloud stitching and point cloud segmentation to obtain point cloud data containing calibration plate objects in multiple radar local coordinate systems, where the radar local coordinate system is the coordinate system of the first frame of the radar when the rotation starts; S4: Rough matching: select one of the multiple radars as the reference radar, use CloudCompare software for rough matching, prepare two frames of point cloud data of the reference radar and the radar to be calibrated, which only contain the calibration plate object, set them in different colors for easy observation, click the Align (point pairs picking) function button, select the reference radar as the reference variable, the radar to be calibrated as the align variable, select the four vertices of the calibration plate as the reference points, and calculate the transformation matrix, which contains the rotation matrix and the translation vector; S5: Fine matching: take the rough matching result as the initial value, use the ICP algorithm for fine matching, calculate the transformation relationship between the radar to be calibrated and the reference radar coordinate system, and finally transform all radar data to the reference radar coordinate system according to the selected calibration parameters to realize multi-radar calibration.

2. The calibration method of the multi-radar of the material grabber according to claim 1, characterized in that: In S3, the gripper posture information includes the rotation angle data collected by the rotary encoder and the angle data between the gripper and the horizontal plane collected by the inclination sensor.

3. The calibration method of the grabber multi-radar according to claim 1, characterized in that: In S3, point cloud stitching refers to stitching point cloud data collected by multiple radars to obtain complete point cloud data.

4. The calibration method of the grabber multi-radar according to claim 1, characterized in that: In S3, point cloud segmentation refers to setting segmentation range parameters to obtain point cloud data containing only the calibration plate object to avoid noise interference caused by other dynamic objects in the scene.

5. The calibration method of the grabber multi-radar according to claim 1, characterized in that: In S4, rough matching refers to using CloudCompare software to perform rough matching of point clouds, selecting point cloud data containing only calibration plate objects of the reference radar and the radar to be calibrated, setting them to different colors for easy observation, selecting four vertices of the calibration plate as reference points, and calculating to obtain a transformation matrix, which includes a rotation matrix and a translation vector.

6. The method for calibrating a multi-radar of a material grabber according to claim 1, characterized in that: In S5, the precise matching refers to taking the rough matching result as the initial value, using the ICP algorithm to perform precise matching of the point cloud, calculating the transformation relationship between the radar to be calibrated and the reference radar coordinate system, and finally converting all radar data to the reference radar coordinate system according to the selected calibration parameters to realize multi-radar calibration.

7. The method for calibrating a multi-radar of a material grabber according to claim 1, characterized in that: In S5, the ICP algorithm refers to an iterative closest point algorithm, which is used to iteratively minimize the difference between point pairs, starting from the transformation matrix of the rough match, and gradually optimizing until the convergence condition is reached.

8. A multi-radar calibration device for a material grabber, characterized in that: The device includes: LiDAR is installed around the body of the grabber to collect environmental point cloud data; The rotary encoder is installed on the turntable to collect the rotation angle data of the grabber; The inclination sensor is installed on the chassis to collect the angle data between the grabber and the horizontal plane; The industrial computer is connected to all sensors to receive and process the data collected by the above sensors; the industrial computer receives the data collected by the laser radar, rotary encoder and inclination sensor in real time; the industrial computer integrates the gripper's posture information and point cloud data, performs point cloud stitching, point cloud segmentation, coarse matching and fine matching operations, and finally obtains the transformation matrix from all radars to the reference radar to complete the multi-radar calibration.

9. The multi-radar calibration device for a material grabber according to claim 8, characterized in that: The industrial computer also includes a GPS module for collecting position data of the material grabber.

10. A calibration tool for multi-radar of a material grabber, characterized by: The tool includes: CloudCompare software performs rough matching of point clouds, specifically: CloudCompare software receives point cloud data of two frames of reference radar and radar to be calibrated that only contain calibration plate objects; CloudCompare software calculates the transformation matrix between the two frames of point clouds, including rotation matrix and translation vector, by letting the user manually select four vertices of the calibration plate as reference points; CloudCompare software outputs the rough matching result, i.e., the transformation matrix between the two frames of point clouds; The ICP algorithm performs precise point cloud matching, specifically: the ICP algorithm receives the rough matching results output by the CloudCompare software as the initial transformation matrix, as well as the point cloud data of two frames of the reference radar and the radar to be calibrated; based on the initial transformation matrix, the ICP algorithm minimizes the difference between point pairs through the iterative closest point algorithm, and gradually optimizes the transformation matrix until the convergence condition is reached; the ICP algorithm calculates the final transformation matrix, which includes the rotation matrix and the translation vector, representing the transformation relationship between the radar to be calibrated and the reference radar coordinate system; the ICP algorithm outputs the final transformation matrix, that is, the transformation relationship between the radar to be calibrated and the reference radar coordinate system.

Images