Calibration method and calibration device for laser radar and storage medium

Abstract

The invention provides a calibration method for a laser radar, and the method comprises the following steps: S11, obtaining the point cloud data of a point cloud detected by the laser radar for a calibration plate, the main plane of the calibration plate having a calibration pattern and facing the laser radar; s12, determining initial pose information of the calibration plate in the laser radar coordinate system based on the point cloud data; and S13, correcting the initial pose information by using the calibration pattern and the intensity information of the point cloud data so as to determine a first conversion matrix between the laser radar coordinate system and the calibration plate coordinate system, and determining calibration parameters for the laser radar based on the first conversion matrix. The invention further provides a corresponding calibration device and a computer readable storage medium. According to the invention, accurate calibration of the laser radar can be realized.

Description

Technical Field

[0001] The present invention relates to a calibration method for a laser radar, a calibration device for a laser radar and a computer-readable storage medium. Background Art

[0002] LiDAR is widely used in autonomous driving, 3D reconstruction, navigation, target detection and other fields. LiDAR can obtain 3D spatial information of the detected object. During the use of LiDAR, it is usually necessary to calibrate the LiDAR. However, existing calibration methods have problems such as inconvenient operation, inaccuracy or instability.

[0003] For example, key features for calibration can be selected from the point cloud data detected by the laser radar, and calibration can be achieved based on the position information of the detected key features and the prior knowledge of the key features. The calibration result obtained in this way is difficult to ensure accuracy. For example, the detection of key features may have errors due to the resolution of the laser radar. In addition, it is also known that the operator selects key features for calibration from the point cloud data detected by the laser radar in order to achieve calibration. This manual operation makes the calibration process complicated and time-consuming, and the accuracy of the calibration result depends on the operator's experience.

[0004] It is therefore desirable to provide an improved calibration method for lidar. Summary of the invention

[0005] The object of the present invention is to provide an improved calibration method for laser radar so as to achieve accurate calibration.

[0006] According to a first aspect of the present invention, a calibration method for a laser radar is provided, wherein the calibration method comprises the following steps: S11, acquiring point cloud data about a point cloud detected by the laser radar for a calibration plate, wherein a main plane of the calibration plate has a calibration pattern and faces the laser radar; S12, determining initial pose information of the calibration plate in a laser radar coordinate system based on the point cloud data; S13, correcting the initial pose information using intensity information of the calibration pattern and the point cloud data, so as to determine a first transformation matrix between the laser radar coordinate system and the calibration plate coordinate system, and determining calibration parameters for the laser radar based on the first transformation matrix.

[0007] By using the intensity information of the calibration pattern and the point cloud data to correct the initial pose information, the accuracy of the calibration result can be improved. The intensity information can reflect the reflectivity of the scanned target. By comparing the intensity information of the point cloud data with the scanned calibration pattern, the initial pose information can be corrected, thereby improving the accuracy of the calibration. In particular, accurate calibration of the laser radar can be achieved in a convenient manner, wherein the calibration process can be automatically performed with the aid of a calibration device, thereby at least reducing or even eliminating manual operations by the operator during the calibration process, such as the operation of the operator selecting features for calibration. Using the intensity information and the calibration pattern to correct the initial pose information can enable this highly automated calibration to still obtain highly accurate calibration parameters.

[0008] According to an exemplary embodiment of the present invention, step S13 may include: S131, using the intensity information of the point cloud data to project the point cloud representing the calibration plate along the main direction perpendicular to the main plane of the calibration plate determined by the initial pose information, so as to generate a projection image; and S132, performing image matching on the projection image and the template image of the calibration pattern and correcting the initial pose information according to the matching result of the image matching, so as to determine the first transformation matrix between the lidar coordinate system and the calibration plate coordinate system.

[0009] The generated projection image can reflect the intensity information of the point cloud data. In the image matching process, the projection image of the entire calibration plate is matched with the template image, which can make full use of the large number of point clouds representing the calibration plate, rather than relying only on a small number of point clouds associated with key features. The result of image matching is basically not adversely affected by the resolution of the lidar, and can make full use of the intensity information of the point cloud data. As a result, the calibration method can be more robust and easy to implement.

[0010] According to an exemplary embodiment of the present invention, in step S132, image matching is performed on the projection image and the template image of the calibration pattern in at least one of the following ways: using the normalized cross-correlation matching method to perform image matching on the projection image and the template image of the calibration pattern; and / or, the projection image is matched with at least two image parts with different grayscale values ​​in the template image to determine at least two corresponding sub-correlation coefficients between the projection image and the at least two image parts, and a correlation coefficient characterizing the degree of matching between the projection image and the template image is determined based on the at least two sub-correlation coefficients; and / or, the template image is rotated in its image plane, and image matching is performed on the projection image and the rotated template image to determine the rotation angle of the template image with the highest matching degree with the projection image, and the relative posture between the projection image and the template image is determined according to the rotation angle.

[0011] According to an exemplary embodiment of the present invention, in step S11, the initial point cloud detected by the laser radar may be segmented to obtain at least one candidate point cloud cluster, and a target point cloud cluster may be selected from the at least one candidate point cloud cluster as the point cloud cluster corresponding to the calibration plate. Optionally, the point cloud points in each candidate point cloud cluster may be divided into at least two groups according to intensity information, and the target point cloud cluster may be selected according to the ratio of the number of point cloud points in the at least two groups. Alternatively or additionally, the target point cloud cluster may be selected according to the ratio of the sizes of the bounding boxes of each candidate point cloud cluster along at least two different directions.

[0012] According to a second aspect of the present invention, a calibration device for a laser radar is provided, wherein the calibration device comprises: one or more processors; and a memory storing computer program instructions, which, when executed by the one or more processors, enable the one or more processors to execute the calibration method according to the present invention.

[0013] According to a third aspect of the present invention, a computer-readable storage medium is provided, which stores computer program instructions, wherein the computer program instructions, when executed by one or more processors, enable the one or more processors to perform the calibration method according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present invention will be described in more detail below with reference to the accompanying drawings, so that the principles, features and advantages of the present invention can be better understood. The accompanying drawings include:

[0015] Figure 1 The diagram schematically shows a calibration scenario using a calibration method according to an exemplary embodiment of the present invention;

[0016] Figure 2 The flowchart of a calibration method for a laser radar according to an exemplary embodiment of the present invention is schematically shown;

[0017] Figure 3 The flowchart of step S11 in the calibration method according to an exemplary embodiment of the present invention is schematically shown;

[0018] Figure 4 Schematically showing a point cloud included in the initial point cloud data acquired by step S11 in the calibration method according to an exemplary embodiment of the present invention;

[0019] Figure 5 Schematically shows the candidate point cloud clustering obtained by step S11 in the calibration method according to an exemplary embodiment of the present invention;

[0020] Figure 6Schematically illustrating step S12 in the calibration method according to an exemplary embodiment of the present invention;

[0021] Figure 7 and Figure 8 Schematically illustrating step S13 of the calibration method according to an exemplary embodiment of the present invention;

[0022] Fig. 9 A scenario of performing calibration using a calibration method according to an exemplary embodiment of the present invention is schematically shown; and

[0023] Fig.10 A flow chart of a calibration method for a laser radar according to an exemplary embodiment of the present invention is schematically shown.

[0024] Reference numerals list

[0025] 1 LiDAR

[0026] 2 Calibration plate

[0027] 20 represents the point cloud of the calibration plate

[0028] 21 Calibration pattern

[0029] 3 Calibration device

[0030] 31 Processor

[0031] 32 Memory

[0032] 4 Additional sensors

[0033] 101 candidate point cloud clustering

[0034] 102 Target point cloud clustering

[0035] 103 Calibration board OBB

[0036] 104 Projected Image

[0037] 105 Template Images

[0038] 106 Rotated template image

[0039] 107 First image part

[0040] 108 Second image part DETAILED DESCRIPTION

[0041] In order to make the technical problems, technical solutions and beneficial technical effects to be solved by the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and multiple exemplary embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the scope of protection of the present invention.

[0042] It should be understood that, in this document, the expressions "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance, nor should they be understood as implicitly indicating the number of technical features indicated. Features defined as "first" or "second" may explicitly or implicitly indicate that at least one of the features is included. In this document, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0043] Figure 1 A scenario of performing calibration using a calibration method according to an exemplary embodiment of the present invention is schematically shown. Figure 2 The flowchart of the calibration method for the laser radar 1 according to an exemplary embodiment of the present invention is schematically shown. The laser radar 1 can be installed on various devices such as a vehicle or a road test device. The vehicle may include a vehicle for transporting goods or people, such as a motor vehicle, a rail vehicle, a ship, an aircraft, etc. The road test device may include, for example, a street lamp, a signboard or other dedicated roadside detection equipment.

[0044] like Figure 1 As shown, in order to calibrate the laser radar 1, a calibration plate 2 is placed in the field of view of the laser radar 1. The main plane of the calibration plate 2 has a calibration pattern 21 and faces the laser radar 1, so that the laser radar 1 can detect the calibration plate 2 and its calibration pattern 21. Here, the main plane of the calibration plate 2 faces the laser radar 1, but the main plane of the calibration plate 2 does not have to be perpendicular to the main observation direction of the laser radar 1. Figure 1 In the illustrated embodiment, the calibration pattern 21 is a black and white checkerboard pattern. It should be understood that the calibration pattern 21 may also be implemented in other forms. For example, the calibration pattern 21 may include geometric patterns such as triangles, trapezoids, rectangles, circles, or more complex graphics.

[0045] Then, the laser radar 1 can detect the calibration plate 2 and obtain corresponding point cloud data. The point cloud data measured by the laser radar 1 can be transmitted to the calibration device 3 for the laser radar 1. The calibration device 3 is configured to be able to communicate with the laser radar 1 to achieve the transmission of data and / or control instructions. The point cloud data measured by the laser radar 1 can also be transmitted to the calibration device 3 with the aid of a data carrier. The calibration device 3 includes: one or more processors 31; and a memory 32, which stores computer program instructions, which, when executed by the one or more processors 31, enable the one or more processors 31 to execute the calibration method according to the present invention.

[0046] like Figure 2 As shown, the calibration method at least includes step S11, step S12 and step S13.

[0047] First, in step S11, point cloud data about a point cloud detected by the laser radar 1 with respect to the calibration plate 2 is obtained. The point cloud data detected by the laser radar 1 includes not only coordinate information of each point in the point cloud, but also intensity information of each point.

[0048] Then, in step S12, based on the point cloud data, the initial position and posture information of the calibration plate 2 in the laser radar coordinate system is determined.

[0049] In step S13, the initial pose information is corrected using the calibration pattern 21 and the intensity information of the point cloud data to determine the first conversion matrix between the laser radar coordinate system and the calibration plate coordinate system, and the calibration parameters for the laser radar 1 are determined based on the first conversion matrix. The first conversion matrix may include information about the first rotation matrix and the first translation vector. The first conversion matrix can be directly used as a calibration parameter of the laser radar 1, for example, to calibrate the laser radar 1.

[0050] Thus, the laser radar 1 can be accurately calibrated in a convenient manner. The calibration process can be automatically performed with the aid of the calibration device 3, thereby at least reducing or even eliminating the manual operation of the operator during the calibration process, such as the operation of the operator selecting features for calibration. By correcting the initial pose information using the intensity information and the calibration pattern 21, this highly automated calibration can still obtain highly accurate calibration parameters.

[0051] Generally, the resolution of the laser radar 1 is limited, especially in the vertical direction. For example, when the vertical angular resolution of the laser radar 1 is 1°, if the calibration plate 2 is 5 meters away from the laser radar 1, the accuracy of the detection result of the laser radar 1 on the calibration plate 2 in the vertical direction will be greater than 5 cm. The sparsity of the point cloud measured by the laser radar 1 will affect the accuracy of the calibration. The existing calibration method usually relies on the position information of the point cloud data and the geometric features of the calibration plate 2 to achieve calibration, but it is often difficult to ensure the accuracy of the calibration. In the existing calibration method, calibration can be achieved with the help of additional manual operation to improve the accuracy of the calibration. For example, the operator selects the key features for calibration from the point cloud data detected by the laser radar 1, such as selecting the corner points on the calibration plate 2 from the point cloud data. This manual operation can improve the accuracy of the calibration to a certain extent, but on the other hand, it will also make the calibration process complicated and time-consuming, and rely on the experience of the operator. In addition, in the process of calibrating the laser radar 1 based on key features, if the detection data representing the key features themselves have low accuracy, the calibration results obtained are often inaccurate. In particular, the measurement results of such key features may be inaccurate due to the resolution of the laser radar 1. Therefore, even with the help of manual operation, the accuracy of the calibration results cannot be guaranteed.

[0052] According to the present invention, after determining the initial pose information of the calibration plate 2 in the laser radar coordinate system, the initial pose information can be corrected using the intensity information of the calibration pattern 21 and the point cloud data, thereby obtaining more accurate calibration parameters. The intensity information can reflect the reflectivity of the scanned target. In particular, when the laser radar 1 scans the calibration plate 2, the intensity information of the point cloud representing the main plane of the calibration plate 2 is highly correlated with the color of the main plane. For example, the white area of ​​the calibration pattern 21 has a higher reflectivity to the laser, and the echo intensity of the laser pulse scanning the white area is also higher, and the corresponding generated point cloud will also have a higher intensity. In contrast, the point cloud corresponding to the black area of ​​the calibration pattern 21 will have a lower intensity. Therefore, by comparing the intensity information of the point cloud data with the calibration pattern 21, the initial pose information can be corrected, thereby improving the accuracy of the calibration.

[0053] Optionally, in step S12, the initial pose information of the calibration plate 2 in the laser radar coordinate system may be determined based on the coordinate information contained in the point cloud data, without considering the intensity information contained in the point cloud data.

[0054] The laser radar 1 may be a non-repetitive scanning laser radar. Compared with a conventional repetitive scanning laser radar, the scanning path of a non-repetitive scanning laser radar may not be repeated, the laser scanning area increases over time, and the field of view coverage rate significantly increases over time. For a non-repetitive scanning laser radar, the calibration method according to an exemplary embodiment of the present invention is particularly helpful in improving the accuracy of the calibration result. In particular, the point cloud data detected by the laser radar 1 may have richer and more reliable intensity information.

[0055] Figure 3 The flowchart of step S11 in the calibration method according to an exemplary embodiment of the present invention is schematically shown.

[0056] like Figure 3 As shown, step S11 of the calibration method may include at least one of step S111, step S112 and step S113.

[0057] In step S111 , initial point cloud data detected by the laser radar 1 may be acquired. Figure 4 The point cloud included in the initial point cloud data obtained by step S11 in the calibration method according to an exemplary embodiment of the present invention is schematically shown. It can be seen that the initial point cloud data may include a point cloud 20 representing the calibration plate 2 and a point cloud representing the surrounding environment of the calibration plate 2. For example, in addition to the point cloud 20 representing the calibration plate 2, the initial point cloud data also includes a point cloud representing the ground and a point cloud representing an interference object.

[0058] In step S112, point clouds representing the ground may be removed from the initial point cloud data. For example, a sac-based segmentation method may be used to process the initial point cloud data so as to filter out point clouds representing the ground.

[0059] In step S113, a clustering algorithm may be used to segment the initial point cloud data, especially the initial point cloud data from which the point cloud representing the ground has been removed, to obtain at least one candidate point cloud cluster. For example, a Euclidean clustering method may be used. It should be understood that various applicable clustering algorithms may be used for clustering in step S113, such as a DBSCAN clustering method, a region growth clustering method, a K-means clustering method, etc. Figure 5 The candidate point cloud cluster 101 obtained by step S11 in the calibration method according to an exemplary embodiment of the present invention is schematically shown. Figure 5 , two candidate point cloud clusters 101 are schematically shown.

[0060] Removing the point cloud representing the ground in step S112 before step S113 is particularly beneficial to improving the efficiency and accuracy of point cloud segmentation in step S113.

[0061] In step S114 , a target point cloud cluster 102 may be selected from the at least one candidate point cloud cluster 101 as the point cloud cluster corresponding to the calibration plate 2 .

[0062] For example, the target point cloud cluster 102 may be selected according to the area occupied by the point cloud cluster. Since the size information of the calibration plate 2 is known, point cloud clusters corresponding to interference objects whose size is obviously larger or smaller than the calibration plate 2 may be screened out from the candidate point cloud clusters 101 .

[0063] In an exemplary embodiment, the target point cloud cluster 102 may be selected based on the ratio of the sizes of the bounding boxes of the candidate point cloud clusters 101 along at least two different directions. Since the thickness of the calibration plate 2 is significantly smaller than its size in a direction perpendicular to the thickness direction, this feature of the calibration plate 2 may be used to select the target point cloud cluster 102.

[0064] For example, in step S11, especially in step S114, the oriented bounding box (OBB) of each candidate point cloud cluster 101 can be obtained. The OBB can tightly surround the point cloud contained in each candidate point cloud cluster 101. For the calibration plate 2 in the shape of a rectangular parallelepiped, the OBB can closely approximate the calibration plate 2. The OBB representing the point cloud cluster of the calibration plate 2 can accurately match the shape of the calibration plate 2. The OBB can be used to accurately select the point cloud cluster corresponding to the calibration plate 2 from the candidate point cloud cluster 101. In addition, the OBB can also be used to determine the initial pose information of the calibration plate 2 in the lidar coordinate system (to be further described below).

[0065] Optionally, the OBB of the point cloud included in the point cloud clustering can be constructed based on the principal component analysis (PCA). By constructing the OBB, the three axial directions D1, D2, D3 of the OBB and the three side lengths L1, L2 and L3 along the three axial directions can be determined, wherein the three axial directions D1, D2, D3 are orthogonal to each other. For the calibration plate 2 having a rectangular parallelepiped shape, the three side lengths L1, L2 and L3 of the corresponding OBB can at least approximately represent the length, width and thickness of the calibration plate 2.

[0066] Exemplarily, the selected target point cloud cluster 102 at least satisfies the following conditions: the ratio of the minimum side length to the maximum side length of its OBB is less than 0.1. If among the three side lengths L1, L2 and L3 of the OBB of the candidate point cloud cluster 101, L1 is the smallest and L3 is the largest, then the candidate point cloud cluster 101 can be selected as the target point cloud cluster 102 only when L1:L3<0.1.

[0067] Alternatively, the selected target point cloud cluster 102 at least satisfies the following condition: one of the three side lengths of its OBB is less than one tenth of the other two side lengths. For example, among the three side lengths L1, L2 and L3 of the OBB of the target point cloud cluster 102, L1<0.1*L2 and L1<0.1*L3.

[0068] Thus, the target point cloud cluster 102 can be selected in a convenient manner.

[0069] In an exemplary embodiment, the point cloud points in each candidate point cloud cluster 101 may be divided into at least two groups according to intensity information, and the target point cloud cluster 102 may be selected according to the ratio of the number of point cloud points in the at least two groups.

[0070] For example, when the calibration pattern 21 is a checkerboard pattern, the area of ​​the black area and the white area of ​​the calibration plate 2 are approximately equal. Accordingly, the selected target point cloud cluster 102 can be made to at least meet the following conditions: the point cloud points in the point cloud cluster are divided into a high-intensity group and a low-intensity group according to the intensity information, and the ratio of the number of point cloud points in the high-intensity group to the number of point cloud points in the low-intensity group is between 0.5 and 2. Optionally, the K-means clustering algorithm (K-means algorithm) can be used to divide the point cloud points in the point cloud cluster into two parts, namely, a high-intensity group with higher intensity and a low-intensity group with lower intensity. The number of point cloud points in the high-intensity group is N. h , the number of point cloud points in the low intensity group is N l Only in 0.5 <N l / N h <2, the candidate point cloud cluster 101 can be selected as the target point cloud cluster 102. Therefore, the target point cloud cluster 102 can be selected with the help of intensity information, which helps to eliminate interference. In the process of selecting the target point cloud cluster 102, comprehensive consideration of intensity information and size information (for example, combining the above-mentioned method of selecting the target point cloud cluster 102 based on the ratio of the side lengths of the OBB and the ratio of the number of point cloud points in the at least two groups) can be particularly effective in eliminating interference, so that the calibration method can be applied to more diverse and complex scenes.

[0071] According to the calibration pattern 21 , the point cloud points in each candidate point cloud cluster 101 may also be divided into more than two groups according to the intensity information. The condition that the number of point cloud points in each group needs to meet may be determined according to the calibration pattern 21 .

[0072] According to an exemplary embodiment, in step S12, the initial pose information of the calibration plate 2 in the laser radar coordinate system may be determined according to the directed bounding box representing the point cloud of the calibration plate 2. The initial pose information may, for example, represent the position coordinates and direction vector of the calibration plate 2 in the laser radar coordinate system. Figure 6Step S12 in the calibration method according to an exemplary embodiment of the present invention is schematically shown.

[0073] For example, an OBB (also referred to herein as calibration plate OBB103) representing the point cloud of the calibration plate 2 can be constructed based on PCA. Thus, the center point coordinates P, the three axial directions D1, D2, D3, and the three side lengths L1, L2, and L3 along the three axial directions of the calibration plate OBB 103 can be determined. The center point coordinates and the three axial directions of the calibration plate OBB 103 can represent the initial position and posture of the calibration plate 2 in the laser radar coordinate system. Through the three side lengths L1, L2, and L3, the three axial directions D1, D2, and D3 of the calibration plate OBB 103 can be associated with the length direction, width direction, and thickness direction of the calibration plate 2. According to the center point coordinates, the three axial directions, and the three side lengths of the calibration plate OBB 103, the initial position and posture information of the calibration plate 2 in the laser radar coordinate system can be determined.

[0074] The initial pose information determined based on the calibration plate OBB 103 is determined based on the point cloud representing the calibration plate 2 as a whole. Compared with relying on key features to determine the pose of the calibration plate 2 in the laser radar coordinate system, this method has higher robustness. In particular, it can reduce the impact of the resolution of the laser radar 1.

[0075] As described above, in some embodiments, the OBB of each candidate point cloud cluster 101 may be obtained in step S11 in order to select the target point cloud cluster 102. Then, in step S12, the OBB of the target point cloud cluster 102 obtained in step S11 may be further used to determine the initial pose information of the calibration plate 2 in the laser radar coordinate system.

[0076] The initial position information can be used to roughly determine the transformation relationship between the laser radar coordinate system and the calibration plate coordinate system. For example, the initial transformation matrix T between the laser radar coordinate system and the calibration plate coordinate system can be determined. ini Alternatively, the initial pose information can be directly expressed as the initial transformation matrix T between the laser radar coordinate system and the calibration plate coordinate system ini However, in order to obtain accurate calibration results, in step S13, the initial pose information can be corrected using the calibration pattern 21 and the intensity information of the point cloud data, so as to determine the first transformation matrix between the laser radar coordinate system and the calibration plate coordinate system.

[0077] Back to Figure 2In an exemplary embodiment, step S13 may include, in particular: step S131, projecting the point cloud representing the calibration plate 2 along the main direction perpendicular to the main plane of the calibration plate 2 determined by the initial pose information using the intensity information of the point cloud data, so as to generate a projection image; and step S132, performing image matching between the projection image and the template image of the calibration pattern 21 and correcting the initial pose information according to the matching result of the image matching, so as to determine the first transformation matrix between the laser radar coordinate system and the calibration plate coordinate system. The template image of the calibration pattern 21 may be understood as an image that best reflects the calibration pattern 21. The template image of the calibration pattern 21 may be pre-stored in the memory 32 of the calibration device 3, or may be pre-stored in other storage devices and may be obtained by the calibration device 3.

[0078] Therefore, by matching the projection image of the point cloud representing the calibration plate 2 with the template image, the initial posture information of the calibration plate 2 can be corrected, thereby obtaining more accurate calibration parameters. The generated projection image can reflect the intensity information of the point cloud data. In the image matching process, the projection image of the entire calibration plate 2 is matched with the template image, which can make full use of the large number of point clouds representing the calibration plate 2, rather than relying only on a small number of point clouds associated with key features. The result of image matching is basically not adversely affected by the resolution of the laser radar 1, and the intensity information of the point cloud data can be fully utilized. As a result, the calibration can be made more robust and easy to implement.

[0079] Reference Figure 6 , the calibration plate OBB 103 has a minimum dimension L1 in the axial direction D1. It can be seen that D1 corresponds to the thickness direction of the calibration plate 2, that is, the main direction perpendicular to the main plane of the calibration plate 2. In step S131, the point cloud representing the calibration plate 2 can be projected along the direction D1 using the intensity information of the point cloud data to obtain a projection image.

[0080] Figure 7 and Figure 8 The step S13 of the calibration method according to an exemplary embodiment of the present invention is schematically shown. In this embodiment, the calibration pattern 21 is a checkerboard pattern.

[0081] Optionally, in step S131, a point with lower intensity in the point cloud representing the calibration plate 2 corresponds to an image unit with lower brightness or higher grayscale value in the projected image 104, and a point with higher intensity in the point cloud representing the calibration plate 2 corresponds to an image unit with higher brightness or lower grayscale value in the projected image 104. The image unit may include, for example, a pixel or a color block. Exemplarily, the point with the lowest intensity in the point cloud representing the calibration plate 2 may be projected as a white pixel or color block, and the point with the highest intensity in the point cloud representing the calibration plate 2 may be projected as a black pixel or color block. Figure 7Schematically shows a projection image 104 generated in an exemplary embodiment of the present invention. The projection image 104 may have a resolution consistent with the template image 105 of the calibration pattern 21. In the case where the laser radar 1 is a non-repetitive scanning laser radar, the projection image 104 may have richer and more reliable brightness / grayscale information.

[0082] Optionally, in step S132, the relative position between the projection image 104 and the template image 105 can be determined by performing image matching between the projection image 104 and the template image 105 of the calibration pattern 21. For example, the normalized cross-correlation matching method (NCC) can be used to perform image matching between the projection image 104 and the template image 105 of the calibration pattern 21. Thus, the correlation coefficient S between the projection image 104 and the template image 105 can be obtained. NCC . Generally, there may be a certain deviation between the projected image 104 and the initial template image 105. Optionally, the template image 105 may be rotated in its image plane, and image matching is performed between the projected image 104 and the rotated template image 106, so as to determine the rotation angle of the template image 105 with the highest matching degree with the projected image 104. The relative position between the projected image 104 and the template image 105 may be determined according to the rotation angle.

[0083] like Figure 7 As shown, a brute force search method can be used to determine the rotation angle of the template image 105 that has the highest matching degree with the projection image 104. For example, firstly, the correlation coefficient S between the projection image 104 and the template image 105 is obtained based on the NCC. NCC1 Then, the projection image 104 may be matched with the template image 105 rotated clockwise by Δθ degrees in the image plane to obtain the correlation coefficient S between the projection image 104 and the rotated template image 106. NCC2 If S NCC2 >S NCC1 , it means that compared with the initial template image 105, the template image 106 rotated by Δθ has a higher matching degree with the projection image 104. Similarly, the steps of rotating the template image 105 and obtaining the correlation coefficient are repeated. By comparing the correlation coefficients, the rotation angle θ of the template image 105 (initial or rotated) with the highest matching degree with the projection image 104 can be determined.

[0084] According to the image matching result with the highest matching degree, the position offset of the calibration plate 2 in the projection coordinate system can be further determined. For example, the position offset of the calibration plate 2 in the projection coordinate system can be determined by the following formula:

[0085]

[0086] In the above formula, u' and v' represent the position offset of the calibration plate 2 in the projection coordinate system. x and c y Indicates the coordinates of the center point. opt and v opt represents the optimal pixel position determined by NCC. θ represents the rotation angle of the template image 105 that has the highest matching degree with the projection image 104.

[0087] Then, the initial posture information can be corrected using the relative posture between the projection image 104 and the template image 105. For example, the first transformation matrix L T b It can be determined by the following formula:

[0088]

[0089] Among them, T ini represents the initial transformation matrix, θ represents the rotation angle of the template image 105 that has the highest matching degree with the projection image 104 , u′ and v′ represent the position offset of the calibration plate 2 in the projection coordinate system, and s represents the pixel size of the template image 105 .

[0090] Generally, the number of point cloud points obtained by the laser radar 1 from detecting the main plane of the calibration plate 2 is relatively large. Therefore, in the initial pose information determined by, for example, a directed bounding box, the main direction (thickness direction) of the calibration plate 2 and the position in the main direction have high accuracy, while the orientation of the calibration plate 2 in the plane perpendicular to the main direction has relatively high accuracy. The calibration method described above can effectively make up for this deficiency.

[0091] In an exemplary embodiment, in order to improve the accuracy of image matching, the projection image 104 and at least two image parts with different grayscale values ​​in the template image 105 may be matched respectively to determine at least two corresponding sub-correlation coefficients between the projection image 104 and the at least two image parts. Based on the at least two sub-correlation coefficients, a correlation coefficient representing the degree of matching between the projection image 104 and the template image 105 may be determined. In this way, the accuracy of the correlation coefficient may be improved. For example, Figure 8 As shown, the template image 105 may include a first image portion 107 and a second image portion 108 having different grayscale values. In this embodiment, the first image portion 107 is a black portion of the template image 105, and the second image portion 108 is a white portion of the template image 105. Figure 8In the figure, the blank parts in the first image part 107 and the second image part 108 are represented by light gray. Through an image matching algorithm, such as an NCC algorithm, it can be determined that the sub-correlation coefficient between the projection image 104 and the first image part 107 is S_1, and the sub-correlation coefficient between the projection image 104 and the second image part 108 is S_2. Based on the sub-correlation coefficients S_1 and S_2, a correlation coefficient representing the degree of matching between the projection image 104 and the template image 105 can be determined. For example, the correlation coefficient representing the degree of matching between the projection image 104 and the template image 105 can be determined based on the sub-correlation coefficients by averaging, summing, or weighted summing.

[0092] Fig. 9 A scenario of performing calibration using a calibration method according to an exemplary embodiment of the present invention is schematically shown. Fig.10 A flow chart of a calibration method for a laser radar 1 according to an exemplary embodiment of the present invention is schematically shown.

[0093] like Fig. 9 As shown, the laser radar 1 is installed on a vehicle, for example. The vehicle is also equipped with another sensor 4. The other sensor 4 may include another laser radar 1, a radar or a camera, etc. A calibration method according to an exemplary embodiment of the present invention can be used to implement joint calibration between the laser radar 1 and the other sensor 4. To this end, the calibration plate 2 can be placed in the overlapping area of ​​the field of view of the laser radar 1 and the field of view of the other sensor 4.

[0094] like Fig.10 As shown, in addition to steps S11, S12 and S13, the calibration method may further include steps S21 and S22.

[0095] In step S21, the detection data detected by the other sensor 4 for the calibration plate 2 can be obtained. Fig. 9 As shown, the additional sensor 4 and the laser radar 1 can be installed on the same vehicle, for example. The detection data and point cloud data measured by the additional sensor 4 and the laser radar 1 can be transmitted to the calibration device 3. The calibration device 3 is configured to be able to communicate with the additional sensor 4 and the laser radar 1 to achieve the transmission of data and / or control instructions. The calibration device 3 can be configured as a separate device relative to the vehicle. The detection data and point cloud data measured by the additional sensor 4 and the laser radar 1 can also be transmitted to the calibration device 3 by means of a data carrier. Optionally, the calibration device 3 can also be installed on the vehicle.

[0096] The additional sensor 4 can be a camera in particular. The camera can take a picture of the object to be measured and provide image data of the object to be measured, and the image data can include static pictures and / or videos. The calibration plate 2 is detected by the camera to obtain image data of the calibration plate 2.

[0097] In step S22, a second transformation matrix between the sensor coordinate system of the further sensor 4 and the calibration plate coordinate system may be determined based on the detection data. The second transformation matrix may include information about a second rotation matrix and a second translation vector.

[0098] It should be understood that step S21 and step S22 may be performed before, after or simultaneously with step S11, step S12 and / or step S13.

[0099] In an exemplary embodiment, in particular when the further sensor 4 is a camera, a second transformation matrix between a sensor coordinate system of the further sensor 4 and a calibration plate coordinate system may be determined based on a perspective n point (PnP) method.

[0100] The two-dimensional image coordinates of the calibration points on the calibration plate 2 in the image coordinate system can be determined by the image data of the calibration plate 2 captured by the camera. For example, the two-dimensional image coordinates of the calibration points in the image coordinate system can be expressed as: c =[u,v,1] T As an example, the corner points in the checkerboard pattern on the calibration plate 2 may be selected as calibration points. In particular, more than 6 calibration points may be selected.

[0101] Since the size of the calibration plate 2 and the calibration pattern 21 are known, the three-dimensional coordinates of the calibration point in the calibration plate coordinate system can be determined. The three-dimensional coordinates of the calibration point in the calibration plate coordinate system can be expressed as: b =[x,y,z,1] T .

[0102] The 2D image coordinates of the calibration points and the 3D positions of the calibration points in the calibration plate coordinate system follow the perspective projection model. c and p b Satisfy the following formula:

[0103] Z c =K[R2,t2]p b

[0104] Among them, K is the intrinsic parameter matrix of the camera, Z represents the scaling factor, R2 and t2 represent the rotation matrix and translation vector of the camera coordinate system relative to the calibration plate coordinate system, that is, the second rotation matrix and the second translation vector. Thus, the second transformation matrix can be determined. The second transformation matrix can be expressed as: T PnP =[R2,t2].

[0105] Then, in step S13, a third conversion matrix between the laser radar 1 and the other sensor 4 may be determined as a calibration parameter based on the first conversion matrix and the second conversion matrix. C T can be represented by the following formula, for example:

[0106]

[0107] By using the third transformation matrix, joint calibration between the laser radar 1 and the other sensor 4 can be achieved.

[0108] It should be understood that when describing exemplary embodiments, the specification may describe the calibration method or calibration process in a specific order of steps, or the flowchart may present the calibration method or calibration process in a specific order of steps. However, in the case where the calibration method or calibration process does not rely on the specific order of steps described herein, the calibration method or calibration process should not be limited to the steps in the specific order described. As will be understood by those of ordinary skill in the art, other orders of steps are also possible. Therefore, the specific order of the steps set forth in the specification should not be interpreted as a limitation on the claims. In addition, the claims to the calibration method or calibration process should not be limited to performing their steps in the order in which they are written, and those skilled in the art can easily understand that these orders can be changed and still fall within the scope of the present application.

[0109] The calibration method according to the present invention can be implemented by hardware, or by hardware executing corresponding software. The present invention also relates to a computer program product, which includes computer program instructions. When the computer program instructions are executed by one or more processors, the processors can execute the calibration method according to the present invention. The computer program instructions can be stored in a computer-readable storage medium. The computer-readable storage medium may include, for example: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above. The processor may be, for example, a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc.

[0110] Although specific embodiments of the present invention are described in detail herein, they are provided only for the purpose of explanation and should not be considered to limit the scope of the present invention. Various substitutions, changes and combinations may be conceived without departing from the spirit and scope of the present invention. The features of each embodiment may be separated or combined to form other implementations not explicitly described or illustrated herein, and still fall within the scope of protection of the present invention.

Claims

1. A calibration method for a laser radar, wherein: The calibration method comprises the following steps: S11, acquiring point cloud data about a point cloud detected by a laser radar (1) with respect to a calibration plate (2), wherein a main plane of the calibration plate (2) has a calibration pattern (21) and faces the laser radar (1); S12, determining the initial position information of the calibration plate (2) in the laser radar coordinate system based on the point cloud data; S13, using the calibration pattern (21) and the intensity information of the point cloud data to correct the initial posture information, so as to determine a first transformation matrix between the laser radar coordinate system and the calibration plate coordinate system, and determine the calibration parameters for the laser radar (1) based on the first transformation matrix.

2. The laser radar calibration method according to claim 1, wherein: Step S13 includes: S131, using the intensity information of the point cloud data, projecting the point cloud representing the calibration plate (2) along a main direction perpendicular to the main plane of the calibration plate (2) determined by the initial posture information, so as to generate a projection image (104); and S132, performing image matching on the projection image (104) and the template image (105) of the calibration pattern (21) and correcting the initial posture information according to the matching result of the image matching, so as to determine a first transformation matrix between the laser radar coordinate system and the calibration plate coordinate system.

3. The laser radar calibration method according to claim 2, wherein: In step S131, points with lower intensity in the point cloud representing the calibration plate (2) correspond to image units with lower brightness or higher grayscale values ​​in the projected image (104), and points with higher intensity in the point cloud representing the calibration plate (2) correspond to image units with higher brightness or lower grayscale values ​​in the projected image (104).

4. The laser radar calibration method according to claim 2 or 3, wherein: In step S132, the relative posture between the projection image (104) and the template image (105) of the calibration pattern (21) is determined by performing image matching between the projection image (104) and the template image (105), and the initial posture information is corrected using the relative posture between the projection image (104) and the template image (105).

5. The laser radar calibration method according to any one of claims 2 to 4, wherein: In step S132, image matching is performed on the projection image (104) and the template image (105) of the calibration pattern (21) in at least one of the following ways: Performing image matching between the projection image (104) and the template image (105) of the calibration pattern (21) using a normalized cross-correlation matching method; and / or Matching the projection image (104) and at least two image parts with different grayscale values ​​in the template image (105) respectively to determine at least two corresponding sub-correlation coefficients between the projection image (104) and the at least two image parts, and determining a correlation coefficient representing the degree of matching between the projection image (104) and the template image (105) based on the at least two sub-correlation coefficients; and / or The template image (105) is rotated within its image plane, and image matching is performed between the projection image (104) and the rotated template image (106) to determine the rotation angle of the template image (105) with the highest matching degree with the projection image (104), and the relative position between the projection image (104) and the template image (105) is determined based on the rotation angle.

6. The laser radar calibration method according to any one of claims 1 to 5, wherein: In step S11, the initial point cloud detected by the laser radar (1) is segmented to obtain at least one candidate point cloud cluster (101), and a target point cloud cluster (102) is selected from the at least one candidate point cloud cluster (101) as a point cloud cluster corresponding to the calibration plate (2), wherein: The point cloud points in each candidate point cloud cluster (101) are divided into at least two groups according to the intensity information, and the target point cloud cluster (102) is selected according to the ratio of the number of point cloud points in the at least two groups; and / or A target point cloud cluster (102) is selected according to the ratio of the sizes of the bounding boxes of each candidate point cloud cluster (101) along at least two different directions.

7. The laser radar calibration method according to claim 6, wherein: When the calibration pattern (21) is a checkerboard pattern, the selected target point cloud cluster (102) at least satisfies the following conditions: The point cloud points in the point cloud clustering are divided into a high intensity group and a low intensity group according to the intensity information, and the ratio of the number of point cloud points in the high intensity group to that in the low intensity group is between 0.5 and 2.

8. The laser radar calibration method according to claim 6 or 7, wherein: In step S11, a directed bounding box of each candidate point cloud cluster (101) is obtained, wherein the selected target point cloud cluster (102) satisfies at least one of the following conditions: The ratio of the minimum side length to the maximum side length of the directed bounding box of the target point cloud cluster (102) is less than 0.1; One of the three side lengths of the directed bounding box of the target point cloud cluster (102) is smaller than one tenth of the other two side lengths.

9. The laser radar calibration method according to any one of claims 1 to 8, wherein: In step S12, the initial position information of the calibration plate (2) in the laser radar coordinate system is determined based on the directed bounding box representing the point cloud of the calibration plate (2).

10. The laser radar calibration method according to any one of claims 1 to 9, wherein: The laser radar (1) is a non-repeating scanning laser radar; and / or The calibration pattern (21) is a checkerboard pattern.

11. The laser radar calibration method according to any one of claims 1 to 10, wherein: The calibration method further comprises the following steps: S21, acquiring detection data detected by another sensor (4) with respect to the calibration plate (2); S22, determining a second conversion matrix between a sensor coordinate system of the other sensor (4) and a calibration plate coordinate system based on the detection data; In step S13, based on the first conversion matrix and the second conversion matrix, a third conversion matrix between the laser radar (1) and the other sensor (4) is determined as a calibration parameter.

12. The laser radar calibration method according to claim 11, wherein: The further sensor (4) is a camera; and / or In step S22, a second transformation matrix between the sensor coordinate system of the further sensor (4) and the calibration plate coordinate system is determined based on the perspective n-point method.

13. A calibration device for a laser radar, wherein: The calibration device (3) comprises: one or more processors (31); and A memory (32) storing computer program instructions which, when executed by the one or more processors (31), enable the one or more processors (31) to perform the calibration method according to any one of claims 1 to 12.

14. A computer-readable storage medium storing computer program instructions, wherein: The computer program instructions, when executed by one or more processors, enable the one or more processors to perform the calibration method according to any one of claims 1-12.

Images