External parameter calibration method of laser radar, electronic equipment and calibration system

Abstract

The embodiment of the invention discloses an external parameter calibration method of a laser radar, electronic equipment and a calibration system.The method comprises the steps that a first image is obtained, the first image is obtained by a camera shooting unit of the laser radar on the basis of a calibration target plate, the calibration target plate comprises at least one color block, and pixel coordinates of feature points of the color blocks are obtained on the basis of the first image; the method comprises the steps of obtaining a direction vector under a radar coordinate system according to a preset angle mapping relation and pixel coordinates, and finally obtaining a first external parameter calibration conversion relation according to the direction vector and first measurement equipment coordinates, the first measurement equipment coordinates being coordinates of feature points of color blocks under a measurement equipment coordinate system, and the second measurement equipment coordinates being coordinates of feature points of color blocks under a measurement equipment coordinate system. The first external parameter calibration conversion relation is a conversion relation from a radar coordinate system to a measuring equipment coordinate system. According to the method, the camera unit of the laser radar is adopted for calibration based on the first image output by the calibration target, the corresponding external parameter calibration conversion relation is obtained, the calibration result is more reliable, and the external parameter calibration precision is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of laser radar technology, and in particular to an external parameter calibration method, electronic equipment and calibration system of a laser radar. Background Art

[0002] At present, in the field of autonomous driving, LiDAR is one of the most widely used environmental perception sensors. Most of the current mainstream sensor solutions for autonomous driving use a multi-LiDAR fusion method, which includes a main LiDAR with a longer range and better point cloud quality, and a blind LiDAR that can cover the blind spots of the long-range main LiDAR, thereby achieving lane-level positioning. However, due to the sparse point clouds of some LiDARs, external parameter calibration is difficult and the accuracy of external parameter calibration is low. Summary of the invention

[0003] In view of this, some embodiments of the present invention provide an external parameter calibration method, an electronic device and a calibration system for a laser radar, which can obtain a first external parameter calibration conversion relationship according to a first image output by a camera unit of the laser radar based on a calibration target plate, thereby improving the accuracy of the external parameter calibration.

[0004] In a first aspect, an embodiment of the present invention provides an external parameter calibration method for a laser radar, the method comprising:

[0005] Acquire a first image, wherein the first image is obtained by a camera unit of the laser radar based on a calibration target plate, and the calibration target plate includes at least one color block;

[0006] Based on the first image, obtaining pixel coordinates of feature points of the color block;

[0007] Obtaining a direction vector in a radar coordinate system according to a preset angle mapping relationship and the pixel coordinates;

[0008] A first external parameter calibration conversion relationship is obtained according to the direction vector and the first measuring device coordinates, wherein the first measuring device coordinates are the coordinates of the feature points of the color block in the measuring device coordinate system, and the first external parameter calibration conversion relationship is the conversion relationship from the radar coordinate system to the measuring device coordinate system.

[0009] In a second aspect, an embodiment of the present invention provides an electronic device, including:

[0010] at least one processor;

[0011] A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the external parameter calibration method of the laser radar as described above.

[0012] In a third aspect, an embodiment of the present invention provides an external parameter calibration system for a laser radar, including:

[0013] A laser radar installed on a carrier for carrying the laser radar; and the electronic device as described above.

[0014] In a fourth aspect, a computer-readable storage medium is provided in an embodiment of the present invention, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer device to execute the external parameter calibration method of the laser radar as described above.

[0015] Beneficial effects of the embodiments of the present invention: Different from the prior art, the external parameter calibration method of the laser radar provided by the embodiments of the present invention first obtains a first image, wherein the first image is obtained by the camera unit of the laser radar based on the calibration target plate, and the calibration target plate includes at least one color block, then based on the first image, the pixel coordinates of the feature points of the color block are obtained, and the direction vector in the radar coordinate system is obtained according to the preset angle mapping relationship and the pixel coordinates, and finally the first external parameter calibration conversion relationship is obtained according to the direction vector and the first measuring device coordinates, wherein the first measuring device coordinates are the coordinates of the feature points of the color block in the measuring device coordinate system, and the first external parameter calibration conversion relationship is the conversion relationship from the radar coordinate system to the measuring device coordinate system. The external parameter calibration method can calibrate the conversion relationship between the radar coordinate system and the measuring device coordinate system, and obtain the corresponding external parameter calibration conversion relationship. At the same time, compared with the method of performing external parameter calibration through sparse point cloud data, the external parameter calibration method uses the camera unit of the laser radar to calibrate the first image output by the calibration target, so that the calibration result is more reliable and the accuracy of the calibration result is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0017] Figure 1 Schematic diagram of an application environment of an external parameter calibration method for a laser radar provided by an embodiment of the present invention;

[0018] Figure 2 It is a flow chart of an external parameter calibration method of a laser radar provided by an embodiment of the present invention;

[0019] Figure 3 It is a structural schematic diagram of a calibration target plate provided by an embodiment of the present invention;

[0020] Figure 4is a schematic diagram of a first image provided by an embodiment of the present invention;

[0021] Figure 5 yes Figure 2 Schematic diagram of the process of step S12;

[0022] Figure 6 is a schematic diagram of a second image provided by an embodiment of the present invention;

[0023] Figure 7 is a schematic diagram of a rotation angle α and a vertical angle ω in a radar coordinate system provided by an embodiment of the present invention;

[0024] Figure 8 is a schematic diagram of a sorted calibration target plate provided by an embodiment of the present invention;

[0025] Fig. 9 It is a flowchart of another external parameter calibration method of a laser radar provided in an embodiment of the present invention;

[0026] Fig.10 is a schematic diagram of an external parameter calibration device for a laser radar provided in an embodiment of the present invention;

[0027] Fig.11 It is a structural schematic diagram of a laser radar provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0029] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0030] It should be noted that, if there is no conflict, the various features in the embodiments of the present invention can be combined with each other, all within the scope of protection of the present invention. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order from the module division in the device or the flow chart. In addition, the words "first", "second", "third", etc. used herein do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0031] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification and in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0032] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0033] See also Figure 1 , Figure 1 Schematic diagram of the application environment of an external parameter calibration method for a laser radar provided by an embodiment of the present invention. Figure 1 As shown, the application scenario includes: a laser radar 101 and a vehicle 102. The laser radar 101 is installed on various vehicles 102 that need to detect the surrounding environment. The vehicle 102 can be, for example, a vehicle, a ship, an aircraft, etc.

[0034] Optionally, when the vehicle 102 is a vehicle, the laser radar 101 can be installed on the front, rear, roof, side of the vehicle or any other body position where the laser radar 101 can be fixed. It is understood that the laser radar 101 can also be fixed on other device structure platforms that are externally connected to the vehicle. It is understood that the external device structure platform can keep running synchronously with the vehicle when the vehicle is running.

[0035] The laser radar 101 may include radars with different functions, and the radars with different functions are installed at different locations on the vehicle. For example, the main laser radar is generally installed on the front or roof of the vehicle, etc., with a longer distance measurement and better point cloud quality, while the blind spot radar is installed on the side of the vehicle to cover the long-range blind spot of the main laser radar, but the point cloud is sparse.

[0036] Laser radar can establish a radar coordinate system. Different laser radars can establish different radar coordinate systems. The main purpose of the external parameter calibration of the laser radar 101 is to determine the radar coordinate system T lidar The conversion relationship between the specific coordinate system and the sensor belongs to the external parameter calibration. The specific coordinate system refers to the coordinate system of the external device other than the radar to be calibrated, such as the measurement device coordinate system T station , or, vehicle coordinate system T vehicle However, the point clouds of some laser radars (such as blind spot radar) are sparse. If their point clouds are directly used for external parameter calibration, the calibration results will be unreliable and have poor accuracy.

[0037] An embodiment of the present invention provides an external parameter calibration method for a laser radar. The method first acquires a first image, wherein the first image is obtained by a camera unit of the laser radar based on a calibration target plate, and the calibration target plate includes at least one color block. Then, based on the first image, the pixel coordinates of the feature points of the color block are obtained, and the direction vector in the radar coordinate system is obtained according to a preset angle mapping relationship and the pixel coordinates. Finally, a first external parameter calibration conversion relationship is obtained according to the direction vector and the first measuring device coordinates, wherein the first measuring device coordinates are the coordinates of the feature points of the color block in the measuring device coordinate system, and the first external parameter calibration conversion relationship is the conversion relationship from the radar coordinate system to the measuring device coordinate system. The external parameter calibration method can calibrate the conversion relationship between the radar coordinate system and the measuring device coordinate system, and obtains the corresponding external parameter calibration conversion relationship. At the same time, compared with the method of performing external parameter calibration through sparse point cloud data, the external parameter calibration method uses the camera unit of the laser radar to calibrate the first image output by the calibration target, so that the calibration result is more reliable and the accuracy of the calibration result is improved.

[0038] The execution subject of the external parameter calibration method provided in the embodiment of the present invention (for example, Figure 1 The electronic device 103 in the vehicle 102 can be connected to the laser radar 101 and the measuring device 104 by wire or wireless connection to obtain the corresponding image or coordinate data corresponding to the calibration target plate 105, and perform relevant calculations and analysis to execute the external parameter calibration method to obtain the external parameter calibration conversion relationship. The electronic device 103 can be set in the vehicle 102, or the electronic device 103 is set outside the vehicle 102 and communicates with the vehicle 102 or the laser radar 101 over a long distance.

[0039] It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or developed in the future.

[0040] It should be noted that the above-mentioned electronic device 103 can be hardware or software. When the electronic device 103 is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the electronic device 103 is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here. It should be understood that the number of electronic devices 103 is only illustrative and can have any number of electronic devices according to implementation needs.

[0041] It should be noted that the vehicle 102 may also be any device for carrying the laser radar 101, such as a drone, a robot, or a sweeper. Figure 1 Only the vehicle 102 is shown as an example.

[0042] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of an external parameter calibration method for a laser radar provided in an embodiment of the present invention. Figure 2 As shown, the method S100 may specifically include the following steps:

[0043] S11: Acquire a first image, wherein the first image is obtained by a camera unit of the laser radar based on a calibration target plate, and the calibration target plate includes at least one color block;

[0044] In one embodiment, before executing the external parameter calibration method, preparations such as calibration target setting are required. First, a calibration target plate is customized. In one example, the schematic diagram of the calibration target plate is as follows: Figure 3 As shown, the laser radar built-in chip emits / receives light in the infrared spectrum band, which is invisible light. A 940nm backlight source needs to be customized, and the calibration target plate needs to be designed to be transmissive and attached to the backlight source. The size of the calibration target plate is not less than 60cmx60cm. The calibration target plate includes multiple color blocks. The color blocks need to be regular shapes (the embodiment of the present invention takes a circle as an example, and the radius of the circle is not less than 7cm). The interval between the color blocks is not less than 4cm. In addition, a smaller color block is required on the calibration target plate (the area is half of the circular color block, and the embodiment of the present invention is a square with a side length of 4cm).

[0045] It should be noted that the size of the calibration target plate, the shape, number and size of the color blocks can be set as needed. For example, the shape of the color block can be circular, square or rectangular, etc. In one embodiment of the present invention, each calibration target plate includes a square color block and six circular color blocks. It can be understood that six non-coplanar calibration points can be selected according to the calibration algorithm. As another optional embodiment, there can be at least four color blocks on a calibration target, which is not limited by the present application.

[0046] like Figure 1 As shown, the calibration target plate is set at a certain distance outside the laser radar, and there is no shielding between the calibration target plate and the laser radar. Preferably, the distance between the calibration target plate and the laser radar is between 1.5m and 2.0m, and several blocks ( Figure 1 (Take 3 pieces of 105 as an example) The calibration target plates are not in the same plane.

[0047] The built-in camera unit (such as a built-in camera) of the laser radar is used to shoot the calibration target plate to generate and output a first image, such as Figure 4As shown, the first image may be a grayscale image, which includes a plurality of color blocks on the calibration target plate. The laser radar transmits the first image to an electronic device, which processes and analyzes the first image.

[0048] S12: based on the first image, obtaining pixel coordinates of feature points of the color block;

[0049] After receiving the first image, the electronic device obtains the pixel coordinates of the feature point of the color block based on the first image. The feature point of the color block can be any point in the color block, for example, the center point of the color block or a corner point in the color block.

[0050] The first image is processed to obtain pixel coordinates of feature points of the color block. Specifically, Figure 5 As shown, step S12 includes:

[0051] S121, performing threshold processing on the first image to obtain a second image;

[0052] Thresholding processing is binarization processing. First, the current pixel value of each pixel in the first image is obtained. For the current pixel value of any pixel, if the current pixel value is greater than or equal to the preset pixel value, the first pixel value is used as the output pixel value of the pixel. If the current pixel value is less than the preset pixel value, the second pixel value is used as the output pixel value of the pixel. Finally, the second image is obtained according to the output pixel value of each pixel in the first image.

[0053] The preset pixel value, the first pixel value and the second pixel value are all set as needed. In the embodiment of the present invention, the first pixel value may be 255, the second pixel value may be 0, and the obtained second image is as follows: Figure 6 shown.

[0054] S122, performing contour processing on the second image to obtain a contour image;

[0055] The second image is processed by edge detection algorithm to find the edge in the second image. The continuous edges can form a contour to form a contour image. If the second image includes a square color block and six circular color blocks, a square contour and six circular contours can be obtained and they can be combined into the final contour image.

[0056] S123 . Obtain pixel coordinates of feature points of the color block according to the contour image.

[0057] In some embodiments, the minimum inscribed rectangle of the contour image is first obtained, the pixel coordinates of the center point of the contour image are determined according to the minimum inscribed rectangle, and finally the pixel coordinates of the center point are determined as the pixel coordinates of the feature points of the color block.

[0058] In some embodiments, the minimum circumscribed circle of the contour image may be obtained, and then the pixel coordinates of the center point of the contour image may be determined according to the minimum circumscribed circle, and finally the pixel coordinates of the center point may be determined as the pixel coordinates of the feature point of the color block.

[0059] In some embodiments, if the color block is a square color block and the contour image is a square, the corner points of the square can be taken as the feature points of the color block. For example, one of the four vertices of the square can be taken as the feature point of the color block. The pixel coordinates of each vertex can be determined according to the contour image, and then the pixel coordinates of the feature point of the color block can be selected therefrom.

[0060] In some embodiments, if the color block is an irregular circular color block, the contour image is an irregular circle. The two pixels with the longest distance can be obtained, and the distance is used as the diameter. The center of the circle is then obtained based on the diameter, and the center of the circle is used as the feature point of the color block, and its pixel coordinates are calculated.

[0061] S13: Obtaining a direction vector in a radar coordinate system according to a preset angle mapping relationship and the pixel coordinates;

[0062] Each laser radar shipped out of the factory has an angle calibration file for the laser radar, which includes an angle mapping relationship. The angle mapping relationship is the mapping of 2D pixel coordinates to the horizontal rotation angle α and vertical angle ω in the radar coordinate system, and then the corresponding 3D direction vector is obtained based on the horizontal rotation angle α and vertical angle ω.

[0063] Specifically, the radar coordinate system and angle relationship are as follows: Figure 7 As shown, if the pixel coordinates of the feature point of the color block are (u i ,v i ), then the horizontal rotation angle corresponding to the pixel coordinate in the radar coordinate system is α i , the vertical angle is ω i , if the feature point of the color block corresponds to point P in the radar coordinate system, its corresponding coordinates in the radar coordinate system are Then the coordinates of point P in the radar coordinate system are With angle α i ,ω i The relationship can be expressed by the following formula:

[0064]

[0065] in, is the coordinate of point P in the radar coordinate system, Indicates the distance from point P to the origin of the radar coordinate system.

[0066] In the embodiment of the present invention, since the point cloud output by the laser radar has an extremely low resolution and the point cloud is sparse, it is impossible to obtain Therefore, through the mapping relationship between pixel coordinates and angles, we can only obtain the point pointing from the origin of the radar coordinate system. Vector.

[0067] Therefore, according to the mapping relationship between 2D pixel coordinates and angles, a 3D direction vector in radar coordinates can be obtained, wherein the direction vector is a direction vector with the origin of the radar coordinate system and point P as the end point, pointing from the origin of the radar coordinate system to point P.

[0068] S14: Obtain a first external parameter calibration conversion relationship according to the direction vector and the first measuring device coordinates, wherein the first measuring device coordinates are coordinates of feature points of the color block in the measuring device coordinate system, and the first external parameter calibration conversion relationship is a conversion relationship from the radar coordinate system to the measuring device coordinate system.

[0069] The measuring equipment includes but is not limited to total stations, three-dimensional laser scanners, and high-resolution cameras, which can directly obtain high-precision points or point clouds or 3D coordinates. The 3D coordinates obtained are the coordinates of the measured object in the coordinate system of the measuring equipment.

[0070] The color blocks on the calibration target are measured by using a measuring device to obtain the first measuring device coordinates of the feature points of each color block in the measuring device coordinate system. In order to ensure that the 2D pixel coordinates of the feature points of the color blocks obtained above correspond to their 3D measuring device coordinates in the measuring device coordinate system, the feature points of the color blocks may be sorted according to the pixel coordinates of the feature points of the color blocks before the measuring device acquires the first measuring device coordinates, so that the measuring device sequentially acquires the first measuring device coordinates of the feature points of the sorted color blocks in order.

[0071] Alternatively, after obtaining the pixel coordinates, before converting the pixel coordinates into the direction vector in the radar coordinate system according to the angle mapping relationship, the feature points of the color block are sorted, and then the pixel coordinates of the feature points of the sorted color block are converted into the direction vector in the radar coordinate system according to the pixel coordinates of the feature points of the sorted color block and the angle mapping relationship.

[0072] When sorting the feature points of the color blocks, first determine the feature points of the color block corresponding to the contour image with the largest area as the feature points of the first color block, and then determine the feature points of the color block corresponding to the contour image with the smallest area as the feature points of the second color block. Then, based on the pixel coordinates, calculate the first distance from the feature points of each of the color blocks to the feature points of the first color block, and calculate the second distance from the feature points of each of the color blocks to the feature points of the second color block. Finally, based on the first distance and the second distance, sort the feature points of the color blocks.

[0073] For example, if the calibration target includes 1 square outline and 6 circular outlines, first determine the feature points of the first color block and the feature points of the second color block, then determine the largest first distance, and determine the feature point of the color block corresponding to the largest first distance as the feature point of the first color block, then calculate the second distance from all the feature points of the color blocks to the feature point of the second color block and the third distance from all the feature points of the color blocks to the feature point of the first color block according to the pixel coordinates, and determine the feature point of the color block with the smallest second distance and third distance as the feature point of the second color block, then calculate the distance from the feature points of the remaining color blocks except the feature points of the first color block and the feature points of the second color block to the feature points of the first color block according to the pixel coordinates. The fourth distance of the feature points of the color block is calculated, and the feature point of the color block with the smallest fourth distance is determined to be the feature point of the third color block. Then, the fifth distances from the feature points of the remaining color blocks except the feature points of the first color block, the feature points of the second color block and the feature points of the third color block are calculated according to the pixel coordinates to the feature points of the first color block, and the feature point of the color block with the smallest fifth distance is determined to be the feature point of the fourth color block. Finally, the sixth distances from the feature points of the remaining color blocks except the feature points of the first four color blocks are calculated according to the pixel coordinates to the feature points of the first color block, and the seventh distance to the feature points of the second color block are determined to be the feature point of the sixth color block with the smallest sixth distance, and the feature point of the fifth color block with the smallest seventh distance.

[0074] If the pixel coordinates of the feature point of the first color block with the largest contour area are defined as (u max ,v max ), the pixel coordinates of the feature point of the second color block with the smallest contour area are defined as (u min ,v min ), the pixel coordinates of the feature points of other color blocks are defined as ((u 1 ,v 1 ),(u 2 ,v 2 ),...(u 6 ,v 6)), where the distance between the feature points of two color blocks is calculated based on the pixel coordinates. The calculation formula is as follows:

[0075]

[0076] Among them, d i,j is the distance between the feature points of the two color blocks, and the pixel coordinates of the feature points of the two color blocks are (u i ,v i ),(u j ,v j ).

[0077] When sorting the feature points of the color blocks according to the pixel coordinates, first calculate ((u 1 ,v 1 ),(u 2 ,v 2 ),......(u 6 ,v 6 )) respectively to (u max ,v max ) pixel distance, where the pixel with the largest distance is the feature point of the first color block, and then calculate ((u 1 ,v 1 ),(u 2 ,v 2 ),......(u 6 ,v 6 )) satisfies (u min ,v min ) pixel distance and the first contour center pixel distance are the smallest as the feature point of the second color block, and then calculate the pixel distances from the remaining 4 contour centers to the first contour center. The closest one is the feature point of the third color block, and then calculate the distances from the remaining 3 contour centers to (u max , v max ) pixel distance, the closest one is the feature point of the fourth color block, and the remaining two contours are respectively (u max ,v max ), (u min ,v min ) can distinguish the 5th and 6th inner contours. Finally, the order of all inner contours is as follows Figure 8 As shown ( Figure 8 In the figure, “f” stands for “5”).

[0078] After sorting the feature points of the color blocks, the electronic device can transmit the pixel coordinates of the feature points of the sorted color blocks (or the size of the color blocks) and the corresponding serial numbers to the measuring device, so that the measuring device can collect the first measuring device coordinates of the feature points of the color blocks in the measuring device coordinate system according to the pixel coordinates of the feature points of the color blocks (or the size of the color blocks) in the corresponding serial number order.

[0079] In some embodiments, the electronic device can also make a one-to-one correspondence between the pixel coordinates of the feature points of the color block and the measuring device coordinates. Specifically, the electronic device obtains the pixel coordinates of the feature points of the color block, the size of the color block, and the sorting number, and then obtains the first measuring device coordinates and the size of the color block obtained by the measuring device from measuring the feature points of the color block. The electronic device then makes a one-to-one correspondence between the first measuring device coordinates and the pixel coordinates of the feature points of the color block according to the sorting number based on the size of the color block and the sorting number.

[0080] In some embodiments, other sorting methods may be used to sort the feature points of the color blocks, as long as the pixel coordinates of the feature points of the color blocks correspond one-to-one with the coordinates of the first measuring device, and no limitation is made here.

[0081] After obtaining the first measuring device coordinates and the corresponding direction vector, a multi-point perspective algorithm can be used to obtain the first external parameter calibration conversion relationship based on the direction vector and the first measuring device coordinates. For example, the PnP (Perspective-n-Point, referred to as PnP) algorithm can be used to determine the first external parameter calibration conversion relationship. The PnP algorithm is a method for solving the correspondence between 3D and 2D points. It is used to estimate the camera's posture through n 3D space points and their positions. It includes DLT (Direct Linear Transform, referred to as DLT), EPnP (Efficient Perspective-n-Point, referred to as EPnP) and other algorithms.

[0082] A series of 3D point coordinates are known and the pixel position of the corresponding point in the image coordinate system The solution formula of the PnP algorithm is:

[0083]

[0084] Among them, K is the camera intrinsic parameter matrix, and R,t is the extrinsic parameter matrix that needs to be solved.

[0085] In the embodiment of the present invention, the 3D direction vector output in step S13 and the PnP algorithm are used to calculate the first external parameter calibration conversion relationship from the radar coordinate system to the measurement device coordinate system. The specific formula is as follows:

[0086] K=I (4)

[0087] u i =cot(w i )cos(α i ) (5)

[0088] v i =-cot(wi )sin(α i ) (6)

[0089]

[0090]

[0091] in, and Corresponding to the feature points of the same color block on the calibration target plate, The angle α i ,ω i And the corresponding pixel coordinates, b refers to the feature point of the color block.

[0092] Therefore, the multi-point perspective algorithm can be used to cleverly obtain the first external parameter calibration conversion relationship from the radar coordinate system to the measurement equipment coordinate system.

[0093] In some embodiments, it is also necessary to calculate the conversion relationship from the radar coordinate system to other specific coordinate systems, such as the carrier coordinate system (such as the vehicle coordinate system) used to carry the radar. In this case, a measuring device can be used as an intermediate conversion device to obtain the corresponding conversion matrix through the conversion of radar coordinate system-measuring device coordinate system-carrier coordinate system.

[0094] Specifically, Fig. 9 As shown, the external parameter calibration method S200 also includes:

[0095] S21, determining a carrier feature point on a carrier for carrying the laser radar;

[0096] The carrier feature points are selected as needed, and points on the carrier that are easy to measure can be selected. If the carrier is a vehicle, the carrier feature points can be the four-wheel centers of the vehicle. If the carrier is a drone, the carrier feature points can be the gimbal of the drone, etc.

[0097] S22, obtaining a second measuring device coordinate of the carrier feature point output by the measuring device, wherein the second measuring device coordinate is in the measuring device coordinate system;

[0098] The carrier feature point is measured by a measuring device to obtain its second measuring device coordinate in the measuring device coordinate system. In the following embodiments, the carrier feature point is taken as a vehicle feature point and the carrier coordinate system is taken as a vehicle coordinate system to describe the calibration process of the second external parameter calibration conversion relationship.

[0099] Specifically, if there are k vehicle feature points, the coordinates of the second measuring device can be Among them, ve represents the vehicle feature point.

[0100] S23, obtaining the carrier coordinates of the carrier feature point in the carrier coordinate system;

[0101] The vehicle feature points are in the vehicle coordinate system T vehicle The 3D coordinates are

[0102] S24, determining a second external parameter calibration conversion relationship according to the carrier coordinates and the second measuring device coordinates, wherein the second external parameter calibration conversion relationship is a conversion relationship from the measuring device coordinate system to the carrier coordinate system;

[0103] Then calculate the second external parameter calibration conversion relationship from the measuring device coordinate system to the vehicle coordinate system It can be modeled by the following formula:

[0104]

[0105] This type of problem is a typical ICP (Iterative Closest Point, abbreviated as ICP) problem. In the solution process, the SVD (Singular Value Decomposition, abbreviated as SVD) algorithm can be used, that is, the singular value decomposition algorithm to solve the second external parameter calibration conversion relationship

[0106] S25. Determine a third external parameter calibration conversion relationship according to the first external parameter calibration conversion relationship and the second external parameter calibration conversion relationship, wherein the third external parameter calibration conversion relationship is a conversion relationship from the radar coordinate system to the carrier coordinate system.

[0107] Get the first external parameter calibration conversion relationship Conversion relationship with the second external parameter calibration Afterwards, the third external parameter calibration conversion relationship is calculated by the following formula:

[0108]

[0109]

[0110] Therefore, by using the measuring device as an intermediate device, the conversion relationship from the radar coordinate system to the measuring device coordinate system is first obtained, then the conversion relationship from the measuring device coordinate system to the carrier coordinate system is obtained, and finally the conversion relationship from the radar coordinate system to the carrier coordinate system is obtained.

[0111] To sum up, the external parameter calibration method can calibrate the conversion relationship between the radar coordinate system and the measuring device coordinate system to obtain the corresponding external parameter calibration conversion relationship. At the same time, compared with the method of extrinsic parameter calibration through sparse point cloud data, the external parameter calibration method uses the camera unit of the laser radar to calibrate based on the first image output by the calibration target, which makes the calibration result more reliable and improves the accuracy of the calibration result.

[0112] See also Fig.10 , Fig.10 FIG. 1 is a schematic diagram of the structure of an external parameter calibration device for a laser radar provided by an embodiment of the present invention. Fig.10 As shown, the external parameter calibration device 200 of the laser radar includes: a first acquisition module 201, a first acquisition module 202, a second acquisition module 203 and a third acquisition module 204.

[0113] Among them, the first acquisition module 201 is used to acquire a first image, wherein the first image is obtained by the camera unit of the laser radar based on a calibration target plate, and the calibration target plate includes at least one color block. The first acquisition module 202 is used to obtain the pixel coordinates of the feature points of the color block based on the first image. The second acquisition module 203 is used to obtain the direction vector in the radar coordinate system according to a preset angle mapping relationship and the pixel coordinates. The third acquisition module 204 is used to obtain a first external parameter calibration conversion relationship according to the direction vector and the first measuring device coordinates, wherein the first measuring device coordinates are the coordinates of the feature points of the color block in the measuring device coordinate system, and the first external parameter calibration conversion relationship is the conversion relationship from the radar coordinate system to the measuring device coordinate system.

[0114] Optionally, the first obtaining module 202 is specifically used for:

[0115] Performing threshold processing on the first image to obtain a second image;

[0116] Performing contour detection processing on the second image to obtain a contour image;

[0117] The pixel coordinates of the feature points of the color block are obtained according to the contour image.

[0118] Optionally, performing thresholding processing on the first image to obtain the second image includes:

[0119] Obtaining a current pixel value of each pixel in the first image;

[0120] For a current pixel value of any pixel, if the current pixel value is greater than or equal to a preset pixel value, the first pixel value is used as the output pixel value of the pixel.

[0121] If the current pixel value is less than the preset pixel value, taking the second pixel value as the output pixel value of the pixel;

[0122] The second image is obtained according to the output pixel value of each pixel in the first image.

[0123] Optionally, obtaining pixel coordinates of feature points of the color block according to the contour image includes:

[0124] Obtaining the minimum inscribed rectangle of the contour image;

[0125] Obtaining pixel coordinates of a center point of the contour image according to the minimum inscribed rectangle;

[0126] The pixel coordinates of the center point are determined as the pixel coordinates of the feature point of the color block.

[0127] Optionally, the external parameter calibration device 200 further includes a sorting module 205 for sorting the feature points of the color block according to the pixel coordinates of the feature points of the color block, so that the measuring device sequentially collects the first measuring device coordinates of the sorted feature points of the color block in sequence.

[0128] Optionally, the sorting module 205 is specifically used for:

[0129] Determine the feature point of the color block corresponding to the contour image with the largest area as the feature point of the first color block;

[0130] Determine the feature points of the color block corresponding to the contour image with the smallest area as the feature points of the second color block;

[0131] Calculating, according to the pixel coordinates, a first distance between a feature point of each of the color blocks and a feature point of the first color block;

[0132] Calculating, according to the pixel coordinates, a second distance from a feature point of each of the color blocks to a feature point of the second color block;

[0133] The feature points of the color blocks are sorted based on the first distance and the second distance.

[0134] Optionally, the third obtaining module 204 is specifically used for:

[0135] A multi-point perspective algorithm is used to obtain the first external parameter calibration conversion relationship according to the direction vector and the first measuring device coordinates.

[0136] Optionally, the external parameter calibration device 200 further includes a first determination module 206 , a second acquisition module 207 , a third acquisition module 208 , a second determination module 209 and a third determination module 210 .

[0137] The first determination module 206 is used to determine the carrier feature points on the carrier used to carry the laser radar;

[0138] The second acquisition module 207 is used to acquire the second measurement device coordinates of the carrier feature point output by the measurement device, wherein the second measurement device coordinates are in the measurement device coordinate system;

[0139] The third acquisition module 208 is used to obtain the carrier coordinates of the carrier feature point in the carrier coordinate system;

[0140] The second determination module 209 is used to determine a second external parameter calibration conversion relationship according to the carrier coordinates and the second measuring device coordinates, wherein the second external parameter calibration conversion relationship is a conversion relationship from the measuring device coordinate system to the carrier coordinate system;

[0141] The third determination module 210 is used to determine a third external parameter calibration conversion relationship according to the first external parameter calibration conversion relationship and the second external parameter calibration conversion relationship, wherein the third external parameter calibration conversion relationship is a conversion relationship from the radar coordinate system to the carrier coordinate system.

[0142] In an embodiment of the present invention, the external parameter calibration device of the laser radar can also be constructed by hardware devices. For example, the external parameter calibration device of the laser radar can be constructed by one or more chips, and each chip can work in coordination with each other to complete the external parameter calibration method of the laser radar described in the above embodiments. For another example, the external parameter calibration device of the laser radar can also be constructed by various logic devices, such as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, or any combination of these components.

[0143] The external parameter calibration device of the laser radar provided in the embodiment of the present invention can implement each process that can be implemented by the external parameter calibration method of the above-mentioned laser radar. To avoid repetition, it will not be repeated here.

[0144] It should be noted that the above-mentioned external parameter calibration device of the laser radar can execute the external parameter calibration method of the laser radar provided in the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in the embodiment of the external parameter calibration device of the laser radar, please refer to the external parameter calibration method of the laser radar provided in the embodiment of the present invention.

[0145] To sum up, the external parameter calibration device of the laser radar can calibrate the conversion relationship between the radar coordinate system and the measuring equipment coordinate system, and obtain the corresponding external parameter calibration conversion relationship. At the same time, compared with the method of extrinsic parameter calibration through sparse point cloud data, the external parameter calibration device uses the built-in camera unit of the radar to calibrate based on the first image output by the calibration target, so that the calibration result is more reliable and the accuracy of the calibration result is improved.

[0146] The present invention also provides a laser radar. Fig.11 , Fig.11 It is a schematic diagram of the hardware structure of a laser radar provided in an embodiment of the present invention.

[0147] like Fig.11 As shown, the laser radar 300 includes at least one processor 301 and a memory 302 ( Fig.11 (a bus connection and a processor are used as an example).

[0148] Among them, the processor 301 is used to provide computing and control capabilities to control the laser radar 300 to perform corresponding tasks, for example, to control the laser radar 300 to execute the laser radar external parameter calibration method in any of the above-mentioned method embodiments, the method including acquiring a first image, wherein the first image is obtained by the camera unit of the laser radar based on a calibration target plate, and the calibration target plate includes at least one color block, and then based on the first image, the pixel coordinates of the feature points of the color block are obtained, and according to the preset angle mapping relationship and the pixel coordinates, the direction vector in the radar coordinate system is obtained, and finally, the first external parameter calibration conversion relationship is obtained according to the direction vector and the first measuring device coordinates, wherein the first measuring device coordinates are the coordinates of the feature points of the color block in the measuring device coordinate system, and the first external parameter calibration conversion relationship is the conversion relationship from the radar coordinate system to the measuring device coordinate system.

[0149] In this embodiment, the laser radar can calibrate the transformation relationship between the radar coordinate system and the measuring device coordinate system to obtain the corresponding external parameter calibration transformation relationship. At the same time, compared with the method of extrinsic parameter calibration through sparse point cloud data, the laser radar uses the radar's built-in camera unit to calibrate based on the first image output by the calibration target, making the calibration result more reliable and improving the accuracy of the calibration result.

[0150] The processor 301 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0151] The memory 302, as a non-transient computer-readable storage medium, can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the external parameter calibration method of the laser radar in the embodiment of the present invention. The processor 301 can implement the external parameter calibration method of the laser radar in any of the above-mentioned method embodiments by running the non-transient software programs, instructions and modules stored in the memory 302. To avoid repetition, it will not be repeated here.

[0152] Specifically, the memory 302 may include a volatile memory (VM), such as a random access memory (RAM); the memory 302 may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD) or other non-transitory solid-state storage device; the memory 302 may also include a combination of the above types of memory.

[0153] In an embodiment of the present invention, the memory 302 may also include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0154] In an embodiment of the present invention, the laser radar 300 may also include other components for realizing the functions of the device, which will not be described in detail here.

[0155] The embodiment of the present invention further provides a computer-readable storage medium, such as a memory including a program code, and the program code can be executed by a processor to complete the external parameter calibration method of the laser radar in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0156] The embodiment of the present invention further provides a computer program product, which includes one or more program codes, and the program codes are stored in a computer-readable storage medium. The processor of the electronic device reads the program code from the computer-readable storage medium, and the processor executes the program code to complete the method steps of the external parameter calibration method of the laser radar provided in the above embodiment.

[0157] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] Through the description of the above implementation methods, ordinary technicians in this field can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Ordinary technicians in this field can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A laser radar external parameter calibration method, characterized in that: The method comprises: Acquire a first image, wherein the first image is obtained by a camera unit of the laser radar based on a calibration target plate, and the calibration target plate includes at least one color block; Based on the first image, obtaining pixel coordinates of feature points of the color block; Obtaining a direction vector in a radar coordinate system according to a preset angle mapping relationship and the pixel coordinates; A first external parameter calibration conversion relationship is obtained according to the direction vector and the first measuring device coordinates, wherein the first measuring device coordinates are the coordinates of the feature points of the color block in the measuring device coordinate system, and the first external parameter calibration conversion relationship is the conversion relationship from the radar coordinate system to the measuring device coordinate system.

2. The method according to claim 1, characterized in that The obtaining pixel coordinates of the feature points of the color block based on the first image includes: Performing threshold processing on the first image to obtain a second image; Performing contour detection processing on the second image to obtain a contour image; The pixel coordinates of the feature points of the color block are obtained according to the contour image.

3. The method according to claim 2, characterized in that The step of performing thresholding processing on the first image to obtain a second image includes: Obtaining a current pixel value of each pixel in the first image; For a current pixel value of any pixel, if the current pixel value is greater than or equal to a preset pixel value, the first pixel value is used as the output pixel value of the pixel. If the current pixel value is less than the preset pixel value, taking the second pixel value as the output pixel value of the pixel; The second image is obtained according to the output pixel value of each pixel in the first image.

4. The method according to claim 2, characterized in that: The step of obtaining pixel coordinates of feature points of the color block according to the contour image includes: Obtaining the minimum inscribed rectangle of the contour image; Obtaining pixel coordinates of a center point of the contour image according to the minimum inscribed rectangle; The pixel coordinates of the center point are determined as the pixel coordinates of the feature point of the color block.

5. The method according to claim 1, characterized in that Before obtaining the direction vector in the radar coordinate system according to the preset angle mapping relationship and the pixel coordinates, the method further includes: The feature points of the color blocks are sorted according to the pixel coordinates of the feature points of the color blocks, so that the measuring device sequentially collects the first measuring device coordinates of the sorted feature points of the color blocks in order.

6. The method according to claim 5, characterized in that The step of sorting the feature points of the color blocks according to the pixel coordinates of the feature points of the color blocks comprises: Determine the feature point of the color block corresponding to the contour image with the largest area as the feature point of the first color block; Determine the feature points of the color block corresponding to the contour image with the smallest area as the feature points of the second color block; Calculating, according to the pixel coordinates, a first distance between a feature point of each of the color blocks and a feature point of the first color block; Calculating, according to the pixel coordinates, a second distance from a feature point of each of the color blocks to a feature point of the second color block; The feature points of the color blocks are sorted based on the first distance and the second distance.

7. The method according to claim 1, characterized in that The obtaining of a first external parameter calibration conversion relationship according to the direction vector and the first measuring device coordinates includes: A multi-point perspective algorithm is used to obtain the first external parameter calibration conversion relationship according to the direction vector and the first measuring device coordinates.

8. The method according to any one of claims 1 to 7, characterized in that: After obtaining a first external parameter calibration conversion relationship according to the direction vector and the first measuring device coordinates, the method further includes: Determining carrier feature points on a carrier for carrying the laser radar; Acquire a second measuring device coordinate of the carrier feature point output by the measuring device, wherein the second measuring device coordinate is in the measuring device coordinate system; Obtaining the carrier coordinates of the carrier feature point in the carrier coordinate system; Determining a second external parameter calibration conversion relationship according to the carrier coordinates and the second measuring device coordinates, wherein the second external parameter calibration conversion relationship is a conversion relationship from the measuring device coordinate system to the carrier coordinate system; A third external parameter calibration conversion relationship is determined according to the first external parameter calibration conversion relationship and the second external parameter calibration conversion relationship, wherein the third external parameter calibration conversion relationship is a conversion relationship from the radar coordinate system to the carrier coordinate system.

9. An electronic device, characterized in that: include: at least one processor; A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the external parameter calibration method of the laser radar as described in any one of claims 1-8.

10. A radar external parameter calibration system, characterized in that: include: A laser radar installed on a carrier for carrying the laser radar; and an electronic device as described in claim 9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer device to execute the external parameter calibration method of the laser radar as described in any one of claims 1-8.

Images