Radar and camera joint calibration method, system, computer device, storage medium and program product

Abstract

The present application discloses a joint calibration method, system, computer equipment, storage medium and program product for radar and camera, wherein the method includes: obtaining an image of a calibration plate taken by a camera; calculating the pixel coordinates of the center of the graphic mark according to the positioning mark and the graphic mark; obtaining the point cloud data of the calibration plate collected by the radar, and calculating the first graphic center three-dimensional coordinates of the center of the graphic mark in the radar coordinate system according to the data corresponding to the graphic mark in the point cloud data; calculating the initial external parameters between the radar and the camera according to the pixel coordinates of the graphic center, the first graphic center three-dimensional coordinates and the parameter information of the camera. In the joint calibration method of radar and camera of the present application, the calibration plate can take into account the calibration requirements of high precision and high robustness by combining the positioning mark and the graphic mark.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving and robotics, and in particular to a radar and camera joint calibration method, system, computer equipment, storage medium and program product. Background Art

[0002] In today's fields of autonomous driving, robotics, etc., more and more people are combining LiDAR with cameras for environmental perception. LiDAR has the advantages of high ranging accuracy, strong environmental adaptability, and rich three-dimensional information. Cameras have rich information such as color and texture. However, cameras are prone to failure in scenes such as bad weather, low light environment, and sudden changes in light intensity. Through the joint perception of LiDAR and cameras, combining the advantages of the two and making up for their respective shortcomings can greatly promote the improvement of autonomous driving performance and engineering implementation.

[0003] In downstream perception tasks, information from multiple different sensors is fused. Joint extrinsic calibration of LiDAR and camera is an important step in fusing the two sensors. In the joint extrinsic calibration of LiDAR and camera, the estimated result is a transformation matrix that can be decomposed into a translation component (3 degrees of freedom) and a rotation component (3 degrees of freedom). This matrix represents the relative spatial position between the LiDAR and camera.

[0004] However, the joint calibration scheme of lidar and camera in related technologies has problems such as poor accuracy and low robustness. For example, when using a hollow circle for external parameter calibration, there are often problems such as poor accuracy and flying points at the edge of the point cloud, resulting in poor calibration accuracy. When using multi-target joint calibration, interference from other targets in the venue and target movement may cause target extraction failure and calibration failure, and the robustness of the scheme is poor. When using a checkerboard scheme for external parameter calibration, there are high requirements for the integrity of the checkerboard, and the checkerboard detection at the edge is prone to failure, but the edge of the field of view is also an important part of the calibration, so the robustness of the scheme is poor. Summary of the invention

[0005] The embodiments of the present application provide a radar and camera joint calibration method, system, computer device, storage medium and program product, which are used to solve at least one of the above-mentioned technical problems.

[0006] In a first aspect, an embodiment of the present application provides a joint calibration method of a radar and a camera, the method comprising:

[0007] Acquire an image of a calibration plate taken by the camera, wherein the calibration plate is provided with a positioning mark and a graphic mark;

[0008] Calculate the pixel coordinates of the center of the graphic of the graphic identifier according to the positioning identifier and the graphic identifier;

[0009] Acquire point cloud data of the calibration plate collected by the radar, and calculate the three-dimensional coordinates of the first graphic center of the center of the graphic identifier in the radar coordinate system according to the data corresponding to the graphic identifier in the point cloud data;

[0010] Initial external parameters between the radar and the camera are calculated according to the pixel coordinates of the center of the graphic, the three-dimensional coordinates of the center of the first graphic, and the parameter information of the camera.

[0011] In some embodiments, calculating the pixel coordinates of the center of the graphic of the graphic identifier according to the positioning identifier and the graphic identifier includes:

[0012] Calculate the pixel coordinates of the corner points of the positioning mark according to the positioning mark;

[0013] Calculating a homography matrix between a world coordinate system and a camera coordinate system according to the pixel coordinates of the corner points of the positioning mark and the world coordinates of the corner points of the positioning mark;

[0014] The center pixel coordinates of the graphic of the graphic mark are calculated according to the center world coordinates of the graphic mark and the homography matrix.

[0015] In some embodiments, the joint calibration method of the radar and the camera further includes: calculating the world coordinates of the corner points of the positioning mark and the world coordinates of the center of the graphic mark according to the size of the calibration plate and the size of the positioning mark.

[0016] In some embodiments, the point cloud data of the calibration plate is collected by radar, and the three-dimensional coordinates of the first graphic center of the center of the graphic identifier in the radar coordinate system are calculated according to the data corresponding to the graphic identifier in the point cloud data, including:

[0017] The point cloud data in the calibration site is collected by using a radar, and the calibration plate is arranged in the calibration site;

[0018] Filtering the point cloud data to obtain target point cloud data corresponding to the calibration plate;

[0019] The first graphic center three-dimensional coordinates of the center of the graphic identifier in the radar coordinate system are calculated according to the point cloud data corresponding to the graphic identifier in the target point cloud data.

[0020] In some embodiments, calculating the first graphic center three-dimensional coordinates of the center of the graphic identifier in the radar coordinate system according to the point cloud data corresponding to the graphic identifier in the target point cloud data includes:

[0021] Converting the target point cloud data into a grayscale image;

[0022] The three-dimensional coordinates of the first graphic center of the center of the graphic identifier in the radar coordinate system are calculated according to the graphic features corresponding to the graphic identifier in the grayscale image.

[0023] In some embodiments, the point cloud data includes a plurality of point cloud planes, and filtering the point cloud data to obtain target point cloud data corresponding to the calibration plate includes:

[0024] Acquire plane features of the plurality of point cloud planes, the plane features comprising at least one of the following: a plane unit normal vector, a plane size, and a plane distance;

[0025] The point cloud data is filtered based on the plane features to obtain target point cloud data corresponding to the calibration plate.

[0026] In some embodiments, the radar and camera joint calibration method further includes:

[0027] Calculate the central translation vector and rotation matrix of the positioning mark according to the positioning mark;

[0028] According to the central translation vector and rotation matrix of the positioning mark, combined with the central world coordinates of the graphic mark, the three-dimensional coordinates of the second graphic center of the center of the graphic mark in the camera coordinate system are calculated;

[0029] According to the three-dimensional coordinates of the center of the first graphic and the three-dimensional coordinates of the center of the second graphic, the initial external parameters are optimized to obtain target external parameters.

[0030] In some embodiments, the graphic logo is a circular logo, and the grayscale difference between the inner and outer parts of the circular logo is greater than a preset value.

[0031] In some embodiments, there are multiple positioning marks, there are multiple circular marks, and the multiple positioning marks and the multiple circular marks are arranged at intervals.

[0032] In some embodiments, there are multiple calibration plates, and the multiple calibration plates are set in different positions in the calibration site. The setting positions include but are not limited to the height of the calibration plate and the ground, the angle of the calibration plate and the ground, the angle of the calibration plate relative to the camera or radar, and the distance of the calibration plate relative to the camera or radar.

[0033] In a second aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the joint calibration method of radar and camera described in any embodiment of the present application.

[0034] In a third aspect, an embodiment of the present application provides a joint calibration system of a radar and a camera, characterized in that it includes: a camera configured to capture an image of a calibration plate; a radar configured to collect point cloud data of the calibration plate; and a computer device as described in any embodiment of the present application.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program / instruction stored thereon, characterized in that when the computer program / instruction is executed by a processor, the steps of the joint calibration method of radar and camera described in any embodiment of the present application are implemented.

[0036] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the steps of the joint calibration method of radar and camera described in any embodiment of the present application are implemented.

[0037] In the joint calibration method of radar and camera of the present application, the calibration plate can take into account the calibration requirements of high precision and high robustness by combining positioning marks and graphic marks. Using the positioning marks, the algorithm can easily realize the recognition and positioning of the calibration plate, and efficiently obtain the position and posture of the calibration plate relative to the camera. Even when the calibration plate is partially missing, it can still be recognized normally and perform high-precision feature point extraction. The circular feature mark of the calibration plate is conducive to point cloud feature extraction. The combination of the two and their complementary advantages can improve the accuracy and robustness of external parameter calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 This is a schematic diagram of the arrangement of the calibration plate in one embodiment of the present application;

[0040] Figure 2 A schematic diagram of a calibration plate in one embodiment of the present application;

[0041] Figure 3 This is a flow chart of an embodiment of a joint calibration method of radar and camera of the present application;

[0042] Figure 4 This is a flow chart of another embodiment of the joint calibration method of radar and camera of the present application;

[0043] Figure 5 This is a flow chart of another embodiment of the joint calibration method of radar and camera of the present application;

[0044] Figure 6 This is a flow chart of another embodiment of the joint calibration method of radar and camera of the present application;

[0045] Figure 7 A schematic diagram of the structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0047] It should also be noted that, in this article, the terms "include" and "comprise" include not only those elements, but also other elements not explicitly listed, or elements inherent to such processes, methods, articles or equipment. In the absence of further restrictions, the elements defined by the sentence "include..." do not exclude the existence of other identical elements in the process, method, article or equipment that includes the elements.

[0048] The purpose of this application is to provide a laser radar and camera joint external parameter calibration scheme, which has strong robustness and high accuracy, and effectively solves the problem of poor robustness and accuracy in the current calibration scheme. The laser radar and camera modules calibrated by this scheme can be applied to products such as self-driving cars, drones, embodied intelligent robots, quadruped robots, wheeled robots and tracked robots, or other products that require laser radar and camera modules, and this application does not limit this.

[0049] In this application, before performing the laser radar and camera joint external parameter calibration, a calibration plate needs to be pre-arranged at the calibration site, and a positioning mark and a graphic mark are set on the calibration plate. The calibration plate includes at least one positioning mark and at least one graphic mark.

[0050] In some embodiments, the number of calibration plates may be one or more. Exemplarily, multiple calibration plates may be arranged within the field of view of the laser radar and the camera to improve the coverage of the calibration points. When there are multiple calibration plates, the multiple calibration plates are set in different positions in the calibration site, and the setting position includes but is not limited to the height of the calibration plate and the ground, the angle of the calibration plate and the ground, the angle of the calibration plate relative to the camera or radar, and the distance of the calibration plate relative to the camera or radar.

[0051] like Figure 1 As shown, it is a schematic diagram of the arrangement of the calibration plates in one embodiment of the present application. This embodiment includes three calibration plates: calibration plate 11, calibration plate 12 and calibration plate 13. The three calibration plates are arranged in front of the laser radar and camera module 20, wherein the calibration plate 12 is arranged directly in front of the laser radar and camera module, the calibration plate 11 is arranged in the left front of the laser radar and camera module, and the calibration plate 13 is arranged in the right front of the laser radar and camera module, which can improve the coverage of the calibration points. In the present application, multiple calibration plates arranged in space are staggered in the three dimensions of horizontal, distance and height. Furthermore, there is a certain angle between the calibration plate and the vertical direction (to avoid multiple calibration plates having the same angle relative to the ground), so that the collected calibration data (for example, point cloud data collected by the laser radar and image data taken by the camera) is richer, thereby helping to improve the accuracy and robustness of the calibration results.

[0052] In some embodiments, the graphic mark is a circular mark, and the grayscale difference between the inside and outside of the circular mark is greater than a preset value. The preset value can be determined according to different device requirements, such as 60, 70, or even 100 or 200. For example, the color of the circular mark should have a large grayscale difference with the area outside the circle, such as the circular mark is black and the area outside the circle is white, which is convenient for circular feature extraction. The number of positioning marks is multiple, the number of the circular marks is multiple, and the multiple positioning marks and the multiple circular marks are arranged at intervals.

[0053] In some embodiments, the positioning identifier is an identification code, such as a QR code, an Apriltag identifier, etc., which can be used to distinguish the current calibration plate from other calibration plates.

[0054] like Figure 2 The figure shows a schematic diagram of a calibration plate in an embodiment of the present application. It includes three positioning marks D1, D2 and D3, and three graphic marks T1, T2 and T3. The three positioning marks and the three graphic marks are arranged in a matrix, and the positioning marks and the graphic marks are arranged at intervals. The three positioning marks use Apriltag marks, and the three graphic marks use circular marks.

[0055] Exemplarily, the Apriltag is selected from the identification family (i.e., the family of Apriltag types). Once the identification family and ID number are known, the coding pattern of the positioning identification can be uniquely determined. The recognition algorithm can obtain the ID number by specifying the identification family and parsing the coding pattern. Different positioning identifications can be distinguished according to the ID number.

[0056] The circular signs T1, T2 and T3 are white, and the area outside the circle is black, ensuring that there is a large grayscale difference between the two colors, which is convenient for circular feature extraction. The Apriltag signs are arranged at intervals from the circular signs. The Apriltag signs in the venue belong to the same sign family and have a unique identification code structure.

[0057] It should be noted that the above Figure 2 The calibration plate in the figure is only an example and is not limited to this application.

[0058] In some embodiments, the positioning identifier can also use Aruco identifiers, etc., and the coordinates of the identification corner points and the identification posture can also be applied to this solution, and this application does not limit this. The function of the positioning identifier is to determine the world coordinates, pixel coordinates, and identification posture of the identification center. The positioning identifier can be uniquely specified by the identification family and ID number. For example, the "tag36h11" identification family is selected, and the three identifications with ID numbers 1, 2, and 3 are different. The encodings of these three identifications are different, and there is no corresponding structure in other identification families. Therefore, an identification can be uniquely determined by the identification family and ID number, and different identifications can be distinguished by different ID numbers.

[0059] In some embodiments, the circular mark can be replaced by other marks of shapes that can determine the center point (for example, a square, a regular polygon, etc.), but the circular mark has the highest accuracy. Because the applicant found that other shapes have corners, the laser is a light spot with a certain area, and the light spot has different echo intensity characteristics at the corner point of the mark and at the edge of the mark, which leads to certain errors when extracting polygon features based on reflectivity information. However, the circle can solve this problem, and the echo intensity characteristics are the same at all edges, which is conducive to high-precision extraction of the center.

[0060] like Figure 3 FIG. 1 is a flow chart of an embodiment of a joint calibration method of a radar and a camera of the present application, and the method comprises the following steps:

[0061] S10, acquiring an image of a calibration plate captured by the camera, wherein a positioning mark and a graphic mark are provided on the calibration plate.

[0062] Exemplarily, the image of the calibration plate is captured by a camera for subsequent calibration processing. The calibration plate and the positioning mark and graphic mark on the calibration plate can be referred to in the above embodiment, and will not be described in detail here.

[0063] S20: Calculate pixel coordinates of the center of the graphic of the graphic identifier according to the positioning identifier and the graphic identifier.

[0064] In some embodiments, step S20 may include: calculating the pixel coordinates of the corner points of the positioning marker based on the positioning marker; calculating the homography matrix between the world coordinate system and the camera coordinate system based on the pixel coordinates of the corner points of the positioning marker and the world coordinates of the corner points of the positioning marker; and calculating the pixel coordinates of the center of the graphic of the graphic marker based on the center world coordinates of the graphic marker and the homography matrix.

[0065] The world coordinates of the corner points of the positioning mark and the world coordinates of the center of the graphic mark can be predetermined or determined during the calibration process. An optional determination method is to calculate the world coordinates of the corner points of the positioning mark and the world coordinates of the center of the graphic mark (in the world coordinate system of the calibration plate) according to the size of the calibration plate and the size of the positioning mark. Figure 2 For example, Figure 2 The upper left corner of the calibration plate is the origin (0, 0) in the world coordinate system. The total width of the calibration plate is 1260mm and the total height is 900mm. The length of the white rectangle surrounding each positioning mark and circular mark in the figure is 300mm and the width is 60mm. The distance between the four edges of the calibration plate and the long sides of the adjacent white rectangles is 60mm (hereinafter referred to as "black border width"). The four corner points of the positioning mark in the upper left corner of the calibration plate are (21, 22, 23, 24). According to the above-set world coordinate system origin, calibration plate size, white rectangle size and black border width, the world coordinates of the four corner points 21, 22, 23, and 24 of the positioning mark in the upper left corner can be calculated to be (120, 420), (420, 420), (420, 120), and (120, 120), respectively, in mm. Similarly, it can be calculated that Figure 2 The world coordinates of the centers O1, O3, and O5 of the circular logos T1, T2, and T3 (ie, the world coordinates of the centers of the graphic logos) are (270, 530), (990, 530), and (630, 270), respectively.

[0066] Exemplarily, the world coordinates of the corner points of the positioning mark and the world coordinates of the center of the graphic mark can be predetermined, for example, pre-calculated and stored in the device. When needed, the world coordinates of the corner points of the positioning mark and the world coordinates of the center of the graphic mark on the corresponding calibration plate can be found by identifying the positioning mark (such as ID).

[0067] Exemplarily, a camera is used to take a picture of the site (including the calibration plate), and the positioning marks in the picture are identified using a positioning recognition algorithm to obtain the pixel coordinates of the corner points of each positioning mark. The homography matrix between the world coordinate system and the camera coordinate system is calculated from the pixel coordinates of the corner points of the positioning mark and the world coordinates of the corner points of the positioning mark. The pixel coordinates of the center of the circular mark of the calibration plate are calculated by combining the world coordinates of the center of the circular mark with the homography matrix (for example, the pixel coordinates of the center of the circular mark can be obtained by multiplying the homography matrix with the world coordinates of the center of the circular mark).

[0068] Optionally, after determining the pixel coordinates of the corner points of the positioning mark and the world coordinates of the corner points of the positioning mark, the findHomography function of the opencv library is used to calculate the homography matrix.

[0069] Optionally, an image recognition algorithm may be used to identify the graphic mark on the calibration plate, and the corresponding center pixel coordinates may be calculated based on the identified graphic mark.

[0070] S30, acquiring point cloud data of the calibration plate collected by the radar, and calculating the three-dimensional coordinates of the first graphic center of the center of the graphic identifier in the radar coordinate system according to the data corresponding to the graphic identifier in the point cloud data.

[0071] Exemplarily, a laser radar is used to record a point cloud of the site, and a RANSAC (Random Sample Consensus) algorithm is used to extract a planar point cloud within the site, and the data corresponding to the graphic logo in the point cloud data is determined, and based on this, the three-dimensional coordinates of the center of the graphic logo in the laser radar coordinate system are calculated.

[0072] S40. Calculate initial external parameters between the radar and the camera according to the center pixel coordinates of the graphic, the three-dimensional coordinates of the center of the first graphic, and the parameter information of the camera.

[0073] Exemplarily, the PnP (Perspective-n-Point, multi-point perspective imaging) algorithm is used to solve the initial external parameters (including rotation matrix and translation vector). For example, through the pixel coordinates of the above-mentioned multiple graphic centers (e.g., the center of the circle) and the three-dimensional coordinates of the multiple graphic centers (e.g., the center of the circle) in the laser radar coordinate system (i.e., the three-dimensional coordinates of the first graphic center), combined with the parameter information of the camera (e.g., the intrinsic parameter matrix, the camera distortion coefficient), the PnP algorithm can be used to iteratively obtain the relative posture (rotation matrix and translation vector) between the camera and the laser radar, which is used as the initial external parameter.

[0074] In some embodiments, in order to ensure that there is as much available plane point cloud data as possible in the point cloud data collected by the radar, multiple calibration plates can be set at the same time, and the multiple calibration plates are staggered in the three dimensions of horizontal, distance, and height. Furthermore, there is a certain angle between the calibration plate and the vertical direction (to avoid multiple calibration plates having the same angle relative to the ground). After the radar collects point cloud data from multiple calibration plates, the point cloud data of the calibration plates that meet the requirements are screened out through preset conditions for subsequent joint calibration of the radar and camera.

[0075] In the joint calibration method of radar and camera of the present application, the calibration plate can take into account the calibration requirements of high precision and high robustness by combining positioning marks and graphic marks. Using the positioning marks, the calibration plate can be easily identified and positioned through a preset recognition algorithm, and the position of the calibration plate relative to the camera can be efficiently obtained. Even when part of the calibration plate is missing, it can still be recognized normally and feature points can be extracted with high precision. The graphic mark of the calibration plate is a non-hollow mark, which is conducive to point cloud feature extraction. The combination of the two and their complementary advantages can improve the accuracy and robustness of external parameter calibration.

[0076] like Figure 4 FIG. 1 is a flowchart of another embodiment of the joint calibration method of radar and camera of the present application. Based on the above embodiment, in this embodiment, the point cloud data of the calibration plate is collected by radar, and the three-dimensional coordinates of the first graphic center of the center of the graphic mark in the radar coordinate system are calculated based on the data corresponding to the graphic mark in the point cloud data, including:

[0077] S31, using a radar to collect point cloud data in a calibration site, wherein the calibration plate is arranged in the calibration site. Exemplarily, a laser radar is used to record the site point cloud, and a RANSAC (Random Sample Consensus) algorithm is used to extract the plane point cloud in the site.

[0078] S32. Filter the point cloud data to obtain target point cloud data corresponding to the calibration plate.

[0079] Exemplarily, the point cloud data includes multiple point cloud planes. Exemplarily, the plane point cloud is screened according to the plane features, and the point cloud of the calibration plate that does not meet the plane features is eliminated. The plane features may include a plane unit normal vector, a plane size (length, height), a plane distance (the plane distance refers to the average distance from a point in the point cloud plane to the radar origin), etc.

[0080] In some embodiments, the point cloud data is filtered to obtain target point cloud data corresponding to the calibration plate, including: obtaining plane features of the multiple point cloud planes, the plane features including at least one of the following: plane unit normal vector, plane size and plane distance; filtering the point cloud data based on the plane features to obtain target point cloud data corresponding to the calibration plate.

[0081] For example, the plane is filtered according to the plane normal (unit direction vector perpendicular to the plane) returned by RANSAC. For example, the extracted plane normal is normalized to the plane unit normal vector (0, 0, 1). Since the calibration plate is facing the lidar camera module, the threshold requirement can be set as: the absolute value in the Z direction should be less than 0.5. However, the third dimension of the currently extracted plane is 1, so the currently extracted plane is deleted from the point cloud set.

[0082] Furthermore, the plane can be filtered according to the plane size. For example, the plane point cloud is filtered using a radius filter algorithm to calculate the overall size of the plane point cloud. Size calculation method:

[0083] Calculate the X+Z of each point in the plane point cloud. The point corresponding to the maximum value of X+Z is the upper right corner, and the point corresponding to the minimum value of X+Z is the lower left corner.

[0084] Calculate the ZX of each point in the plane point cloud. The point corresponding to the maximum ZX value is the upper left corner point, and the point corresponding to the minimum ZX value is the lower right corner point.

[0085] After calculating the coordinates of the four corner points of the calibration plate, the side length can be calculated. Assuming the allowed range of length is (0.8~1.2m) and the allowed range of height is (0.4~0.8m), if the calculated plane point cloud is 1m long and 0.5m high, then the plane meets the conditions; if the calculated plane point cloud is 2m long and 1m high, then the plane does not meet the conditions and should be filtered out.

[0086] In addition, the point cloud plane can be filtered according to the plane distance. For example, after the site is fixed, the distance from the point cloud plane to the radar is basically fixed. Therefore, the candidate planes can be filtered by the plane distance. For example, assuming that the plane distance allowable range is 8~10m, if the plane distance of the extracted point cloud plane is 7m, the plane can be filtered out.

[0087] S33, converting the target point cloud data into a grayscale image. Exemplarily, the calibration plate point cloud is converted into an image matrix, and the reflectivity information is converted into a grayscale value.

[0088] S34. Calculate the three-dimensional coordinates of the first graphic center of the center of the graphic identifier in the radar coordinate system according to the graphic features corresponding to the graphic identifier in the grayscale image.

[0089] Exemplarily, the graphic logo is a circular logo, and the image processing algorithm is used to extract the elliptical features in the point cloud, and the three-dimensional coordinates of the center of the elliptical feature (the center of the circular logo) in the laser radar coordinate system are calculated. It should be noted that in the process of implementing this application, the applicant found that the circle in the three-dimensional space is an ellipse after being projected onto the preset pixel matrix. Therefore, it is necessary to extract the elliptical features instead of the circle from the image converted from the point cloud into a pixel matrix, and the use of the elliptical model is compatible with the circular feature extraction, ensuring the reliability and accuracy of the feature extraction.

[0090] In this embodiment, the RANSAC algorithm is used to extract the plane point cloud in the venue, and the plane is screened according to the plane features to eliminate the point cloud of the non-target calibration plate. The plane features may include the plane unit normal vector, the plane size (length, height), the plane distance, etc. The point cloud of the target calibration plate is converted into an image matrix, the reflectivity information is converted into a grayscale value, and the image processing algorithm is used to extract the graphic features in the point cloud, and the three-dimensional coordinates of the center of the graphic logo in the laser radar coordinate system are calculated. The calibration plate in the venue is extracted using the RANSAC algorithm, and the plane point cloud is filtered based on the plane normal, calibration plate size and other features, so that the efficient extraction of the calibration plate point cloud can be achieved, so that the three-dimensional coordinates of the center of the graphic logo in the laser radar coordinate system can be efficiently obtained through the extracted point cloud.

[0091] like Figure 5 FIG. 2 is a flowchart of another embodiment of the joint calibration method of radar and camera of the present application. In this embodiment, the joint calibration method of radar and camera also includes:

[0092] S50, calculating the central translation vector and rotation matrix of the positioning mark according to the positioning mark. Exemplarily, a camera is used to take a picture of the site (including the calibration plate), and the positioning marks in the picture are identified using a positioning recognition algorithm to solve the translation vector (i.e., three-dimensional coordinates) and rotation matrix of the center of each positioning mark.

[0093] Taking the Apriltag positioning mark as an example, use the opencv library function to convert the image into a grayscale image, and use the Apriltag library to create an Apriltag detector object. Use the detect method of the Apriltag detector object to detect the Apriltag positioning mark in the image, and combine the size of the Apriltag positioning mark in the camera to obtain the translation vector and rotation matrix of each Apriltag positioning mark center. Among them, the detect method can refer to the relevant technology, and this application does not limit it.

[0094] S60, calculating the second graphic center three-dimensional coordinates of the center of the graphic marker in the camera coordinate system according to the central translation vector and rotation matrix (relative to the camera coordinate system) of the positioning marker and the central world coordinates of the graphic marker.

[0095] by Figure 2 Take the positioning mark D2 and the graphic mark T1 as an example, and set the upper left corner of the calibration plate as the origin Ob of the world coordinate system of the calibration plate, and the origin Oc of the world coordinate system of the camera. According to the central translation vector and rotation matrix of the center O4 of the positioning mark D2, combined with the three-dimensional coordinates of the center O4 in the world coordinate system of the calibration plate (which can be expressed as vector ObO4), the three-dimensional coordinates of the center O4 in the world coordinate system of the camera can be calculated (which can be expressed as vector OcO4). At this time, the vector between the origin Ob of the world coordinate system of the calibration plate and the origin Oc of the world coordinate system of the camera can be obtained based on vector operations: OcOb=OcO4-ObO4.

[0096] For the center O1 of the graphic mark T1, when the three-dimensional coordinates of the center O1 in the world coordinate system of the calibration plate are known (which can be expressed as vector ObO1), combined with the vector OcOb, the three-dimensional coordinates of the center O1 in the camera coordinate system can be calculated: OcO1=OcOb+ObO1, ObO1 represents the three-dimensional coordinates of O1 in the world coordinate system of the calibration plate, and it can be converted into the three-dimensional coordinates of O1 in the world coordinate system of the camera by adding OcOb. Similarly, it can be calculated Figure 2 The three-dimensional coordinates of the centers O3 and O5 in the camera's world coordinate system.

[0097] S70. Optimize the initial extrinsic parameters according to the three-dimensional coordinates of the center of the first graphic (in a radar coordinate system) and the three-dimensional coordinates of the center of the second graphic (in a camera coordinate system) to obtain target extrinsic parameters.

[0098] In this embodiment, after obtaining the initial extrinsic parameters, the initial extrinsic parameter matrix is ​​nonlinearly optimized by combining the three-dimensional coordinates of the center of the circle in the camera coordinate system and the three-dimensional coordinates of the center of the circle in the lidar coordinate system, and an exact solution of the extrinsic parameter matrix is ​​output, thereby improving the accuracy and robustness of the joint calibration of the radar and camera.

[0099] like Figure 6 As shown, it is a flow chart of another embodiment of the joint calibration method of radar and camera of the present application. In this embodiment, Apriltag positioning mark and circular mark are taken as examples, but the present application is not limited to this. This embodiment includes the following steps:

[0100] 1. Picture shooting and processing.

[0101] 1.1. Picture shooting: Use a camera to take pictures of the site, use the opencv library function to convert the pictures into grayscale images, use the Apriltag library to create an Apriltag detector object, and specify the Apriltag family to be used. Figure 2 The Apriltag positioning mark generated by the "tag36h11" family is shown. The "tag36h11" family is specified for identification during positioning mark identification.

[0102] Marker detection and corner pixel coordinate determination: Use the detect method of the Apriltag detector object to detect the Apriltag positioning markers in the grayscale image. Combined with the size of the Apriltag positioning markers in the camera, the corner pixel coordinates of each Apriltag positioning marker, the translation vector and rotation matrix of the Apriltag positioning marker center can be obtained.

[0103] like Figure 2 As shown, the pixel coordinates of the four corner points of each Apriltag positioning mark can be obtained through the Apriltag recognition algorithm (such as Figure 2 In the figure, the corner points 4, 5, 6, and 7 of the positioning mark D1 are marked, and the numbers are incremented counterclockwise. In addition, each corner point may be subjected to an anti-distortion process, and the anti-distortion process may adopt relevant existing technologies, which is not limited in the present application.

[0104] 1.2. Calculate the world coordinates of the Apriltag positioning mark on the calibration plate and the world coordinates of the center of the circular mark.

[0105] According to the size of the calibration plate / locating mark (for example, Apriltag positioning mark), calculate the world coordinates of the corner points and center of the Apriltag positioning mark with the upper left corner of the calibration plate as the origin. Figure 2 As shown, for the positioning mark D2, the coordinates of its corner points 21, 22, 23, and 24 are (120, 420), (420, 420), (420, 120), and (120, 120), respectively, and the unit is mm. The world coordinates of the corner points of other positioning marks can be calculated in the same way. The world coordinates of the center of the Apriltag positioning mark can also be calculated in the same way. The world coordinates of the centers O1, O3, and O5 of the three circular marks T1-T3 are (270, 530), (990, 530), and (630, 270), respectively. The obtained two-dimensional coordinates are converted into homogeneous form to obtain the corresponding three-dimensional coordinates.

[0106] 1.3. After obtaining the world coordinates of the corner points of the Apriltag positioning marker, combine them with the pixel coordinates of the corner points of the Apriltag positioning marker identified in step 1.1 and use the findHomography function of the opencv library to calculate the homography matrix.

[0107] 1.4. Calculation of pixel coordinates of the circle center: Multiply the homography matrix with the world coordinates of the centers of the three circular markers O1, O3, and O5 to obtain the pixel coordinates of the circle center.

[0108] 1.5. Calculation of the three-dimensional coordinates of the first center of the circle: Derive the three-dimensional coordinates of the center of the circular mark in the camera coordinate system (that is, the three-dimensional coordinates of the first center of the circle). The derivation method is as follows:

[0109] From step 1.1, we can get the translation vector and rotation matrix (relative to the camera coordinate system) of the Apriltag positioning mark center. From step 1.2, we can calculate the world coordinates of the circular mark center. Through vector operations, we can get the translation vector of the origin of the calibration plate (i.e., the upper left corner of the calibration plate). Let the origin of the camera world coordinate system be Oc, and the origin of the calibration plate world coordinate system be Ob, then:

[0110] As mentioned in the above embodiment, after calculating the two-dimensional world coordinates of the center of the circle, the two-dimensional coordinates are converted into homogeneous form for subsequent three-dimensional calculations. Taking the marker O4 as an example, OcO4 represents the three-dimensional coordinates of O4 in the camera world coordinate system, which can be obtained by the marker pose, and ObO4 represents the three-dimensional coordinates of O4 in the calibration plate world coordinate system, which can be obtained by the constructed calibration plate world coordinate system. The two are subtracted to obtain OcOb.

[0111] OcOb=OcO4-ObO4

[0112] When there are multiple Apriltag positioning markers, OcOb can take the average of multiple results to improve the accuracy of OcOb, thereby ultimately improving the accuracy of the calibration results. Then the coordinates of the center O1 in the camera coordinate system (that is, the vector from the origin Oc to the center O1) are:

[0113] OcO1=OcOb+ObO1, ObO1 represents the three-dimensional coordinate of O1 in the world coordinate system of the calibration plate. Adding OcOb can convert it into the three-dimensional coordinate of O1 in the world coordinate system of the camera.

[0114] The coordinates of other circle centers in the camera coordinate system can be obtained in the same way:

[0115] OcO3=OcOb+ObO3

[0116] OcO5=OcOb+ObO5

[0117] At this point, the three-dimensional coordinates of the center of the circular mark on the calibration plate in the camera coordinate system can be calculated.

[0118] 2. Point cloud recording and processing.

[0119] 2.1. Point cloud recording: Use LiDAR to record the point cloud of the site. The point cloud should at least include information such as XYZ, reflectivity (or intensity). Assuming the radar point cloud coordinate system is a right-handed coordinate system, X represents the horizontal direction, Y represents the distance direction, and Z represents the height direction. The following solutions will be introduced by default based on this coordinate system.

[0120] 2.2. Calibration plate point cloud extraction: In each frame of point cloud, the RANSAC algorithm is used to extract plane feature point cloud (plane features can include plane unit normal vector, plane size (length, height), plane distance, etc.).

[0121] First, filter the plane according to the plane normal returned by RANSAC. For example, the extracted plane normal is normalized to (0, 0, 1). Since the calibration plate is facing the lidar camera module, the threshold requirement can be set as: the absolute value in the Z direction should be less than 0.5. However, the third dimension of the currently extracted plane is 1, so the currently extracted plane is deleted from the point cloud set.

[0122] If the plane normal meets the above threshold requirements, the radius filter algorithm is used to filter the plane point cloud and calculate the overall size of the plane point cloud. Size calculation method:

[0123] Calculate the X+Z of each point in the plane point cloud. The point corresponding to the maximum value of X+Z is the upper right corner, and the point corresponding to the minimum value of X+Z is the lower left corner.

[0124] Calculate the ZX of each point in the plane point cloud. The point corresponding to the maximum ZX value is the upper left corner point, and the point corresponding to the minimum ZX value is the lower right corner point.

[0125] After calculating the coordinates of the four corner points of the calibration plate, the side length can be calculated. Assume that the allowed range of length is (0.8~1.2m) and the allowed range of height is (0.4~0.8m). If the calculated plane point cloud is 1m long and 0.5m high, then the plane is a qualified plane; if the calculated plane point cloud is 2m long and 1m high, then the plane is a non-qualified plane and should be filtered out. In addition, the point cloud plane can also be filtered according to the plane distance. For example, after the site is fixed, the distance from the point cloud plane to the radar is basically fixed. Therefore, the candidate planes can be screened by the plane distance. For example, assuming that the allowed range of the plane distance is 8~10m, if the plane distance of the extracted point cloud plane is 7m, the plane can be filtered out.

[0126] 2.3. For planes that meet the conditions, add the point cloud to the plane point set and delete it from the total point cloud set. For planes that do not meet the conditions, delete them directly from the total point cloud set. After the point cloud set is updated, continue to execute step 2.2 until the number of newly added planes that meet the conditions is 0, then stop searching.

[0127] 2.4. According to the coordinates of the point cloud, the plane point cloud that meets the conditions is extracted and inserted into the preset pixel matrix. For example, the horizontal and vertical pixel numbers of the laser radar used in this application are 800*96, so an 800*96 matrix can be preset, and the point cloud reflectivity (or intensity) can be filled into the matrix in sequence to form a point cloud image. Each frame of the point cloud can generate one point cloud image.

[0128] 2.5. Circular feature extraction and determination of the three-dimensional coordinates of the second center of the circle: Use the opencv library to perform image smoothing and binarization on the point cloud image generated in step 2.4, and extract the image contour. Fit the elliptical contour, and filter the elliptical contour according to the ratio of the major and minor axes and the size of the major and minor axes of the ellipse (a circle in three-dimensional space is an ellipse after being projected onto a preset pixel matrix, that is, there is a mapping relationship between the circle and the ellipse, and the corresponding ratio of the major and minor axes and the size of the major and minor axes of the ellipse can be determined based on the mapping relationship combined with the size of the circle. The ratio of the major and minor axes and the size of the major and minor axes can be obtained by measurement after the calibration plate is arranged in the venue, so as to filter the fitted elliptical contour), and obtain the coordinates of the center of the ellipse that meets the conditions. Through the pixel value corresponding to the coordinates of the center of the ellipse, the three-dimensional coordinates corresponding to the point can be found in the point cloud. At this point, the three-dimensional coordinates of the center of the circular mark of the calibration plate in the radar coordinate system (that is, the three-dimensional coordinates of the second center of the circle) can be obtained.

[0129] Reference Figure 2 In the figure, there are three circular marks T1, T2 and T3. When acquiring point cloud data, the point cloud data corresponding to different circular marks may be lost to varying degrees, that is, there may be unavailable circular mark point cloud data that affects the calibration accuracy. Therefore, in this application, multiple circular marks are set, and preset conditions (major-minor axis ratio and major-minor axis size) are set to select better related data for subsequent joint calibration of radar and camera, thereby improving the accuracy of the calibration results.

[0130] 3. PnP pose estimation.

[0131] After obtaining the pixel coordinates of the center of the circular mark on the calibration plate in the image coordinate system (step 1.4), the three-dimensional coordinates of the center in the radar coordinate system (step 2.5), and the camera intrinsic parameter results, the solvePnP function of the opencv library can be used to obtain the external parameter matrix between the lidar and the camera. This external parameter matrix can be used as the initial external parameter matrix.

[0132] 4. Nonlinear optimization.

[0133] After obtaining the initial extrinsic parameter matrix, the extrinsic parameter matrix can be nonlinearly optimized by combining the three-dimensional coordinates of the center of the circle in the camera coordinate system (step 1.5) and the three-dimensional coordinates of the center of the circle in the radar coordinate system (step 2.5) to obtain the optimized extrinsic parameter matrix (target extrinsic parameter). For example, Ceres Solver is used to optimize the initial extrinsic parameter matrix according to the three-dimensional coordinates of the center of the circle in the camera coordinate system and the three-dimensional coordinates of the center of the circle in the lidar coordinate system. The specific implementation method is not limited in this application.

[0134] This step can also be used to perform nonlinear optimization using the edge overlap of the calibration plate. It can be implemented according to the specific plan to obtain the best result.

[0135] In some embodiments, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the steps of the joint calibration method of radar and camera described in any embodiment of the present application.

[0136] In some embodiments, the present application provides a radar and camera joint calibration system, comprising: a camera configured to capture an image of a calibration plate; a radar configured to collect point cloud data of the calibration plate; and a computer device as described in any embodiment of the present application. The calibration plate adopts the calibration plate structure in any of the aforementioned embodiments of the present application.

[0137] In some embodiments, an embodiment of the present application provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of the joint calibration method of radar and camera described in any embodiment of the present application are implemented.

[0138] In some embodiments, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the joint calibration method of radar and camera described in any embodiment of the present application.

[0139] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of actions combined, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0140] Figure 7 FIG. 1 is a schematic diagram of the hardware structure of a computer device for executing a joint calibration method of a radar and a camera provided in another embodiment of the present application. Figure 7 As shown, the device includes:

[0141] One or more processors 710 and memory 720, Figure 7 A processor 710 is taken as an example.

[0142] The device for executing the joint calibration method of radar and camera may further include: an input device 730 and an output device 740 .

[0143] The processor 710, the memory 720, the input device 730 and the output device 740 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0144] The memory 720, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the radar and camera joint calibration method in the embodiment of the present application. The processor 710 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 720, that is, the radar and camera joint calibration method in the above method embodiment is implemented.

[0145] The memory 720 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the joint calibration device of the radar and camera, etc. In addition, the memory 720 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 720 may optionally include a memory remotely arranged relative to the processor 710, and these remote memories may be connected to the joint calibration device of the radar and camera via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0146] The input device 730 can receive input digital or character information and generate signals related to user settings and function control of the radar and camera joint calibration device. The output device 740 can include display devices such as display screens.

[0147] The one or more modules are stored in the memory 720, and when executed by the one or more processors 710, perform the joint calibration of the radar and the camera in any of the above method embodiments.

[0148] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0149] The computer device of the embodiment of the present application exists in various forms, including but not limited to:

[0150] (1) Mobile communication devices: These devices are characterized by their mobile communication functions and their main purpose is to provide voice and data communications. These terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

[0151] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, such as iPad.

[0152] (3) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to the general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, manageability, etc.

[0153] (4) Other electronic devices with data interaction functions.

[0154] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A joint calibration method for radar and camera, characterized in that: The method comprises: Acquire an image of a calibration plate taken by the camera, wherein the calibration plate is provided with a positioning mark and a graphic mark; Calculate the pixel coordinates of the center of the graphic of the graphic identifier according to the positioning identifier and the graphic identifier; Acquire point cloud data of the calibration plate collected by the radar, and calculate the three-dimensional coordinates of the first graphic center of the center of the graphic identifier in the radar coordinate system according to the data corresponding to the graphic identifier in the point cloud data; Calculate the initial external parameters between the radar and the camera according to the pixel coordinates of the center of the graphic, the three-dimensional coordinates of the center of the first graphic, and the parameter information of the camera; Calculating the center pixel coordinates of the graphic of the graphic identifier according to the positioning identifier and the graphic identifier includes: Calculate the pixel coordinates of the corner points of the positioning mark according to the positioning mark; Calculating a homography matrix between a world coordinate system and a camera coordinate system according to the pixel coordinates of the corner points of the positioning mark and the world coordinates of the corner points of the positioning mark; According to the central world coordinates of the graphic identifier and the homography matrix, the graphic center pixel coordinates of the graphic identifier are calculated; wherein, the graphic identifier is a non-hollow identifier, and the positioning identifier is an Apriltag positioning identifier, and the corner pixel coordinates of the positioning identifier are calculated according to the positioning identifier, including: using the opencv library function to convert the image into a grayscale image, and using the Apriltag library to create an Apriltag detector object; using the detect method of the Apriltag detector object to perform identification detection on the Apriltag positioning identifier in the grayscale image, and combining the size of the Apriltag positioning identifier involved in the camera to obtain the corner pixel coordinates of each Apriltag positioning identifier.

2. The method according to claim 1, characterized in that Also includes: The world coordinates of the corner points of the positioning mark and the world coordinates of the center of the graphic mark are calculated according to the size of the calibration plate and the size of the positioning mark.

3. The method according to claim 1, characterized in that The point cloud data of the calibration plate is collected by radar, and the three-dimensional coordinates of the first graphic center of the center of the graphic mark in the radar coordinate system are calculated according to the data corresponding to the graphic mark in the point cloud data, including: The point cloud data in the calibration site is collected by using a radar, and the calibration plate is arranged in the calibration site; Filtering the point cloud data to obtain target point cloud data corresponding to the calibration plate; The first graphic center three-dimensional coordinates of the center of the graphic identifier in the radar coordinate system are calculated according to the point cloud data corresponding to the graphic identifier in the target point cloud data.

4. The method according to claim 3, characterized in that Calculating the first graphic center three-dimensional coordinates of the center of the graphic mark in the radar coordinate system according to the point cloud data corresponding to the graphic mark in the target point cloud data includes: Converting the target point cloud data into a grayscale image; The three-dimensional coordinates of the first graphic center of the center of the graphic identifier in the radar coordinate system are calculated according to the graphic features corresponding to the graphic identifier in the grayscale image.

5. The method according to claim 3, characterized in that: The point cloud data includes a plurality of point cloud planes, and filtering the point cloud data to obtain target point cloud data corresponding to the calibration plate includes: Acquire plane features of the plurality of point cloud planes, the plane features comprising at least one of the following: a plane unit normal vector, a plane size, and a plane distance; The point cloud data is filtered based on the plane features to obtain target point cloud data corresponding to the calibration plate.

6. The method according to claim 1, characterized in that Also includes: Calculate the central translation vector and rotation matrix of the positioning mark according to the positioning mark; According to the central translation vector and rotation matrix of the positioning mark, combined with the central world coordinates of the graphic mark, the three-dimensional coordinates of the second graphic center of the center of the graphic mark in the camera coordinate system are calculated; According to the three-dimensional coordinates of the center of the first graphic and the three-dimensional coordinates of the center of the second graphic, the initial external parameters are optimized to obtain target external parameters.

7. The method according to any one of claims 1 to 6, characterized in that The graphic mark is a circular mark, and the grayscale difference between the inner and outer parts of the circular mark is greater than a preset value.

8. The method according to claim 7, characterized in that There are multiple calibration plates, and the multiple calibration plates are arranged in different positions in the calibration site.

9. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.

10. A radar and camera joint calibration system, characterized in that: include: a camera configured to capture an image of a calibration plate; A radar configured to collect point cloud data of a calibration plate; and The computer device as claimed in claim 9.

11. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images