A laser radar and camera joint calibration method and system

Abstract

The present invention provides a joint calibration method and system for a laser radar and a camera, and belongs to the field of data calibration technology. The evaluation method comprises: collecting point cloud data and image data of each target through a laser radar and a camera respectively; predicting the first corner point position of a calibration plate according to the point cloud data and in combination with a point cloud corner point position prediction algorithm; predicting the second corner point position of a calibration plate according to the image data through a corner point detection algorithm; matching the first corner point position and the second corner point position through a corner point matching algorithm to generate a coordinate set for each target; based on the coordinate set, solving the calibration parameters for adjusting the placement position of the laser radar relative to the camera through a PnP algorithm, wherein the calibration parameters include a rotation angle and a translation angle. The present invention can effectively reduce the risk of overfitting, improve the calibration accuracy and efficiency, and is simple to operate, highly automated, and has broad application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of data calibration, and in particular to a joint calibration method and system for a laser radar and a camera. Background Art

[0002] Joint calibration refers to the process of calibrating both LiDAR and camera sensing devices simultaneously to ensure their consistency and precise coordination in spatial positioning and data capture.

[0003] With the continuous development of Lidar and camera technology, joint calibration has become an important means to improve the accuracy of autonomous driving and robot vision systems. Joint calibration is of great significance to the overall stability and reliability of the system, and to reducing costs and energy consumption, especially in safety-critical applications that require highly accurate and reliable information, such as autonomous driving and drone navigation.

[0004] Existing joint calibration requires professionals to manually adjust parameters and perform calibration in a dedicated space to complete the joint calibration process. Such joint calibration methods are often more complicated. Both manual tuning and calibration between calibration sites require high costs and professionals, and are inefficient. Although the calibration cost of the calibration plate is low, the accuracy is insufficient and it is easy to overfit in the calibration plate area, which cannot meet the accuracy requirements of the joint calibration well. Summary of the invention

[0005] The main purpose of the present invention is to provide a joint calibration method and system for a laser radar and a camera, so as to solve the problem that in the prior art, joint calibration requires professionals to manually adjust parameters, resulting in low efficiency, insufficient precision, and easy overfitting in the calibration plate area, and cannot well meet the accuracy requirements of the joint calibration.

[0006] In order to achieve the above object, according to one aspect of the present invention, a joint calibration method of a laser radar and a camera is provided, comprising:

[0007] S1: Collect point cloud data and image data of each target through laser radar and camera respectively;

[0008] S2: Based on the point cloud data and the point cloud corner point position prediction algorithm, the first corner point position of the calibration plate is predicted;

[0009] S3: predicting the position of the second corner point of the calibration plate through a corner point detection algorithm according to the image data;

[0010] S4: Matching the first corner point position and the second corner point position by a corner point matching algorithm to generate a coordinate set for each target;

[0011] S5: Based on the coordinate set, the calibration parameters for adjusting the position of the laser radar relative to the camera are solved by the PnP algorithm, where the calibration parameters include a rotation angle and a translation angle.

[0012] Further, the targets include a first target, a second target and a third target;

[0013] The bottom edge of the first target is parallel to the line between the calibration plate and the first target;

[0014] The bottom side of the second target is parallel to the line between the first target and the third target;

[0015] The bottom edge of the third target is parallel to the line between the calibration plate and the third target;

[0016] The first target, the second target and the third target are all on the same horizontal line.

[0017] Furthermore, S2 specifically includes:

[0018] Load point cloud data through the PointCloud tool in the Open3D library to obtain a pcd object;

[0019] According to the pcd object, extract the point cloud points in the preset area of ​​the point cloud data;

[0020] Use the segment_plane tool in the pcd object to fit the plane where the calibration plate is located and intercept the inner points of the plane;

[0021] Construct a rigid body transformation matrix based on the normal vector and center point of the calibration plate plane;

[0022] The calibration plate plane is transformed to the XY plane through the rigid body transformation matrix;

[0023] Use the half-measure method to find the optimal angle of the points on the XY plane and complete the rough alignment in the XY plane;

[0024] The point set on the XY plane is clustered by the kmeans clustering algorithm to obtain key clusters;

[0025] Through the PnP algorithm, the center points of the key clusters are matched with the template corner points, and the rotation matrix and translation vector of the calibration plate are calculated;

[0026] Through inverse transformation, the template corner point is transformed into 3D space to obtain the first corner point position.

[0027] Furthermore, the corner detection algorithm is specifically a Harris corner detection algorithm.

[0028] Furthermore, S3 specifically includes:

[0029] Preprocess the image data;

[0030] The preprocessed image data is detected by the Harris corner detection algorithm to generate the second corner point position of the calibration plate, where the second corner point position is expressed in the form of a set:

[0031] Cimg={(uk,vk)}

[0032] Among them, C img represents the second corner point position set, u k Indicates the horizontal position of the kth corner point in the image, v k Indicates the vertical position of the kth corner point in the image.

[0033] Furthermore, the preprocessing includes: grayscale processing and smoothing filtering processing.

[0034] Furthermore, the corner point matching algorithm is specifically a nearest neighbor algorithm or a sub-matching algorithm based on feature description.

[0035] Furthermore, the coordinate set is specifically:

[0036] S={(U img ,V img ,X pcg ,Y pcd ,Z pcd )}

[0037] Among them, S represents the coordinate set, U img Indicates the horizontal coordinate position of the corner point in the image, V img Indicates the vertical coordinate position of the corner point in the image, X pcg Indicates the coordinate position of the point cloud data in the three-dimensional coordinate system along the X axis, pcg Indicates the coordinate position of the point cloud data in the Y-axis direction in the three-dimensional coordinate system, Z pcg Indicates the coordinate position of the point cloud data in the Z-axis direction in the three-dimensional coordinate system.

[0038] Furthermore, S5 specifically includes:

[0039] Using the PnP algorithm, calculate the rotation matrix and translation vector of the calibration plate:

[0040]

[0041] Among them, T represents the transformation matrix, R represents the rotation matrix, and t represents the translation vector;

[0042] Generates the rotation angle and translation angle based on the rotation matrix and translation vector.

[0043] According to one aspect of the present invention, a joint calibration system of a laser radar and a camera is provided, comprising:

[0044] processor;

[0045] A memory having computer-readable instructions stored therein, which, when executed by a processor, implements the above-mentioned joint calibration method of the laser radar and the camera.

[0046] The technical solution of the present invention is applied. The joint calibration method of the laser radar and the camera provided by the present invention collects the point cloud data and image data of each target through the laser radar and the camera respectively, and combines the point cloud corner point position prediction algorithm and the corner point detection algorithm to predict the first corner point position and the second corner point position of the calibration plate, and matches the first corner point position and the second corner point position through the corner point matching algorithm to generate a coordinate set for each target. Finally, based on the coordinate set, the calibration parameters for adjusting the placement position of the laser radar relative to the camera are solved through the PnP algorithm. It effectively reduces the risk of overfitting, improves the calibration accuracy and efficiency, and is simple to operate, highly automated, and has broad application prospects.

[0047] In addition to the above-described objects, features and advantages, the present invention has other objects, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0049] Figure 1 A schematic diagram of a process flow of a joint calibration method of a laser radar and a camera provided in an embodiment of the present invention is shown;

[0050] Figure 2 A schematic diagram of the structure of a target placement provided by an embodiment of the present invention is shown;

[0051] Figure 3 A structural schematic diagram of a joint calibration system of a laser radar and a camera provided in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0052] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0053] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0054] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the present invention described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0055] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0056] Reference Manual Attached Figure 1 , shows a flow chart of a joint calibration method of a laser radar and a camera provided in an embodiment of the present invention.

[0057] An embodiment of the present invention provides a joint calibration method for a laser radar and a camera, comprising:

[0058] S1: Collect point cloud data and image data of each target through lidar and camera respectively.

[0059] Among them, lidar is a device that uses lasers to measure the distance and speed of objects. It obtains information by emitting laser pulses and measuring the reflected signals. A camera is an electronic device used to capture and record images, which can convert visible light into electronic signals. Point cloud data is a data set composed of a large number of three-dimensional points in space. Each point contains location information and is usually generated by lidar. Image data is a visual representation composed of a two-dimensional array of pixels. Each pixel contains color information and is usually captured by a camera.

[0060] Optionally, point cloud data and image data of the vehicle are collected.

[0061] It should be noted that the point cloud data and image data of each target are collected separately. This step ensures the diversity and integrity of the data. The point cloud data provides accurate three-dimensional spatial information, while the image data provides rich color and texture information. The joint use of this data can greatly enhance the accuracy and efficiency of subsequent processing, which is particularly important when performing sensor fusion and calibration in complex environments.

[0062] Reference Manual Attached Figure 2 , showing a schematic structural diagram of a target placement provided by an embodiment of the present invention.

[0063] Figure 2 , an example of a feasible placement position is shown, wherein A represents a calibration plate, B, C and D represent the first target, the second target and the third target respectively, the calibration plate is located at the top center of the structure, 1.5 meters from the ground, the length of the bottom edge BCD is 5.2 meters, and it is divided into two sections, each section is 2.6 meters, and the vertical distance from the bottom edge to the bottom of the structure is 7.5 meters, the placement of the second target and the third target is mirror symmetrical with respect to AC, the bottom edge of the first target is parallel to the line between the calibration plate and the first target, and the length of the parallel line and the center distance of the first target are both 1.5 meters, the bottom edge of the second target is parallel to the line between the first target and the third target, and the length of the parallel line and the center distance of the second target are both 1.5 meters.

[0064] In a possible implementation, the targets include a first target, a second target, and a third target.

[0065] The bottom edge of the first target is parallel to a line connecting the calibration plate and the first target.

[0066] The bottom side of the second target is parallel to the line between the first target and the third target.

[0067] The bottom edge of the third target is parallel to a line connecting the calibration plate and the third target.

[0068] The first target, the second target and the third target are all on the same horizontal line.

[0069] S2: Based on the point cloud data and the point cloud corner point position prediction algorithm, the first corner point position of the calibration plate is predicted.

[0070] Among them, the point cloud corner point position prediction algorithm is a calculation method used to identify and predict the position of corner points from point cloud data. Corner points are points on the surface of an object where the geometric shape changes significantly, such as the edge or vertex of an object. The calibration plate is a device used for measurement and calibration, usually with obvious boundaries and corner points. The first corner point position refers to the spatial coordinates of a key corner point on the calibration plate predicted by point cloud data.

[0071] It should be noted that the improved point cloud corner position prediction algorithm is used to predict the positions of all corner points of the calibration plate, not just the four vertices, ensuring accurate data docking and precise calibration of the final device. Accurate corner point prediction allows the system to accurately locate the calibration plate in three-dimensional space, thereby improving the overall quality and efficiency of calibration, especially in applications that require precise alignment of multiple sensors. This method utilizes the high-dimensional information of point cloud data, optimizes the calibration process, and ensures the accuracy and reliability of data processing.

[0072] In a possible implementation, S2 specifically includes:

[0073] Load point cloud data through the PointCloud tool in the Open3D library to obtain a pcd object.

[0074] According to the pcd object, extract the point cloud points within the preset area in the point cloud data.

[0075] Use the segment_plane tool in the pcd object to fit the plane where the calibration plate is located and intercept the inner points of the plane.

[0076] Construct the rigid body transformation matrix according to the normal vector and center point of the calibration plate plane.

[0077] The calibration plate plane is transformed to the XY plane through the rigid body transformation matrix.

[0078] The optimal angle of the points on the XY plane is found by the half method to complete the rough alignment in the XY plane.

[0079] The point set on the XY plane is clustered using the kmeans clustering algorithm to obtain key clusters.

[0080] Through the PnP algorithm, the center points of the key clusters are matched with the template corner points, and the rotation matrix and translation vector of the calibration plate are calculated.

[0081] Through inverse transformation, the template corner point is transformed into 3D space to obtain the first corner point position.

[0082] It should be noted that the PointCloud tool of the Open3D library is used to load and analyze the point cloud data, the segment_plane tool is used to effectively fit the plane and extract the relevant point set, and then the point set is transferred to the XY plane through rigid body transformation. The binary method and kmeans clustering are used to accurately predict the corner points of the calibration plate. This method not only improves the accuracy of data processing, but also optimizes the calculation efficiency. It is a key step to achieve high-quality calibration.

[0083] S3: Predict the position of the second corner point of the calibration plate through a corner point detection algorithm according to the image data.

[0084] Among them, the corner detection algorithm is an image processing technology used to automatically identify corners from an image. These corners are locations where the geometric features in the image change significantly, and are commonly found at the edges and vertices of objects. The second corner point position refers to the position of another key corner point on the calibration plate predicted by the image data and the corner detection algorithm.

[0085] Optionally, the corner detection algorithm is specifically a Harris corner detection algorithm.

[0086] It should be noted that the use of advanced corner detection algorithms (such as the Harris corner detection algorithm) to accurately identify and locate the corners of the calibration plate is crucial to the entire calibration process, because the corner positions in the image provide the necessary two-dimensional reference for the subsequent calibration parameter calculation. Accurate corner detection not only improves the availability of data, but also enhances the accuracy of multi-sensor data fusion, allowing the lidar and camera data to be better aligned and complementary in space. Such processing not only optimizes the accuracy of calibration, but also provides strong support for sensor integration in a variety of application scenarios.

[0087] In a possible implementation, S3 specifically includes:

[0088] Preprocess the image data.

[0089] Specifically, the preprocessing includes: grayscale processing and smoothing filtering processing.

[0090] The preprocessed image data is detected by the Harris corner detection algorithm to generate the second corner point position of the calibration plate, where the second corner point position is expressed in the form of a set:

[0091] C img ={(u k ,v k )}

[0092] Among them, C img represents the second corner point position set, u kIndicates the horizontal position of the kth corner point in the image, v k Indicates the vertical position of the kth corner point in the image.

[0093] S4: Match the first corner point position and the second corner point position through a corner point matching algorithm to generate a coordinate set for each target.

[0094] Among them, the corner point matching algorithm is a computational method for pairing corner points identified in different data sources. It is usually used to ensure the spatial consistency of two sets of data (such as point clouds and image data). The coordinate set is a set composed of the matched corner point positions, including the coordinate position of each corner point in three-dimensional space, which is used for subsequent geometric transformation and calibration calculations.

[0095] It should be noted that the data from the lidar and camera can be effectively integrated through the precise corner matching algorithm. The key to this step is that it ensures that the data about the same physical object captured by two different sensors can be accurately aligned in space. Accurate corner matching is the basis for efficient data fusion. It enables the subsequent PnP algorithm to accurately calculate the required rotation matrix and translation vector, thereby achieving accurate device calibration.

[0096] In a possible implementation, the corner point matching algorithm is specifically a nearest neighbor algorithm or a sub-matching algorithm based on feature description.

[0097] Among them, the nearest neighbor algorithm is a simple and intuitive search algorithm used to find the nearest point to a given point in a point set. In the corner point matching scenario, the algorithm is used to find the closest matching point, usually based on the Euclidean distance (or other distance metrics) for comparison. The algorithm is suitable for fast matching between different data sets, such as matching the corner points detected in image data with the corner points in point cloud data.

[0098] The feature description-based sub-matching algorithm focuses on using complex feature descriptors to achieve accurate data point matching. This algorithm first extracts feature descriptors for each data point (such as corner points in an image), and then uses these descriptors to find the best matching pairs. Feature descriptors can include pixel intensity, gradient, direction, or other local image properties around corner points, which are used to enhance the robustness and accuracy of matching.

[0099] In a possible implementation, the coordinate set is specifically:

[0100] S={(U img ,V img ,X pcg ,Y pcd ,Z pcd )}

[0101] Among them, S represents the coordinate set, U img Indicates the horizontal coordinate position of the corner point in the image, V img Indicates the vertical coordinate position of the corner point in the image, X pcg Indicates the coordinate position of the point cloud data in the three-dimensional coordinate system along the X axis, pcg Indicates the coordinate position of the point cloud data in the Y-axis direction in the three-dimensional coordinate system, Z pcg Indicates the coordinate position of the point cloud data in the Z-axis direction in the three-dimensional coordinate system.

[0102] S5: Based on the coordinate set, the calibration parameters for adjusting the position of the laser radar relative to the camera are solved by the PnP algorithm, where the calibration parameters include a rotation angle and a translation angle.

[0103] Among them, the PnP algorithm (Perspective-n-Point) is an algorithm that calculates the relationship between the three-dimensional point under the camera's perspective and its projected position in the two-dimensional image. During the calibration process, it is used to solve the position and direction of the camera based on several known three-dimensional points and their projected positions in the image. The calibration parameters refer to a set of values ​​or settings used to adjust the measuring device to ensure that its output matches the actual value. The rotation angle refers to the rotation angle parameter that helps determine the direction of the lidar relative to the camera in space during sensor calibration, ensuring the consistency of the perspectives of the two and correct data alignment. The translation angle refers to the calibration process that helps ensure that the lidar and camera are not only aligned in direction, but also correctly configured in physical position to accurately capture and interpret data.

[0104] It should be noted that by applying the PnP algorithm, this step can calculate the optimal camera position and direction based on several known three-dimensional space points and their corresponding two-dimensional image projections. This method not only improves the accuracy of the calibration process, but also greatly enhances the system's adaptability and accuracy to complex environments. In the fields of autonomous driving, robot vision, etc., accurate equipment calibration is crucial because it directly affects the system's perception and response capabilities to the surrounding environment.

[0105] In a possible implementation, S5 specifically includes:

[0106] Using the PnP algorithm, calculate the rotation matrix and translation vector of the calibration plate:

[0107]

[0108] Among them, T represents the transformation matrix, R represents the rotation matrix, and t represents the translation vector.

[0109] Generates the rotation angle and translation angle based on the rotation matrix and translation vector.

[0110] From the above description, it can be seen that the joint calibration method of the laser radar and camera provided by the present invention collects the point cloud data and image data of each target through the laser radar and the camera respectively, and combines the point cloud corner point position prediction algorithm and the corner point detection algorithm to predict the first corner point position and the second corner point position of the calibration plate, and matches the first corner point position and the second corner point position through the corner point matching algorithm to generate a coordinate set for each target. Finally, based on the coordinate set, the calibration parameters for adjusting the placement position of the laser radar relative to the camera are solved through the PnP algorithm. It effectively reduces the risk of overfitting, improves the calibration accuracy and efficiency, and is simple to operate, highly automated, and has broad application prospects.

[0111] Reference Manual Attached Figure 3 , shows a structural schematic diagram of a joint calibration system of a laser radar and a camera provided in an embodiment of the present invention.

[0112] The embodiment of the present invention provides a laser radar and camera joint calibration system 20, including:

[0113] Processor 201;

[0114] The memory 202 stores computer-readable instructions, and when the computer-readable instructions are executed by the processor 201, the above-mentioned joint calibration method of the laser radar and the camera is implemented.

[0115] From the above description, it can be seen that the joint calibration method of the laser radar and camera provided by the present invention collects the point cloud data and image data of each target through the laser radar and the camera respectively, and combines the point cloud corner point position prediction algorithm and the corner point detection algorithm to predict the first corner point position and the second corner point position of the calibration plate, and matches the first corner point position and the second corner point position through the corner point matching algorithm to generate a coordinate set for each target. Finally, based on the coordinate set, the calibration parameters for adjusting the placement position of the laser radar relative to the camera are solved through the PnP algorithm. It effectively reduces the risk of overfitting, improves the calibration accuracy and efficiency, and is simple to operate, highly automated, and has broad application prospects.

[0116] Unless otherwise specifically stated, the relative arrangement of the parts and steps described in these embodiments, numerical expressions and numerical values ​​do not limit the scope of the present invention. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, methods and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technology, methods and equipment should be regarded as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0117] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0118] In the description of the present invention, it is necessary to understand that the directions or positional relationships indicated by directional words such as "front, back, up, down, left, right", "lateral, vertical, perpendicular, horizontal" and "top, bottom" are usually based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description. Unless otherwise specified, these directional words do not indicate or imply that the devices or elements referred to must have a specific direction or be constructed and operated in a specific direction. Therefore, they cannot be understood as limiting the scope of protection of the present invention. The directional words "inside and outside" refer to the inside and outside relative to the contours of each component itself.

[0119] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A joint calibration method for a laser radar and a camera, characterized in that: Applied to a joint calibration platform, wherein the joint calibration platform includes a calibration plate and multiple targets located in the overlapping area of ​​the field of view between the laser radar and the camera; the method includes: S1: Collect point cloud data and image data of each target through laser radar and camera respectively; S2: predicting the first corner point position of the calibration plate according to the point cloud data and in combination with a point cloud corner point position prediction algorithm; S3: predicting the position of the second corner point of the calibration plate by a corner point detection algorithm according to the image data; S4: matching the first corner point position and the second corner point position by a corner point matching algorithm to generate a coordinate set for each target; S5: Based on the coordinate set, a PnP algorithm is used to solve calibration parameters for adjusting the placement position of the laser radar relative to the camera, wherein the calibration parameters include a rotation angle and a translation angle.

2. The joint calibration method of laser radar and camera according to claim 1, characterized in that: The targets include a first target, a second target and a third target; The bottom edge of the first target is parallel to the line between the calibration plate and the first target; The bottom edge of the second target is parallel to the line between the first target and the third target; The bottom edge of the third target is parallel to the line between the calibration plate and the third target; The first target, the second target and the third target are all on the same horizontal line.

3. The joint calibration method of laser radar and camera according to claim 1, characterized in that: The S2 specifically includes: Load point cloud data through the PointCloud tool in the Open3D library to obtain a pcd object; According to the pcd object, extracting point cloud points within a preset area in the point cloud data; Use the segment_plane tool in the pcd object to fit the plane where the calibration plate is located and intercept the inner points of the plane; Constructing a rigid body transformation matrix according to the normal vector and the center point of the calibration plate plane; Transforming the calibration plate plane to the XY plane through the rigid body transformation matrix; Find the optimal angle of the points on the XY plane by using the half method to complete the rough alignment in the XY plane; The point set on the XY plane is clustered by the kmeans clustering algorithm to obtain key clusters; By using the PnP algorithm, the center point of the key cluster is matched with the template corner point, and the rotation matrix and translation vector of the calibration plate are calculated; By inverse transformation, the template corner point is transformed into 3D space to obtain the first corner point position.

4. The joint calibration method of laser radar and camera according to claim 1, characterized in that: The corner detection algorithm is specifically the Harris corner detection algorithm.

5. The joint calibration method of laser radar and camera according to claim 4, characterized in that: The S3 specifically includes: Preprocessing the image data; The preprocessed image data is detected by the Harris corner detection algorithm to generate the second corner point position of the calibration plate, wherein the second corner point position is represented in the form of a set: Cimg={(uk,vk)} Among them, C img represents the second corner point position set, u k Indicates the horizontal position of the kth corner point in the image, v k Indicates the vertical position of the kth corner point in the image.

6. The joint calibration method of laser radar and camera according to claim 5, characterized in that: The preprocessing includes: grayscale processing and smoothing filtering processing.

7. The joint calibration method of laser radar and camera according to claim 1, characterized in that: The corner point matching algorithm is specifically a nearest neighbor algorithm or a sub-matching algorithm based on feature description.

8. The joint calibration method of laser radar and camera according to claim 1, characterized in that: The coordinate set is specifically: S={(U img ,V img ,X pcg ,Y pcd ,Z pcd )} Among them, S represents the coordinate set, U img Indicates the horizontal coordinate position of the corner point in the image, V img Indicates the vertical coordinate position of the corner point in the image, X pcg Indicates the coordinate position of the point cloud data in the three-dimensional coordinate system along the X axis, pcg Indicates the coordinate position of the point cloud data in the 3D coordinate system along the Y axis, pcg Indicates the coordinate position of the point cloud data in the Z-axis direction in the three-dimensional coordinate system.

9. The joint calibration method of laser radar and camera according to claim 1, characterized in that: The S5 specifically includes: The rotation matrix and translation vector of the calibration plate are calculated by the PnP algorithm: Among them, T represents the transformation matrix, R represents the rotation matrix, and t represents the translation vector; The rotation angle and the translation angle are generated according to the rotation matrix and the translation vector.

10. A laser radar and camera joint calibration system, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the joint calibration method of the laser radar and the camera as described in any one of claims 1 to 9 is implemented.

Images