Calibration board and calibration method for joint calibration of laser radar and camera

Abstract

The present invention discloses a calibration plate and a calibration method for joint calibration of a laser radar and a camera, belonging to the technical field of multi-sensor data fusion. The present invention creates a new calibration plate for joint calibration of a laser radar and a camera, the calibration plate including a laser radar marker and an image ArUco marker, the laser radar marker is used for automatic recognition of the laser radar marker, the image ArUco marker is used for automatic recognition of the image marker, the laser radar marker includes a start and end mark and a number of marking points, the start and end marks include a start mark and a stop mark, and the image ArUco marker includes two ArUco markers with equal areas and different IDs; at the same time, a new calibration method for joint calibration of a laser radar and a camera is established based on the calibration plate, the laser radar marker and the image ArUco marker are collected and matched, an optimization model is established, the objective function of the optimization model is output, the external parameters are optimized and solved, and the final calibration result is output, thereby realizing the universal modularization of the joint calibration method of the laser radar and the camera.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-sensor data fusion, and in particular to a calibration plate and a calibration method for joint calibration of a laser radar and a camera. Background Art

[0002] Using multiple sensors enables machines to perceive the world like humans. When multiple sensors are connected and perceived, they need to be fused. Fusion is to achieve temporal and spatial synchronization of different sensors. Joint calibration of LiDAR and cameras is to achieve spatial synchronization.

[0003] The joint calibration between multiple sensors is one of the important foundations for realizing multi-sensor fusion. How to quickly and accurately obtain the position conversion relationship between the coordinate systems of multiple sensors has always been a hot research topic. The biggest problem with the external parameter calibration between different sensors is how to obtain the best measurement. The data types obtained are different. The camera obtains the pixel array of the RGB image, while the lidar obtains the 3-D point cloud distance information. LiDAR is divided into multi-line lidar and single-line lidar. Multi-line lidar has a large viewing angle, high accuracy, and depth detection performance. Multi-line lidar is mainly used for 3D modeling and environmental perception, and synchronous map enhancement positioning. However, multi-line lidar is expensive, while single-line lidar has low cost, fast scanning speed, strong resolution, and high reliability. Single-line lidar responds faster in angular frequency and sensitivity, so it is more accurate in testing the distance and accuracy of surrounding obstacles, but single-line lidar can only scan in a plane and cannot measure the height of objects. In general, the application scenarios of multi-line lidar are more complex and have higher performance requirements, but it is expensive.

[0004] Some existing static LiDAR and camera joint calibration methods use LiDAR range jumps to find edge points and fit calibration plate vertices. The camera then obtains the corresponding vertex coordinates by identifying camera markers and prior information. However, there is a problem that LiDAR points at edge points have large errors, and line fitting can amplify these errors.

[0005] The checkerboard calibration plate provided by the official website is difficult to accurately locate the corner marks of the checkerboard during LiDAR scanning due to the unlimited scanning angle and range of the LiDAR. This reduces the accuracy. In addition, the inconsistent sizes of different checkerboards lead to problems with versatility and scalability.

[0006] Another method is to randomly place lidar calibration markers around the camera marker, obtain the prior feature point position information by measuring the relative position, and find the radar feature points by the laser reflection intensity. However, this method has the disadvantage that the calibration plate is made too casually, resulting in a narrow scope of application and difficulty in finding sufficiently effective lidar marker points. In addition, when the placement of the radar and camera changes significantly, the calibration plate needs to be remade to complete the calibration. In addition, this method usually requires the calibration plate to be stationary and is only applicable to the joint calibration of multi-line lidar and camera.

[0007] In the field of autonomous driving, multi-line lidar is needed to achieve more functions. However, the multi-sensor fusion of lidar and camera is not only reflected in the field of autonomous driving. In fields where the demand for lidar functions is not so high, such as robots, mature single-line lidar is more reliable than multi-line lidar and is suitable for more scenarios. How to realize a universal lidar and camera joint calibration method applicable to different scenarios is a problem that we urgently need to solve. Summary of the Invention

[0008] The purpose of the present invention is to provide a calibration plate and calibration method for joint calibration of laser radar and camera, so as to solve the problems raised in the above background technology.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions: a calibration plate for joint calibration of lidar and camera, the calibration plate including a lidar marker and an image ArUco marker, the lidar marker is used for automatic recognition of the lidar marker, and the image ArUco marker is used for automatic recognition of the image marker.

[0010] The black boundaries of the LiDAR markers and image ArUco markers help to quickly detect image and image ArUco markers;

[0011] The laser radar marker includes a start and end mark and several marking points. The start and end marks include a start mark and an end mark. The start mark includes a black rectangle with a width of two units and a black rectangle with a width of one unit. The start mark and the end mark are horizontally mirrored about the center point of the several marking points. The marking points are black images with a width of one unit. The interval between the marking points is a white blank of one unit. The interval between the marking points and the start and end marks is a white blank of one unit.

[0012] The lidar marker sets the start and end marks, effectively measuring the scanning angle and range of the lidar, speeding up the lidar scanning, improving the accuracy of the lidar scanning, and effectively improving the accuracy of the joint calibration of the lidar and camera.

[0013] The image ArUco marker includes a first ArUco marker and a second ArUco marker, the first ArUco marker and the second ArUco marker have equal areas and different IDs, the first ArUco marker and the second ArUco marker include a black border and a binary matrix, and the binary matrix expresses the ArUco marker ID.

[0014] The first ArUco marker and the second ArUco marker are parallel and equidistant on both sides of the laser radar marker, and the centers of the first ArUco marker and the second ArUco marker are coaxial with the center of the laser radar middle marker in the vertical direction.

[0015] The marker coordinates of all LiDAR markers and image ArUco markers in the calibration plate are determined and known.

[0016] A calibration method for joint calibration of a laser radar and a camera, characterized in that the joint calibration method specifically includes:

[0017] Step 1: Complete the camera intrinsic calibration, enter the intrinsic calibration data, the size of the calibration plate and the coordinates of all markers in the calibration plate into the calibration system, and establish an automatic recognition system for lidar markers and image ArUco markers based on the ROS system. The calibration system includes a ROS startup module, a data visualization module, a data acquisition module, a marker recognition module and an optimization solution module;

[0018] Step 2: The ROS startup module starts the camera and lidar, and records the corresponding topic into the data acquisition module;

[0019] Step 3: The marker recognition module recognizes and matches the lidar marker and the image ArUco marker, and the data visualization module simultaneously displays the progress of feature point data collection of the marker recognition module;

[0020] Step 4: Hold the calibration plate facing the camera and slowly move it within the camera's field of view until the number of data collected by the marker recognition module reaches the threshold;

[0021] Step 5: After data collection is completed, the optimization solution module calculates external parameters and outputs calibration results;

[0022] Step 6: The data visualization module observes the calibration results. If the lidar points in the projection results overlap well with the camera images, the calibration results are considered valid. Otherwise, check for problems in the process and re-calibrate the lidar and camera.

[0023] Slowly move the calibration plate within the camera's field of view. The calibration system identifies the image ArUco markers and filters out invalid data until the number of data collected by the marker recognition module reaches a threshold and outputs the image recognition result.

[0024] The calibration system establishes a laser radar marker and image ArUco marker automatic recognition system based on the ROS system. The calibration system includes a ROS startup module, a data acquisition module, a marker recognition module, a feature point matching module and a data optimization module;

[0025] The ROS startup system is used to start the camera and lidar. The data acquisition system records the encapsulation module of the topic communication corresponding to the camera and lidar calibration, the camera intrinsic calibration data, the size of the calibration plate and the coordinates of all marks in the calibration plate; the data visualization module synchronously displays the calibration data acquisition progress, projects the lidar points onto the image plane to observe the calibration results; the mark recognition system recognizes the lidar mark and the image ArUco mark, and matches the lidar mark and the image ArUco mark; the optimization solution module calculates the external parameters and outputs the calibration results.

[0026] The specific contents of step four and step five include: holding the calibration plate facing the camera, the data visualization module synchronously displays the progress of laser radar marker and image ArUco marker acquisition and matching, slowly moving the calibration plate, and when the laser radar marker and image ArUco marker can be recognized at the same time, it is valid data, recording the number of valid data, and making the calibration program obtain as diverse three-dimensional space sample point data as possible, until the number of data collected by the marker recognition module reaches a threshold, ending the data acquisition process, and optimizing the solution of external parameters.

[0027] The specific steps of collecting ArUco marks on the image include:

[0028] Step 1.1: Set a time window T, collect image data I(t, t+T) within the time window, establish coordinate systems for the first image ArUco mark and the second image ArUco mark respectively, identify the image data and output a data group, the data group includes the center coordinates C of the first image ArUco mark within the time window T u (t, t+T), the first image aruco marked unit plane normal vector N u (t, t+T), the center coordinate C of the aruco mark in the second image d (t, t+T), the unit plane normal vector N of the aruco marker of the second image d (t, t+T);

[0029] Step 1.2: If the image data does not recognize the first image aruco marker and the second image aruco marker, set the center coordinates of the first image aruco marker, the unit plane normal vector of the first image aruco marker, the center coordinates of the second image aruco marker, and the unit plane normal vector of the second image aruco marker to preset values. When the data continuous acquisition threshold J and the data cumulative acquisition threshold M are set, if the recognition results of J consecutive image data or M cumulative image data within the time window T are the preset values, mark the data segment as invalid data, clear the invalid data and output the updated data group;

[0030] Step 1.3: Calculate the mean variance D of the aruco label of the first image and the aruco label of the second image within the time window. The specific calculation formula is:

[0031] D={std[C′ u (t, t+T)]+std[C′ d (t, t+T)]} / 2

[0032] Among them, C′ u (t, t+T) represents the updated coordinates of the center of the aruco marker of the first image at the time (t, t+T), C′ d (t, t+T) represents the updated coordinates of the center of the aruco marker in the second image at the time (t, t+T);

[0033] Calculate the mean S of the normal vector cosine values of the center coordinates of the ArUco marker of the first image and the ArUco marker of the second image within the time window. The specific calculation formula is:

[0034]

[0035] in, represents the updated aruco unit plane normal vector of the first image at time (t, t+T), represents the updated aruco unit plane normal vector of the second image at time (t, t+T);

[0036] Step 1.4: Set the variance mean threshold and the normal vector cosine mean threshold. When the variance mean D is less than the variance mean threshold and the normal vector cosine mean S is less than the normal vector cosine mean threshold, mark the updated data set as valid data and output the image recognition result C′ u (t, t+T) and C′ d The mean of (t, t+T) and N′ u (t, t+T) and N′ d The mean of (t, t+T).

[0037] The specific contents of the laser radar marker collection include: performing laser radar point threshold filtering based on the reflection intensity and distance range of the laser radar point, projecting the filtered laser radar points line by line onto a virtual image plane, clustering them on the image plane using the density-based clustering method DBSCAN, identifying the start and end symbols of the radar marker, and identifying the laser radar marker through prior information; storing the identification results as calibration data, and performing a linear regression with a step size of t. s The above steps are repeated in the sliding time window until the number of radar calibration data reaches the radar marking threshold and the data collection phase is terminated;

[0038] The specific steps for joint calibration of LiDAR and camera include:

[0039] Step 2.1: Establish the coordinate system of the laser radar to output the coordinates of the marker point (x l ,y l , z l ), establish the camera coordinate system;

[0040] Step 2.2: The camera obtains the transformation relationship [R|T] between the camera marker coordinate system and the camera coordinate system through the prior camera marker size information. m2c ;

[0041] Step 2.3: Output the coordinates of the marker point in the camera coordinate system. The conversion formula is:

[0042]

[0043] Among them, (x lm ,y lm , z lm ) represents the coordinates of the marker point in the camera marker coordinate system, (x lc ,y lc , z lc ) represents the coordinates of the marker point in the camera coordinate system;

[0044] Step 2.4: Output the conversion model between the camera coordinate system and the lidar coordinate system, and optimize the conversion model;

[0045] The specific calculation formula of the conversion model between the camera coordinate system and the lidar coordinate system is:

[0046]

[0047] in, Represents the coordinates of the marker point projected into the camera coordinate system, [R|T] is the external parameter;

[0048] Minimizing the reprojection error optimizes the transformation model to obtain the rotation and translation relationship between the laser radar and camera coordinates. The specific formula of the objective function of the optimization model is:

[0049] Compared with the existing technology, the present invention has the following beneficial effects: the present invention creates a calibration plate for joint calibration of lidar and camera. The camera Aruco marker can be placed at the four corners of the camera image frame. The black border helps to quickly detect the image. Different binary information can be set to represent the camera Aruco marker ID, which can avoid camera misrecognition caused by occlusion and other reasons, and enhance the accuracy of image ArUco marker acquisition.

[0050] The laser radar marker sets the start and end marks, effectively measuring the scanning angle and range of the laser radar, speeding up the laser radar scanning, and improving the accuracy of the laser radar scanning;

[0051] A calibration board for joint calibration of LiDAR and camera can be widely applied to the joint calibration of LiDAR and camera in various environments. A new calibration method for joint calibration of LiDAR and camera is established based on the calibration board, realizing the universal modularization of the joint calibration method of LiDAR and camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0053] Figure 1 It is a structural schematic diagram of a calibration plate for joint calibration of a laser radar and a camera according to the present invention;

[0054] Figure 2 This is a schematic diagram of the calibration process for the joint calibration of lidar and camera;

[0055] Figure 3 It is a flowchart of the algorithm for identifying lidar markers;

[0056] Among them, 1. First ArUco marker; 2. Second ArUco marker; 3. Start character; 4. Marking point; 5. End character. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0058] See also Figure 1-Figure 3 , the present invention provides a technical solution:

[0059] Example 1:

[0060] A calibration plate for joint calibration of a laser radar and a camera, comprising a laser radar marker and an image ArUco marker. The laser radar marker is used for automatic recognition of the laser radar marker, and the image ArUco marker is used for automatic recognition of the image marker.

[0061] The black boundaries of the LiDAR markers and image ArUco markers help to quickly detect image and image ArUco markers;

[0062] The laser radar marker includes start and end marks and several marking points. The start and end marks include a start mark and an end mark. The start mark includes a black rectangle with a width of two units and a black rectangle with a width of one unit. The start mark and the end mark are horizontally mirrored about the center points of the several marking points. The marking points are black images with a width of one unit. The interval between the marking points is a white blank of one unit. The interval between the marking points and the start and end marks is a white blank of one unit.

[0063] The lidar marker sets the start and end marks, effectively measuring the scanning angle and range of the lidar, speeding up the lidar scanning, improving the accuracy of the lidar scanning, and effectively improving the accuracy of the joint calibration of the lidar and camera.

[0064] The image ArUco marker includes a first ArUco marker and a second ArUco marker. The first ArUco marker and the second ArUco marker have equal areas but different IDs. The first ArUco marker and the second ArUco marker include a black border and a binary matrix. The binary matrix expresses the ArUco marker ID.

[0065] The first ArUco marker and the second ArUco marker are parallel and equidistant on both sides of the laser radar marker, and the centers of the first ArUco marker and the second ArUco marker are coaxial with the center of the laser radar middle marker in the vertical direction.

[0066] The marker coordinates of all LiDAR markers and image ArUco markers within the calibration plate are determined and known.

[0067] A calibration method for joint calibration of a laser radar and a camera, characterized in that the joint calibration method specifically includes:

[0068] Step 1: Complete the camera intrinsic calibration, enter the intrinsic calibration data, the size of the calibration plate, and the coordinates of all markers on the calibration plate into the calibration system. The calibration system establishes an automatic recognition system for lidar markers and image ArUco markers based on the ROS system. The calibration system includes the ROS startup module, data visualization module, data acquisition module, marker recognition module, and optimization solution module.

[0069] Step 2: The ROS startup module starts the camera and lidar, and records the corresponding topics into the data acquisition module;

[0070] Step 3: The marker recognition module identifies and matches the lidar markers and image ArUco markers, and the data visualization module simultaneously displays the progress of feature point data collection in the marker recognition module;

[0071] Step 4: Hold the calibration plate facing the camera and slowly move it within the camera's field of view until the number of data collected by the marker recognition module reaches the threshold;

[0072] Step 5: End data collection, optimize the solution module to calculate external parameters, and output the calibration results;

[0073] Step 6: The data visualization module observes the calibration results. If the lidar points in the projection result overlap well with the camera image, the calibration result is considered valid. Otherwise, check for problems in the process and re-calibrate the lidar and camera.

[0074] Slowly move the calibration plate within the camera's field of view. The calibration system recognizes the image ArUco markers and filters out invalid data until the number of data collected by the marker recognition module reaches the threshold and outputs the image recognition result.

[0075] The calibration system establishes an automatic recognition system for lidar markers and image ArUco markers based on the ROS system. The calibration system includes a ROS startup module, a data acquisition module, a marker recognition module, a feature point matching module, and a data optimization module.

[0076] The ROS startup system is used to start the camera and lidar. The data acquisition system records the encapsulation module of the topic communication corresponding to the camera and lidar calibration, the camera intrinsic calibration data, the size of the calibration plate and the coordinates of all markers in the calibration plate; the data visualization module synchronously displays the calibration data acquisition progress, and projects the lidar points onto the image plane to observe the calibration results; the marker recognition system identifies the lidar marker and the image ArUco marker, and matches the lidar marker and the image ArUco marker; the optimization solution module calculates the external parameters and outputs the calibration results.

[0077] The specific contents of steps four and five include: holding the calibration plate facing the camera, the data visualization module synchronously displays the progress of the acquisition and matching of the lidar marker and the image ArUco marker, slowly moving the calibration plate, when the lidar marker and the image ArUco marker can be recognized at the same time, it is valid data, and the number of valid data is recorded, so that the calibration program can obtain as diverse three-dimensional space sample point data as possible, until the number of data collected by the marker recognition module reaches the threshold, the data acquisition process is terminated, and the external parameters are optimized and solved.

[0078] The specific steps for collecting ArUco markers on an image include:

[0079] Step 1.1: Set the time window T, collect image data I(t, t+T) within the time window, establish coordinate systems for the first image ArUco marker and the second image ArUco marker respectively, identify the image data and output a data group, which includes the center coordinates C of the aruco marker of the first image within the time window T u (t, t+T), the first image aruco marked unit plane normal vector N u (t, t+T), the center coordinate C of the aruco mark in the second image d (t, t+T), the unit plane normal vector N of the aruco marker of the second image d (t, t+T);

[0080] Step 1.2: If the image data does not recognize the first image aruco marker and the second image aruco marker, set the center coordinates of the first image aruco marker, the unit plane normal vector of the first image aruco marker, the center coordinates of the second image aruco marker, and the unit plane normal vector of the second image aruco marker to preset values. When the data continuous acquisition threshold J and the data cumulative acquisition threshold M are set, if the recognition results of J consecutive image data or M cumulative image data within the time window T are the preset values, mark the data segment as invalid data, clear the invalid data and output the updated data group;

[0081] Step 1.3: Calculate the mean variance D of the aruco label of the first image and the aruco label of the second image within the time window. The specific calculation formula is:

[0082] D={std[C′ u (t, t+T)]+std[C′ d (t, t+T)]} / 2

[0083] Among them, C′ u (t, t+T) represents the updated coordinates of the center of the aruco marker of the first image at the time (t, t+T), C′ d(t, t+T) represents the updated coordinates of the center of the aruco marker in the second image at the time (t, t+T);

[0084] Calculate the mean S of the normal vector cosine values of the center coordinates of the ArUco marker of the first image and the ArUco marker of the second image within the time window. The specific calculation formula is:

[0085]

[0086] in, represents the updated aruco unit plane normal vector of the first image at time (t, t+T), represents the updated aruco unit plane normal vector of the second image at time (t, t+T);

[0087] Step 1.4: Set the variance mean threshold and the normal vector cosine mean threshold. When the variance mean D is less than the variance mean threshold and the normal vector cosine mean S is less than the normal vector cosine mean threshold, mark the updated data set as valid data and output the image recognition result C′ u (t, t+T) and C′ d The mean of (t, t+T) and N′ u (t, t+T) and N′ d The mean of (t, t+T).

[0088] The specific contents of laser radar marker collection include: performing laser radar point threshold filtering based on the reflection intensity and distance range of the laser radar point, projecting the filtered laser radar points line by line onto a virtual image plane, clustering them on the image plane using the density-based clustering method DBSCAN, identifying the start and end symbols of the radar marker, and identifying the laser radar marker through prior information; storing the recognition results as calibration data, and performing a linear regression with a step size of t. s The above steps are repeated in the sliding time window until the number of radar calibration data reaches the radar marking threshold and the data collection phase is terminated;

[0089] The specific steps for joint calibration of LiDAR and camera include:

[0090] Step 2.1: Establish the coordinate system of the laser radar to output the coordinates of the marker point (x l ,y l , z l ), establish the camera coordinate system;

[0091] Step 2.2: The camera obtains the transformation relationship [R|T] between the camera marker coordinate system and the camera coordinate system through the prior camera marker size information. m2c ;

[0092] Step 2.3: Output the coordinates of the marker point in the camera coordinate system. The conversion formula is:

[0093]

[0094] Among them, (x lm ,y lm , z lm ) represents the coordinates of the marker point in the camera marker coordinate system, (x lc ,y lc , z lc ) represents the coordinates of the marker point in the camera coordinate system;

[0095] Step 2.4: Output the conversion model between the camera coordinate system and the lidar coordinate system, and optimize the conversion model;

[0096] The specific calculation formula of the conversion model between the camera coordinate system and the lidar coordinate system is:

[0097]

[0098] in, Represents the coordinates of the marker point projected into the camera coordinate system, [R|T] is the external parameter;

[0099] Minimizing the reprojection error optimizes the transformation model to obtain the rotation and translation relationship between the laser radar and camera coordinates. The specific formula of the objective function of the optimization model is:

[0100] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0101] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A calibration method for joint calibration of a laser radar and a camera, characterized by: The joint calibration method specifically includes: Step 1: Complete the camera intrinsic calibration, enter the intrinsic calibration data, the size of the calibration plate and the coordinates of all markers in the calibration plate into the calibration system, and establish an automatic recognition system for lidar markers and image ArUco markers based on the ROS system. The calibration system includes a ROS startup module, a data visualization module, a data acquisition module, a marker recognition module and an optimization solution module; Step 2: The ROS startup module starts the camera and lidar; Step 3: The marker recognition module recognizes and matches the lidar marker and the image ArUco marker, and the data visualization module simultaneously displays the progress of feature point data collection of the marker recognition module; Step 4: Hold the calibration plate facing the camera and slowly move it within the camera's field of view until the number of data collected by the marker recognition module reaches the threshold; Step 4 includes the image ArUco marker acquisition step: Step 1.1: Set a time window T, collect image data I(t, t+T) within the time window, establish coordinate systems for the first image ArUco mark and the second image ArUco mark respectively, identify the image data and output a data group, the data group includes the center coordinates C of the first image ArUco mark within the time window T u (t, t+T), the first image aruco marked unit plane normal vector N u (t, t+t), the center coordinate C of the aruco mark in the second image d (t, t+T), the unit plane normal vector N of the aruco marker of the second image d (t, t+T); Step 1.2: If the image data does not recognize the first image aruco marker and the second image aruco marker, set the center coordinates of the first image aruco marker, the unit plane normal vector of the first image aruco marker, the center coordinates of the second image aruco marker, and the unit plane normal vector of the second image aruco marker to preset values. When the data continuous acquisition threshold J and the data cumulative acquisition threshold M are set, and the recognition results of J consecutive image data or M cumulative image data within the time window T are preset values, mark the J consecutive image data or M cumulative image data within the time window T as invalid data, clear the invalid data and output the updated data group; Step 1.3: Calculate the mean variance D of the aruco label of the first image and the aruco label of the second image within the time window. The specific calculation formula is: D={std[C′ u (t,t+T)]+std[C′ d (t,t+T)]} / 2, Among them, C u ′ (t, t+T) represents the updated coordinates of the center of the aruco marker in the first image at the time (t, t+T), C d ′ (t, t+T) represents the updated coordinates of the center of the aruco marker in the second image at the time (t, t+T); Calculate the mean S of the normal vector cosine values of the center coordinates of the ArUco marker of the first image and the ArUco marker of the second image within the time window. The specific calculation formula is: in, represents the updated aruco unit plane normal vector of the first image at time (t, t+T), represents the updated aruco unit plane normal vector of the second image at time (t, t+T); Step 1.4: Set the variance mean threshold and the normal vector cosine mean threshold. When the variance mean D is less than the variance mean threshold and the normal vector cosine mean S is less than the normal vector cosine mean threshold, mark the updated data set as valid data and output the image recognition result C. u ′ (t, t+T) and C d ′ The mean of (t, t+T) and N u ′ (t, t+T) and N d ′ The mean of (t, t+T); Step 5: After data collection is completed, the optimization solution module calculates external parameters and outputs calibration results; Step 6: The data visualization module observes the calibration results. If the lidar points in the projection results overlap well with the camera images, the calibration results are considered valid. Otherwise, check for problems in the process and re-calibrate the lidar and camera.

2. The calibration method for joint calibration of a laser radar and a camera according to claim 1, characterized in that: The specific contents of step four and step five include: holding the calibration plate facing the camera, the data visualization module synchronously displays the progress of laser radar marker and image ArUco marker acquisition and matching, slowly moving the calibration plate, and when the laser radar marker and image ArUco marker can be recognized at the same time, it is valid data, recording the number of valid data, and making the calibration program obtain as diverse three-dimensional space sample point data as possible, until the number of data collected by the marker recognition module reaches a threshold, ending the data acquisition process, and optimizing the solution of external parameters.

3. The calibration method for joint calibration of a laser radar and a camera according to claim 2, characterized in that: The specific contents of the laser radar marker collection include: performing laser radar point threshold filtering based on the reflection intensity and distance range of the laser radar point, projecting the filtered laser radar points line by line onto a virtual image plane, clustering them on the image plane using the density-based clustering method DBSCAN, identifying the start and end symbols of the radar marker, and identifying the laser radar marker through prior information; storing the identification results as calibration data, and performing a linear regression with a step size of t. s The above steps are repeated in the sliding time window until the number of radar calibration data reaches the radar marking threshold and the data collection phase is terminated.

4. The calibration method for joint calibration of a laser radar and a camera according to claim 3, characterized in that: The specific steps for joint calibration of LiDAR and camera include: Step 2.1: Establish the coordinate system of the laser radar to output the coordinates of the marker point (x l ,y l ,z l ), establish the camera coordinate system; Step 2.2: The camera obtains the transformation relationship [R|T] between the camera marker coordinate system and the camera coordinate system through the prior camera marker size information. m2c ; Step 2.3: Output the coordinates of the marker point in the camera coordinate system. The conversion formula is: Among them, (x lm ,y lm ,z lm ) represents the coordinates of the marker point in the camera marker coordinate system, (x lc ,y lc ,z lc ) represents the coordinates of the marker point in the camera coordinate system; Step 2.4: Output the conversion model between the camera coordinate system and the lidar coordinate system, and optimize the conversion model; The specific calculation formula of the conversion model between the camera coordinate system and the lidar coordinate system is: in, Represents the coordinates of the marker point projected into the camera coordinate system, [R|T] is the external parameter; Minimizing the reprojection error optimizes the transformation model to obtain the rotation and translation relationship between the laser radar and camera coordinates. The specific formula of the objective function of the optimization model is:

5. A calibration plate for joint calibration of a laser radar and a camera, applied to the calibration method for joint calibration of a laser radar and a camera according to any one of claims 1 to 4, characterized in that: The calibration plate includes a laser radar marker and an image ArUco marker, wherein the laser radar marker is used for automatic recognition of the laser radar marker, and the image ArUco marker is used for automatic recognition of the image marker.

6. The calibration plate for combined calibration of a laser radar and a camera according to claim 5, characterized in that: The laser radar marker includes a start and end mark and several marking points. The start and end marks include a start mark and an end mark. The start mark includes a black rectangle with a width of two units and a black rectangle with a width of one unit. The start mark and the end mark are horizontally mirrored about the center point of the several marking points. The marking points are black images with a width of one unit. The interval between the marking points is a white blank of one unit. The interval between the marking points and the start and end marks is a white blank of one unit.

7. The calibration plate for joint calibration of a laser radar and a camera according to claim 6, characterized in that: The image ArUco marker includes a first ArUco marker and a second ArUco marker, the first ArUco marker and the second ArUco marker have equal areas and different IDs, the first ArUco marker and the second ArUco marker include a black border and a binary matrix, and the binary matrix expresses the ArUco marker ID.

8. The calibration plate for joint calibration of a laser radar and a camera according to claim 7, characterized in that: The first ArUco marker and the second ArUco marker are parallel and equidistant on both sides of the laser radar marker, and the centers of the first ArUco marker and the second ArUco marker are coaxial with the center of the laser radar middle marker in the vertical direction.

9. The calibration plate for combined calibration of a laser radar and a camera according to claim 8, characterized in that: The marker coordinates of all LiDAR markers and image ArUco markers in the calibration plate are determined and known.

Images