Laser radar positioning calibration system based on visible light fusion analysis

Abstract

The invention relates to the technical field of automatic driving, is used for solving the problems that a traditional calibration method is low in precision and poor in robustness, and particularly relates to a laser radar positioning calibration system based on visible light fusion analysis. A plane target calibration method is adopted, a checkerboard calibration plate is taken as a medium, the pose of the calibration plate is adjusted for multiple times, normal vectors and origin distance data of the calibration plate under two sensor coordinate systems are synchronously collected, and a solution model of a rotation matrix R and a translation vector T is constructed based on a coordinate system conversion relation. A normal vector matrix orthogonal constraint is constructed by using sufficient feature points of a plane target fixed plate, the sum of normal vector included angle cosine under two coordinate systems is maximized, an analytic solution of a rotation matrix R is obtained through SVD decomposition, and meanwhile, a distance difference target function from a laser radar and camera coordinate system to a calibration plane is established. The optimal translation vector T is solved through the least square method, mixed pixel and noise interference is eliminated, the precision degree in the joint calibration process is improved, and high-precision external parameter calibration is achieved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a lidar positioning and calibration system based on visible light fusion analysis. Background Art

[0002] LiDAR can quickly and accurately obtain distance information in the environment and has high robustness to weak texture areas. Cameras can obtain dense and rich texture information, but cannot obtain depth. The two types of data are highly complementary. Fusion of data from the two modalities can provide a more comprehensive perception of the surrounding environment. In recent years, with the reduction in the cost of LiDAR equipment, the technical threshold for obtaining three-dimensional point clouds has been reduced year by year. Equipment that integrates visible light images and radar depth data has been widely used in related fields such as autonomous driving, industrial robot navigation, enterprise digital twins, and aerial photography.

[0003] A key core technical issue in the lidar-camera joint calibration method based on matrix decomposition is the high-precision joint calibration between the two devices.

[0004] Currently, most calibration methods in traditional technologies are based on a single target. However, during the single-target calibration process, the cross-modal data feature extraction and matching errors are large, and the accuracy of the calibration process cannot be guaranteed. The robustness is poor. For example, in the existing patent CN2020110028278, a single cylindrical target is used for calibration. This method may cause inaccurate calibration results due to the existence of systematic errors during calibration, and is also greatly affected by the target.

[0005] In response to the above technical problems, this application proposes a solution. Summary of the Invention

[0006] The present invention adopts a planar target calibration method, uses a checkerboard calibration plate as a medium, constructs constraint equations through multiple sets of pose transformations, combines singular value decomposition (SVD) with least squares optimization, achieves high-precision extrinsic parameter calibration, and solves the problems of low precision and poor robustness of traditional calibration methods. A lidar positioning and calibration system based on visible light fusion analysis is proposed.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A lidar positioning and calibration system based on visible light fusion analysis includes a lidar acquisition module, a visible light data acquisition module, a parameter decomposition module, an overlap calibration module, and a fusion positioning module. The lidar acquisition module can collect the lidar reflection of the checkerboard calibration plate and record it as a lidar image.

[0009] The visible light data acquisition module can capture the checkerboard calibration plate through a visible light camera, record the captured images, and perform preliminary processing on the recorded images to obtain visible light images;

[0010] The overlapping calibration module can obtain the visible light image and the laser radar image, and simultaneously create spatial coordinate systems for the laser radar image and the visible light image respectively, to obtain the normal vector data and origin distance data of the checkerboard calibration plate;

[0011] The parameter decomposition module is used to perform a comprehensive comparison of the normal vector data and the origin distance data in the visible light coordinate system and the lidar coordinate system, and solve the analytical solution of the rotation matrix R and the optimal translation vector T through a mathematical modeling method;

[0012] The fusion positioning module records the analytical solutions of the rotation matrix R and the optimal translation vector T obtained multiple times, and synthesizes them to obtain the fusion calibration results of the visible light camera and the lidar.

[0013] As a preferred embodiment of the present invention, the laser radar image captured by the laser radar acquisition module is a three-dimensional depth-of-field image, and the method for the laser radar acquisition module to obtain the three-dimensional depth-of-field image is:

[0014] The laser radar uses reflection imaging to confirm the position of the concave points in the checkerboard calibration plate, and confirms the three-dimensional position of the checkerboard calibration plate in the air according to the set weight ratio of the concave point position and the reflection of the checkerboard calibration plate itself.

[0015] As a preferred embodiment of the present invention, the method for the laser radar acquisition module to obtain the preset weight ratio is:

[0016] If the material of the checkerboard calibration plate is a diffuse reflection material, the weights of the two are similar. If the material of the checkerboard calibration plate is not a diffuse reflection material, or the diffuse reflection effect is insufficient, resulting in overexposure of the point cloud, the weight of the concave point position is increased.

[0017] As a preferred embodiment of the present invention, the visible light data acquisition module performs preliminary processing on the visible light image, including the following steps:

[0018] S1: Extract the brightness of the visible light image and quantify the brightness in a numerical manner to obtain the point with the highest brightness and the point with the lowest brightness in the visible light image;

[0019] S2: The visible light data acquisition module divides the visible light image into n regions, each region is numbered i, i=1, 2, 3, ..., n, and samples the brightness of each region to obtain the regional brightness Li. The visible light data acquisition module performs arithmetic averaging on the brightness of all regions to obtain the overall brightness QL of the image;

[0020] S3: The visible light data acquisition module comprehensively evaluates the overall brightness, maximum brightness and minimum brightness of the picture, generates brightness adjustment parameters, and adjusts the camera imaging parameters according to the brightness adjustment parameters to obtain new imaging parameters and obtain the final confirmed visible light picture.

[0021] As a preferred embodiment of the present invention, the overlapping calibration module identifies a checkerboard calibration plate in a visible light image, selects the intersection points of the black and white squares on the checkerboard calibration plate, and records them as saddle points. The overlapping calibration module records all saddle points to obtain an actual saddle point distribution. The overlapping calibration module obtains setting information of the checkerboard calibration plate from a preset database, wherein the setting information of the checkerboard calibration plate includes the external dimensions of the checkerboard calibration plate, the dimensions of the black and white checkerboard, and the preset saddle point positions. The actual saddle point distribution obtained by identification is overlapped with the preset saddle point positions, and the spatial distribution of the checkerboard calibration plate is restored through the overlap result.

[0022] After obtaining the spatial distribution of the checkerboard calibration plate, the overlapping calibration module creates a coordinate system according to the preset camera origin and expresses the spatial distribution of the checkerboard calibration plate in the form of coordinates, thereby obtaining normal vector data and origin distance data in the visible light coordinate system.

[0023] As a preferred embodiment of the present invention, after the overlapping calibration module obtains the lidar image, it creates a coordinate system according to the set lidar origin, expresses the checkerboard calibration plate in the lidar image in the form of a coordinate matrix, and obtains the normal vector data and origin distance data in the lidar coordinate system.

[0024] As a preferred embodiment of the present invention, the parameter decomposition module obtains normal vector data and origin distance data in a visible light coordinate system and normal vector data and origin distance data in a lidar coordinate system;

[0025] The parameter decomposition module constructs the orthogonal constraint of the normal vector matrix, and obtains the analytical solution of the rotation matrix R by singular value decomposition by maximizing the sum of the cosines of the angles between the normal vectors in the two coordinate systems;

[0026] The parameter decomposition module establishes a distance difference objective function between the laser radar coordinate system and the visible light coordinate system and the plane where the checkerboard calibration plate is located, and solves the optimal translation vector T by the least squares method;

[0027] The parameter decomposition module sends the analytical solution of the optimal translation vector T and rotation matrix R to the fusion positioning module.

[0028] As a preferred embodiment of the present invention, the fusion positioning module obtains the optimal translation vector T multiple times. Xand the rotation matrix R X Then, the optimal translation vector T is obtained through the constraint equation X and the rotation matrix R X Perform final value calibration, unify the visible light coordinate system and the lidar coordinate system, and complete the calibration process, where X is the number of times the optimal translation vector T and rotation matrix R are obtained.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. In the present invention, the calibration plate posture is adjusted multiple times, and its normal vector and origin distance data in the two sensor coordinate systems are synchronously collected. Then, based on the coordinate system conversion relationship, a solution model of the rotation matrix R and the translation vector T is constructed, thereby realizing the construction of the normal vector matrix orthogonal constraint using sufficient feature points of the planar target calibration plate. By maximizing the sum of the cosines of the normal vector angles in the two coordinate systems, the analytical solution of the rotation matrix R is obtained by SVD decomposition. At the same time, an objective function of the distance difference from the laser radar and camera coordinate systems to the calibration plane is established. The optimal translation vector T is solved by the least squares method, and the interference of mixed pixels and noise is eliminated, thereby improving the accuracy of the joint calibration process.

[0031] 2. In the present invention, the dual data acquisition design of the checkerboard calibration plate enables the system to complete coordinate system alignment without the need for additional high-precision calibration tools. The overlapping calibration module automatically creates a spatial coordinate system, replacing the tedious process of selecting manual calibration points. The analytical solution generation algorithm for the rotation matrix R and the translation vector T transforms the traditional calibration steps that require the participation of professional technicians into a standardized process, thereby improving the overall automation level of the system.

[0032] 3. In the present invention, the fusion positioning module comprehensively processes multiple calibration results, giving the system dynamic optimization capabilities. By continuously comparing visible light and lidar data and using mathematical modeling to analyze heterogeneous coordinate system data, it can automatically compensate for parameter drift caused by equipment aging or environmental changes, and reduce the cumulative error caused by physical deviation of the sensor in traditional calibration methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0034] Figure 1 is a system block diagram of the present invention;

[0035] Figure 2 It is a system flow chart of the present invention. DETAILED DESCRIPTION

[0036] The following is a clear and complete description of the technical solutions of the present invention in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0037] Example 1:

[0038] See also Figure 1 - Figure 2 As shown in FIG, the laser radar positioning and calibration system based on visible light fusion analysis includes a laser radar acquisition module, a visible light data acquisition module, a parameter decomposition module, an overlap calibration module, and a fusion positioning module. The laser radar acquisition module can collect the reflection of the laser radar, obtain the laser radar parameters, and perform three-dimensional modeling and analysis on the laser radar parameters to generate a laser radar image. The laser radar image is a three-dimensional depth of field image. Specifically, the method for the laser radar to confirm the three-dimensional depth of field image is as follows:

[0039] A checkerboard calibration plate with concave points is selected and placed in a position that can be captured by both the lidar and the visible light camera. The lidar uses reflection imaging to confirm the location of the concave points on the checkerboard calibration plate. The three-dimensional position of the checkerboard calibration plate in the air is determined by the location of the concave points and the reflection of the checkerboard calibration plate according to the set weight ratio.

[0040] The weight ratio of the concave point position and the reflection of the checkerboard calibration plate itself is determined by the material of the checkerboard calibration plate. If the material of the checkerboard calibration plate is diffuse reflection, the weights of the two are similar, for example, the weight of the concave point position is 0.6, and the weight of the checkerboard calibration plate itself is 0.4. If the material of the checkerboard calibration plate is not diffuse reflection, or the diffuse reflection effect is insufficient, resulting in overexposure of the point cloud, the weight of the concave point position is increased, for example, the weight of the concave point position is 0.8, and the weight of the checkerboard calibration plate is 0.2.

[0041] The visible light data acquisition module can analyze the imaging parameters of the visible light camera to obtain a visible light image. After obtaining the visible light image, the visible light data acquisition module extracts the brightness of the visible light image and quantifies the brightness in a numerical manner, thereby obtaining the point with the highest brightness and the point with the lowest brightness in the visible light image.

[0042] The visible light data acquisition module divides the visible light image into n areas, each area is numbered i, i = 1, 2, 3, ..., n, and samples the brightness of each area to obtain the regional brightness Li. The visible light data acquisition module performs arithmetic averaging on the brightness of all areas to obtain the overall brightness QL of the image. The visible light data acquisition module comprehensively evaluates the overall brightness, maximum brightness, and minimum brightness of the image, generates brightness adjustment parameters, and adjusts the camera imaging parameters by the brightness adjustment parameters to obtain new imaging parameters. The visible light image is regenerated by the new imaging parameters to obtain the final confirmed visible light image;

[0043] The overlap calibration module obtains the final visible light image through the visible light data acquisition module and obtains the lidar image through the lidar acquisition module;

[0044] After acquiring the visible light image, the overlapping calibration module uses an algorithm to identify the checkerboard calibration plate in the visible light image and selects the intersection points of the black and white squares on the checkerboard calibration plate and records them as saddle points. The overlapping calibration module records all saddle points to obtain the actual saddle point distribution. The overlapping calibration module obtains the setting information of the checkerboard calibration plate from a preset database, where the setting information of the checkerboard calibration plate includes the external dimensions of the checkerboard calibration plate, the dimensions of the black and white checkerboard, and the preset saddle point positions. The actual saddle point distribution obtained by identification is overlapped with the preset saddle point positions, and the spatial distribution of the checkerboard calibration plate is restored through the overlap result.

[0045] After obtaining the spatial distribution of the checkerboard calibration plate, the overlapping calibration module creates a coordinate system according to the preset camera origin and expresses the spatial distribution of the checkerboard calibration plate in the form of coordinates, thereby obtaining the normal vector data and origin distance data in the visible light coordinate system;

[0046] After the overlapping calibration module obtains the lidar image, it creates a coordinate system according to the set lidar origin, expresses the checkerboard calibration plate in the lidar image in the form of a coordinate matrix, and obtains the normal vector data and origin distance data in the lidar coordinate system;

[0047] The normal vector data is the normal vector corresponding to the plane of the checkerboard calibration plate, and the origin distance data is the shortest distance between the plane where the checkerboard calibration plate is located and the origin;

[0048] The parameter decomposition module obtains the normal vector data and origin distance data in the visible light coordinate system and the normal vector data and origin distance data in the lidar coordinate system;

[0049] The parameter decomposition module constructs the orthogonal constraint of the normal vector matrix and obtains the analytical solution of the rotation matrix R by SVD decomposition (singular value decomposition) by maximizing the sum of the cosines of the angles between the normal vectors in the two coordinate systems;

[0050] The parameter decomposition module establishes the distance difference objective function between the lidar coordinate system and the visible light coordinate system and the plane where the checkerboard calibration plate is located, and solves the optimal translation vector T by the least squares method;

[0051] The parameter decomposition module sends the analytical solution of the optimal translation vector T and the rotation matrix R to the fusion positioning module; the fusion positioning module unifies the visible light coordinate system and the lidar coordinate system through the analytical solution of the optimal translation vector T and the rotation matrix R, thereby completely calibrating the lidar and visible light camera.

[0052] Example 2:

[0053] See also Figure 1 - Figure 2 As shown, the spatial posture of the checkerboard calibration plate is adjusted multiple times, such as position adjustment, tilt angle adjustment, and distance adjustment. After each adjustment, a calibration work is performed using the laser radar positioning calibration system. The calibration work includes the following steps:

[0054] Step 1: Collect lidar images and visible light images;

[0055] Step 2: Create a lidar coordinate system based on the collected lidar image, and create a visible light coordinate system based on the collected visible light image;

[0056] Step 3: Extract the normal vector data and origin distance data in the lidar coordinate system and the visible light coordinate system;

[0057] Step 4: Calculate the optimal translation vector T again based on the normal vector data and origin distance data in the two coordinate systems X and the rotation matrix R X The collection of, where X is the number of times the calibration operation is performed;

[0058] Step 5: Based on the optimal translation vector T obtained multiple times X and the rotation matrix R X Construct constraint equations to complete the calibration process.

[0059] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. The laser radar positioning and calibration system based on visible light fusion analysis is characterized by: It includes a laser radar acquisition module, a visible light data acquisition module, a parameter decomposition module, an overlapping calibration module and a fusion positioning module. The laser radar acquisition module can collect the laser radar reflection of the checkerboard calibration plate and record it as a laser radar image; The visible light data acquisition module can capture the checkerboard calibration plate through a visible light camera, record the captured images, and perform preliminary processing on the recorded images to obtain visible light images; The overlapping calibration module can obtain the visible light image and the laser radar image, and simultaneously create spatial coordinate systems for the laser radar image and the visible light image respectively, to obtain the normal vector data and origin distance data of the checkerboard calibration plate; The parameter decomposition module is used to perform a comprehensive comparison of the normal vector data and the origin distance data in the visible light coordinate system and the lidar coordinate system, and solve the analytical solution of the rotation matrix R and the optimal translation vector T through a mathematical modeling method; The fusion positioning module records the analytical solutions of the rotation matrix R and the optimal translation vector T obtained multiple times, and synthesizes them to obtain the fusion calibration results of the visible light camera and the lidar.

2. The laser radar positioning and calibration system based on visible light fusion analysis according to claim 1 is characterized in that: The laser radar image captured by the laser radar acquisition module is a three-dimensional depth-of-field image. The method for the laser radar acquisition module to obtain the three-dimensional depth-of-field image is: The laser radar uses reflection imaging to confirm the position of the concave points in the checkerboard calibration plate, and confirms the three-dimensional position of the checkerboard calibration plate in the air according to the set weight ratio of the concave point position and the reflection of the checkerboard calibration plate itself.

3. The laser radar positioning and calibration system based on visible light fusion analysis according to claim 2 is characterized in that: The method for the laser radar acquisition module to obtain the preset weight ratio is: If the material of the checkerboard calibration plate is a diffuse reflection material, the weights of the two are similar. If the material of the checkerboard calibration plate is not a diffuse reflection material, or the diffuse reflection effect is insufficient, resulting in overexposure of the point cloud, the weight of the concave point position is increased.

4. The laser radar positioning and calibration system based on visible light fusion analysis according to claim 1, characterized in that: The visible light data acquisition module performs preliminary processing on the visible light image, including the following steps: S1: Extract the brightness of the visible light image and quantify the brightness in a numerical manner to obtain the point with the highest brightness and the point with the lowest brightness in the visible light image; S2: The visible light data acquisition module divides the visible light image into n regions, each region is numbered i, i=1, 2, 3, ..., n, and samples the brightness of each region to obtain the regional brightness Li. The visible light data acquisition module performs arithmetic averaging on the brightness of all regions to obtain the overall brightness QL of the image; S3: The visible light data acquisition module comprehensively evaluates the overall brightness, maximum brightness and minimum brightness of the picture, generates brightness adjustment parameters, and adjusts the camera imaging parameters according to the brightness adjustment parameters to obtain new imaging parameters and obtain the final confirmed visible light picture.

5. The laser radar positioning and calibration system based on visible light fusion analysis according to claim 1 is characterized in that: The overlapping calibration module identifies the checkerboard calibration plate in the visible light image and selects the intersection points of the black and white checkers on the checkerboard calibration plate and records them as saddle points. The overlapping calibration module records all saddle points to obtain the actual saddle point distribution. The overlapping calibration module obtains setting information of the checkerboard calibration plate from a preset database, wherein the setting information of the checkerboard calibration plate includes the external dimensions of the checkerboard calibration plate, the dimensions of the black and white checkerboard, and the preset saddle point positions. The actual saddle point distribution obtained by identification is overlapped with the preset saddle point positions, and the spatial distribution of the checkerboard calibration plate is restored through the overlap result. After obtaining the spatial distribution of the checkerboard calibration plate, the overlapping calibration module creates a coordinate system according to the preset camera origin and expresses the spatial distribution of the checkerboard calibration plate in the form of coordinates, thereby obtaining normal vector data and origin distance data in the visible light coordinate system.

6. The laser radar positioning and calibration system based on visible light fusion analysis according to claim 1 is characterized in that: After the overlapping calibration module obtains the laser radar image, it creates a coordinate system according to the set laser radar origin, expresses the checkerboard calibration plate in the laser radar image in the form of a coordinate matrix, and obtains the normal vector data and origin distance data in the laser radar coordinate system.

7. The laser radar positioning and calibration system based on visible light fusion analysis according to claim 1, characterized in that: The parameter decomposition module obtains normal vector data and origin distance data in a visible light coordinate system and normal vector data and origin distance data in a lidar coordinate system; The parameter decomposition module constructs the orthogonal constraint of the normal vector matrix, and obtains the analytical solution of the rotation matrix R by singular value decomposition by maximizing the sum of the cosines of the angles between the normal vectors in the two coordinate systems; The parameter decomposition module establishes a distance difference objective function between the laser radar coordinate system and the visible light coordinate system and the plane where the checkerboard calibration plate is located, and solves the optimal translation vector T by the least squares method; The parameter decomposition module sends the analytical solution of the optimal translation vector T and rotation matrix R to the fusion positioning module.

8. The laser radar positioning and calibration system based on visible light fusion analysis according to claim 1, characterized in that: The fusion positioning module obtains the optimal translation vector T multiple times X and the rotation matrix R X Then, the optimal translation vector T is obtained through the constraint equation X and the rotation matrix R X Perform final value calibration, unify the visible light coordinate system and the lidar coordinate system, and complete the calibration process, where X is the number of times the optimal translation vector T and rotation matrix R are obtained.

Images