LiDAR Positioning and Calibration System Based on Visible Light Fusion Analysis

Abstract

This invention relates to the field of autonomous driving technology and addresses the problems of low accuracy and poor robustness in traditional calibration methods. Specifically, it relates to a lidar positioning and calibration system based on visible light fusion analysis. The invention employs a planar target calibration method, using a checkerboard calibration board as a medium. The pose of the calibration board is adjusted multiple times, and its normal vector and distance data from the origin in both sensor coordinate systems are collected simultaneously. Based on coordinate system transformation relationships, a solution model for the rotation matrix R and translation vector T is constructed. This utilizes sufficient feature points of the planar target calibration board to construct orthogonal constraints on the normal vector matrix and maximizes the sum of the cosines of the angles between the normal vectors in the two coordinate systems. An analytical solution for the rotation matrix R is obtained through SVD decomposition. Simultaneously, an objective function for the distance difference between the lidar and camera coordinate systems and the calibration plane is established. The optimal translation vector T is solved using the least squares method, eliminating mixed pixels and noise interference, improving the accuracy of the joint calibration process, and achieving high-precision extrinsic parameter calibration.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically to a lidar positioning and calibration system based on visible light fusion analysis. Background Technology

[0002] LiDAR can quickly and accurately acquire distance information in the environment and has high robustness to areas with weak texture. Cameras can obtain dense and rich texture information, but cannot acquire depth. The two types of data have good complementarity. By fusing the data of the two modalities, a more comprehensive perception of the surrounding environment can be achieved. In recent years, with the reduction in the cost of LiDAR equipment, the technical threshold for acquiring 3D point clouds has been decreasing year by year. Equipment that integrates visible light images and radar depth data has been widely used in fields such as autonomous driving, industrial robot navigation, enterprise digital twins, and aerial surveying.

[0003] A key technical problem in the joint calibration method of lidar-camera based on matrix factorization is the high-precision joint calibration between the two devices.

[0004] Currently, most traditional calibration methods are based on single-target calibration. However, single-target calibration has a large cross-modal data feature extraction and matching error, which cannot guarantee the accuracy of the calibration process and has poor robustness. For example, the existing patent CN2020110028278 uses a single cylindrical target for calibration. This method may result in inaccurate calibration results due to the existence of systematic errors, and is also greatly affected by the target.

[0005] To address the aforementioned technical problems, this application proposes a solution. Summary of the Invention

[0006] This invention employs a planar target calibration method, using a checkerboard calibration board as a medium. By constructing constraint equations through multiple pose transformations and combining singular value decomposition (SVD) and least squares optimization, high-precision extrinsic parameter calibration is achieved, solving the problems of low accuracy and poor robustness of traditional calibration methods. A lidar positioning and calibration system based on visible light fusion analysis is proposed.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] The lidar positioning and calibration system based on visible light fusion analysis includes a lidar acquisition module, a visible light data acquisition module, a parameter decomposition module, an overlap calibration module, and a fusion positioning module. The lidar acquisition module can acquire the reflection of the lidar on the checkerboard calibration board and record it as a lidar image.

[0009] The visible light data acquisition module can capture the chessboard calibration board through a visible light camera, record the captured images, and perform preliminary processing on the recorded images to obtain visible light images.

[0010] The overlapping calibration module can acquire visible light images and lidar images, and simultaneously create spatial coordinate systems for the lidar images and visible light images respectively to obtain the normal vector data and origin distance data of the checkerboard calibration board.

[0011] The parameter decomposition module is used to comprehensively compare the normal vector data and origin distance data in the visible light coordinate system and the lidar coordinate system, and solve the analytical solution of the rotation matrix R and the optimal translation vector T through mathematical modeling methods.

[0012] The fusion positioning module records the analytical solutions of the rotation matrix R obtained multiple times and the optimal translation vector T, and integrates them to obtain the fusion calibration results of the visible light camera and the lidar.

[0013] In a preferred embodiment of the present invention, the lidar image acquired by the lidar acquisition module is a three-dimensional depth image, and the method by which the lidar acquisition module acquires the three-dimensional depth image is as follows:

[0014] The lidar uses reflective imaging to determine the position of the concave points in the checkerboard calibration plate, and determines the three-dimensional position of the checkerboard calibration plate in the air by using the position of the concave points and the reflection of the checkerboard calibration plate itself according to the set weight ratio.

[0015] In a preferred embodiment of the present invention, the method by which the lidar acquisition module obtains the preset weight ratio is as follows:

[0016] If the checkerboard calibration plate is made of a diffuse reflective material, the weights of the two are similar. If the checkerboard calibration plate is not made of a diffuse reflective material, or the diffuse reflective effect is insufficient, resulting in overexposure of the point cloud, the weight of the concave point position is increased.

[0017] In a preferred embodiment of the present invention, the visible light data acquisition module performs preliminary processing of the visible light image, including the following steps:

[0018] S1: Extract the brightness of the visible light image and quantify the brightness numerically to obtain the points with the highest and lowest brightness in the visible light image.

[0019] S2: The visible light data acquisition module divides the visible light image into n regions, each region is numbered i, i=1, 2, 3, ..., n, and samples the brightness of each region to obtain the region brightness Li. The visible light data acquisition module performs an arithmetic average of the brightness of all regions to obtain the overall brightness QL of the image.

[0020] S3: The visible light data acquisition module comprehensively evaluates the overall brightness, maximum brightness, and minimum brightness of the image, generates brightness adjustment parameters, and adjusts the camera imaging parameters through the brightness adjustment parameters to obtain new imaging parameters and finally obtain the confirmed visible light image.

[0021] In a preferred embodiment of the present invention, the overlapping calibration module identifies the checkerboard calibration board in the visible light image and selects the intersection points of the black and white squares on the checkerboard calibration board, recording them as saddle points. The overlapping calibration module records all saddle points to obtain the actual saddle point distribution. The overlapping calibration module obtains the setting information of the checkerboard calibration board through a preset database. The setting information of the checkerboard calibration board includes the external dimensions of the checkerboard calibration board, the dimensions of the black and white checkerboard, and the preset saddle point positions. The actual saddle point distribution and the preset saddle point positions are overlapped according to the identification, and the spatial distribution of the checkerboard calibration board is restored through the overlap result.

[0022] After acquiring the spatial distribution of the checkerboard calibration board, the overlapping calibration module creates a coordinate system based on the preset camera origin and expresses the spatial distribution of the checkerboard calibration board in coordinate form, thereby acquiring the normal vector data and origin distance data in the visible light coordinate system.

[0023] In a preferred embodiment of the present invention, after the overlapping calibration module acquires the lidar image, it creates a coordinate system according to the set lidar origin, expresses the checkerboard calibration board in the lidar image in the form of a coordinate matrix, and obtains the normal vector data and origin distance data in the lidar coordinate system.

[0024] In a preferred embodiment of the present invention, the parameter decomposition module acquires normal vector data and origin distance data in the visible light coordinate system, as well as normal vector data and origin distance data in the lidar coordinate system.

[0025] The parameter decomposition module constructs orthogonal constraints on the normal vector matrix, and obtains the analytical solution of the rotation matrix R by maximizing the sum of the cosines of the angle between the normal vectors in the two coordinate systems through singular value decomposition.

[0026] The parameter decomposition module establishes the objective function of the distance difference between the lidar coordinate system and the visible light coordinate system to the plane where the checkerboard calibration board is located, and solves the optimal translation vector T by the least squares method;

[0027] The parameter decomposition module sends the analytical solutions of the optimal translation vector T and rotation matrix R to the fusion positioning module.

[0028] In a preferred embodiment of the present invention, the fusion positioning module obtains the optimal translation vector T multiple times. Xand rotation matrix R X Then, the optimal translation vector T is determined by the constraint equations. X and rotation matrix R X The final value calibration is performed, and the visible light coordinate system and the lidar coordinate system are unified to complete the calibration process, where X is the number of times the optimal translation vector T and rotation matrix R are obtained.

[0029] Compared with the prior art, the beneficial effects of the present invention are:

[0030] 1. In this invention, by repeatedly adjusting the pose of the calibration board, the normal vector and the distance data from the origin in the coordinate systems of the two sensors are collected simultaneously. Then, based on the coordinate system transformation relationship, a solution model for the rotation matrix R and the translation vector T is constructed. This realizes the construction of orthogonal constraints of the normal vector matrix using sufficient feature points of the planar target calibration board. By maximizing the sum of the cosines of the angle between the normal vectors in the two coordinate systems, the analytical solution of the rotation matrix R is obtained by SVD decomposition. At the same time, the objective function of the distance difference from the coordinate systems of the lidar and the camera to the calibration plane is established. The optimal translation vector T is solved by the least squares method, eliminating mixed pixels and noise interference, and improving the accuracy of the joint calibration process.

[0031] 2. In this invention, the dual data acquisition design of the checkerboard calibration board enables the system to complete coordinate system alignment without additional high-precision calibration tools. The overlapping calibration module automatically creates a spatial coordinate system, replacing the tedious process of manually selecting calibration points. The analytical solution generation algorithm for the rotation matrix R and translation vector T transforms the calibration steps that traditionally require the participation of professional technicians into a standardized and executable process, thereby improving the overall automation level of the system.

[0032] 3. In this invention, the integrated processing of multiple calibration results by the fusion positioning module enables the system to have dynamic optimization capabilities. By continuously comparing visible light and lidar data and using mathematical modeling to analyze heterogeneous coordinate system data, it can automatically compensate for parameter drift caused by equipment aging or environmental changes, and reduce the cumulative error caused by sensor physical deviations in traditional calibration methods. Attached Figure Description

[0033] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0034] Figure 1 This is a system block diagram of the present invention;

[0035] Figure 2 This is a system flowchart of the present invention. Detailed Implementation

[0036] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0037] Example 1:

[0038] Please see Figure 1 - Figure 2 As shown, the lidar positioning and calibration system based on visible light fusion analysis includes a lidar acquisition module, a visible light data acquisition module, a parameter decomposition module, an overlap calibration module, and a fusion positioning module. The lidar acquisition module can collect the reflection data of the lidar, obtain lidar parameters, and perform three-dimensional modeling and analysis on the lidar parameters to generate a lidar image. The lidar image is a three-dimensional depth image. Specifically, the method for lidar to confirm the three-dimensional depth image is as follows:

[0039] A checkerboard calibration plate with concave dots is selected and placed in a position that can be captured by both the lidar and the visible light camera. The lidar uses reflection imaging to confirm the position of the concave dots on the checkerboard calibration plate, and confirms the three-dimensional position of the checkerboard calibration plate in the air by using the position of the concave dots and the reflection of the checkerboard calibration plate itself according to the set weight ratio.

[0040] The weighting of the concave point location and the reflection of the checkerboard calibration plate itself is determined by the material of the checkerboard calibration plate. If the checkerboard calibration plate is made of a diffuse reflective material, the weights of the two are similar, for example, the weight of the concave point location is 0.6 and the weight of the reflection of the checkerboard calibration plate itself is 0.4. If the checkerboard calibration plate is not made of a diffuse reflective material, or the diffuse reflective effect is insufficient, resulting in overexposure of the point cloud, the weight of the concave point location is increased, for example, the weight of the concave point location is 0.8 and the weight of the checkerboard calibration plate is 0.2.

[0041] The visible light data acquisition module can analyze the imaging parameters of the visible light camera to obtain a visible light image. After obtaining the visible light image, the visible light data acquisition module extracts the brightness of the visible light image and quantifies the brightness in a numerical way, thereby obtaining the points with the highest brightness and the points with the lowest brightness in the visible light image.

[0042] The visible light data acquisition module divides the visible light image into n regions, each numbered i, i=1, 2, 3, ..., n, and samples the brightness of each region to obtain the region's brightness Li. The visible light data acquisition module then performs an arithmetic mean of the brightness of all regions to obtain the overall image brightness QL. The visible light data acquisition module comprehensively evaluates the overall brightness, maximum brightness, and minimum brightness of the image, generates brightness adjustment parameters, and adjusts the camera imaging parameters using these parameters to obtain new imaging parameters. The visible light image is then regenerated using these new imaging parameters to obtain the final confirmed visible light image.

[0043] The overlap calibration module acquires the final visible light image through the visible light data acquisition module and the lidar image through the lidar acquisition module;

[0044] After acquiring the visible light image, the overlap calibration module identifies the checkerboard calibration board in the visible light image using an algorithm, selects the intersection points of the black and white squares on the checkerboard calibration board, and records them as saddle points. The overlap calibration module records all saddle points to obtain the actual saddle point distribution. The overlap calibration module obtains the setting information of the checkerboard calibration board through a preset database. The setting information of the checkerboard calibration board includes the external dimensions of the checkerboard calibration board, the dimensions of the black and white checkerboard, and the preset saddle point positions. The actual saddle point distribution and the preset saddle point positions are overlapped, and the spatial distribution of the checkerboard calibration board is restored through the overlap result.

[0045] After acquiring the spatial distribution of the checkerboard calibration board, the overlap calibration module creates a coordinate system based on the preset camera origin and expresses the spatial distribution of the checkerboard calibration board in the form of coordinates, thereby obtaining the normal vector data and origin distance data in the visible light coordinate system.

[0046] After the overlap calibration module acquires the lidar image, it creates a coordinate system based on the set lidar origin, expresses the checkerboard calibration board in the lidar image in the form of a coordinate matrix, and obtains the normal vector data and origin distance data in the lidar coordinate system.

[0047] The normal vector data is the normal vector corresponding to the plane of the chessboard calibration board, and the origin distance data is the shortest distance from the origin to the plane where the chessboard calibration board is located.

[0048] The parameter decomposition module acquires normal vector data and origin distance data in the visible light coordinate system, as well as normal vector data and origin distance data in the lidar coordinate system.

[0049] The parameter decomposition module constructs orthogonal constraints on the normal vector matrix. By maximizing the sum of the cosines of the angle between the normal vectors in the two coordinate systems, the analytical solution of the rotation matrix R is obtained through SVD (singular value decomposition).

[0050] The parameter decomposition module establishes the objective function of the distance difference between the lidar coordinate system and the visible light coordinate system to the plane where the checkerboard calibration board is located, and solves the optimal translation vector T by the least squares method;

[0051] The parameter decomposition module sends the analytical solutions of the optimal translation vector T and rotation matrix R to the fusion positioning module; the fusion positioning module unifies the visible light coordinate system and the lidar coordinate system through the analytical solutions of the optimal translation vector T and rotation matrix R, thereby completing the calibration of the lidar and the visible light camera.

[0052] Example 2:

[0053] Please see Figure 1 - Figure 2 As shown, the spatial attitude of the chessboard calibration board was adjusted multiple times, including position adjustment, tilt angle adjustment, and distance adjustment. After each adjustment, a calibration was performed using a lidar positioning calibration system. The calibration process included the following steps:

[0054] Step 1: Acquire LiDAR and visible light images;

[0055] Step 2: Create a lidar coordinate system based on the acquired lidar images, and create a visible light coordinate system based on the acquired visible light images;

[0056] Step 3: Extracting the normal vector data and origin distance data in the lidar coordinate system and the visible light coordinate system;

[0057] Step 4: Calculate the optimal translation vector T again based on the normal vector data and the distance data from the origin in both coordinate systems. X and rotation matrix R X The data is collected, where X is the number of calibration operations performed;

[0058] Step 5: Based on the optimal translation vector T obtained multiple times X and rotation matrix R X Construct constraint equations to complete the calibration process.

[0059] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A laser radar positioning calibration system based on visible light fusion analysis, characterized by, The laser radar acquisition module can collect the laser radar reflection of the checkerboard calibration board and record it as a laser radar picture. The visible light data acquisition module can collect the checkerboard calibration board through a visible light camera, record the collected picture, and preliminarily process the recorded picture to obtain a visible light picture. The overlapping calibration module can obtain the visible light picture and the laser radar picture, and create a spatial coordinate system for the laser radar picture and the visible light picture respectively to obtain normal vector data and origin distance data of the checkerboard calibration board. The parameter decomposition module is used for comprehensive comparison of the normal vector data and the origin distance data in the visible light coordinate system and the laser radar coordinate system, and solves the analytical solution of the rotation matrix R and the optimal translation vector T through a mathematical modeling method. The parameter decomposition module obtains the normal vector data and the origin distance data in the visible light coordinate system and the normal vector data and the origin distance data in the laser radar coordinate system. The parameter decomposition module constructs an orthogonal constraint of the normal vector matrix, maximizes the sum of the cosine of the angle between the normal vectors in the two coordinate systems, and obtains the analytical solution of the rotation matrix R through singular value decomposition. The parameter decomposition module establishes a distance difference objective function of the laser radar coordinate system and the visible light coordinate system to the plane where the checkerboard calibration board is located, and solves the optimal translation vector T through the least square method. The parameter decomposition module sends the optimal translation vector T and the analytical solution of the rotation matrix R to the fusion positioning module. The fusion positioning module records and comprehensively processes the analytical solution of the rotation matrix R and the optimal translation vector T obtained multiple times to obtain the fusion calibration result of the visible light camera and the laser radar.

2. The visible light fusion analysis based laser radar positioning calibration system according to claim 1, wherein, The laser radar picture collected by the laser radar acquisition module is a three-dimensional depth picture. The laser radar confirms the position of the concave point in the checkerboard calibration board through reflection imaging, and confirms the three-dimensional position of the checkerboard calibration board in the air according to the set weight ratio of the concave point position and the reflection of the checkerboard calibration board itself.

3. The visible light fusion analysis based laser radar positioning calibration system of claim 2, wherein, The method for the laser radar acquisition module to obtain the preset weight ratio is as follows: If the material of the checkerboard calibration board is a diffuse reflection material, the weights of the two are similar.

4. The visible light fusion analysis based laser radar positioning calibration system of claim 1, wherein, If the material of the checkerboard calibration board is not a diffuse reflection material or the diffuse reflection effect is insufficient, the concave point position weight is increased. The preliminary processing of the visible light picture by the visible light data acquisition module includes the following steps: S1: Extract the brightness of the visible light picture and quantize the brightness in numerical form to obtain the highest brightness point and the lowest brightness point in the visible light picture. S2: The visible light data acquisition module divides the visible light picture into n regions, each region is numbered as i, i=1, 2, 3, …, n, and samples the brightness of each region to obtain the region brightness Li. The visible light data acquisition module arithmetically averages all region brightnesses to obtain the overall picture brightness QL. S3: The visible light data acquisition module comprehensively evaluates the overall brightness, the highest brightness and the lowest brightness of the picture, generates a brightness adjustment parameter, and adjusts the camera imaging parameter through the brightness adjustment parameter to obtain new imaging parameters and obtain the final confirmed visible light picture.

5. The visible light fusion analysis based laser radar positioning calibration system of claim 1, wherein, The overlap calibration module identifies the checkerboard calibration board in the visible light picture, selects the intersection points of the black and white grids on the checkerboard calibration board as the saddle points, records all the saddle points to obtain the actual saddle point distribution, and obtains the set information of the checkerboard calibration board from a preset database, wherein the set information of the checkerboard calibration board includes the external size of the checkerboard calibration board, the size of the black and white checkerboard, and the preset saddle point position. The actual saddle point distribution obtained by identification and the preset saddle point position are overlapped, and the spatial distribution of the checkerboard calibration board is restored through the overlapping result. After obtaining the spatial distribution of the checkerboard calibration board, the overlap calibration module creates a coordinate system according to a preset camera origin, and expresses the spatial distribution of the checkerboard calibration board in the form of coordinates, so as to obtain normal vector data and origin distance data under the visible light coordinate system.

6. The visible light fusion analysis based laser radar positioning calibration system of claim 1, wherein, After obtaining the laser radar picture, the overlap calibration module creates a coordinate system according to a preset laser radar origin, expresses the checkerboard calibration board in the laser radar picture in the form of a coordinate matrix, and obtains normal vector data and origin distance data under the laser radar coordinate system.

7. The visible light fusion analysis based laser radar positioning calibration system of claim 1, wherein, The fusion positioning module obtains the optimal translation vector T multiple times. X and rotation matrix R X Then, the optimal translation vector T is determined by the constraint equations. X and rotation matrix R X The final value calibration is performed, and the visible light coordinate system and the lidar coordinate system are unified to complete the calibration process, where X is the number of times the optimal translation vector T and rotation matrix R are obtained.

Images