Three-dimensional perception radar bidirectional registration system based on multi-modal fusion analysis

Abstract

The invention relates to the technical field of multi-modal data registration, in particular to a three-dimensional perception radar bidirectional registration system based on multi-modal fusion analysis. Comprising a bidirectional registration center, a data acquisition module, a joint calibration module, a conversion processing module, a bidirectional registration module, a dynamic compensation module, a point cloud density interpolation unit, an efficiency tracking module and a registration management module. According to the method, forward registration and reverse registration of the 3D point cloud and the 2D image are realized by building a look-up table and a'forward registration analysis-reverse registration analysis' bidirectional mechanism, registration error correction is completed based on a distance error-reflection intensity correlation model, the point cloud'hole 'area is reduced through point cloud density interpolation, and the registration error correction accuracy is improved. The problem of incomplete sparse point cloud mapping is solved, the accuracy of target detection and semantic segmentation in a complex scene is improved, and meanwhile, the bidirectional registration operation is supervised to improve the accuracy and continuous stability of forward registration or reverse registration.

Description

Technical Field

[0001] This invention relates to the field of multimodal data registration technology, and in particular to a two-way registration system for three-dimensional sensing radar based on multimodal fusion analysis. Background Technology

[0002] In fields such as autonomous driving, intelligent robots, and environmental monitoring, 3D perception systems need to integrate 3D point cloud data from LiDAR with image data from RGB cameras to achieve comprehensive environmental perception. Among these, the registration of LiDAR and RGB images is the core link in multimodal fusion, and the registration accuracy and efficiency directly determine the performance of the 3D perception system.

[0003] In existing technologies, the registration of LiDAR and RGB images mostly relies on the intrinsic and extrinsic parameter matrices obtained through joint calibration. However, this has the following problems: Traditional registration processes require real-time calculation of the trigonometric function of the LiDAR scanning angle. Especially for multi-line LiDAR, the large amount of real-time calculation leads to data processing delays, making it difficult to meet real-time perception requirements. Furthermore, existing technologies do not consider the correlation between LiDAR distance error and reflection intensity. When the reflection intensity of the target object varies greatly, the distance measurement error will lead to a decrease in registration accuracy. Additionally, LiDAR point clouds are discrete data, and there are mapping gaps between them and the continuous pixels of the RGB image. Traditional interpolation methods cannot balance integrity and edge accuracy. Moreover, they lack the ability to reverse register from image to point cloud, which cannot meet the bidirectional data interaction requirements of scenarios such as target localization and depth estimation.

[0004] To address the aforementioned technical shortcomings, a solution is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a two-way registration system for 3D sensing radar based on multimodal fusion analysis to solve the aforementioned technical defects. This invention achieves high-precision two-way registration of LiDAR 3D point cloud and RGB image through the collaborative work of various modules, thereby improving the accuracy and continuous stability of forward or reverse registration and thus improving the positioning accuracy of the target.

[0006] The objective of this invention can be achieved through the following technical solution: a two-way registration system for three-dimensional sensing radar based on multimodal fusion analysis, comprising a two-way registration center, a data acquisition module, a joint calibration module, a conversion processing module, a two-way registration module, a dynamic compensation module, a point cloud density interpolation unit, an efficiency tracking module, and a registration management module;

[0007] The data acquisition module is used to simultaneously acquire 3D point cloud data from the lidar and image data from the RGB camera and send them to the bidirectional registration center for storage.

[0008] The joint calibration module is used to obtain the intrinsic parameter matrix (including focal length and principal point coordinates) and extrinsic parameter matrix (including rotation matrix R and translation vector T) of the lidar and RGB camera.

[0009] The conversion processing module constructs a lookup table (LUT) by pre-calculating the sine / cosine values ​​of the vertical angle.

[0010] The bidirectional registration module constructs a bidirectional mechanism of "forward registration analysis - reverse registration analysis" based on the intrinsic and extrinsic parameter matrices output by the joint calibration module, and obtains the pixel coordinates and 3D coordinates corresponding to the point cloud.

[0011] The dynamic compensation module is used to establish a correlation model between distance error and reflection intensity: Δr=m×F+n;

[0012] The point cloud density interpolation unit is based on the bird's-eye view (BEV) grid and fills sparse areas with density to obtain supplementary point cloud coordinates;

[0013] The efficiency tracking module is used to perform continuous stability efficiency analysis on the bidirectional registration operation and obtain alarm signals or stability signals.

[0014] Preferably, the analysis process of the conversion processing module is as follows:

[0015] Vertical angle parameter extraction: Read the hardware parameters of the lidar to obtain the vertical angle Li of the line laser beam, where i represents the number of scan lines. The vertical angle is a fixed parameter of the lidar at the factory and does not need to be measured in real time.

[0016] LUT building unit: pre-calculate the sine value sinLi and the cosine value cosLi of the vertical angle of each line, and store (Li, sine value sinLi, cosine value cosLi) as an array of LUTs;

[0017] For each polar coordinate point (azimuth φ, vertical angle Li, distance Rb) acquired by the lidar, sinLi and cosLi are obtained by querying the LUT, and the Cartesian coordinates (x, y, z) are directly calculated.

[0018] Preferably, the forward registration analysis process is as follows:

[0019] The coordinates (xa, ya, za) of target point A in the lidar coordinate system and the coordinates (ua, va) in the pixel coordinate system are obtained.

[0020] Based on the extrinsic parameter matrix (R, T) and the query LUT, the coordinates (xa, ya, za) of point A in the LiDAR coordinate system are converted to coordinates B (xb, yb, zb) in the camera coordinate system, i.e., [xa; ya; za] = R × [xb; yb; zb] + T;

[0021] Obtain the camera intrinsic parameter matrix K, and project the coordinates B (xb, yb, zb) in the camera coordinate system to the pixel coordinate system (ua', va').

[0022] Preferably, the reverse registration analysis process is as follows:

[0023] For any pixel (u, v) in the image, according to the inverse matrix K of the camera intrinsic parameter matrix K, -1 Calculate the ray direction vector (x_r, y_r, z_r) in the camera coordinate system:

[0024] [x_r;y_r;z_r]=K -1 ×[u;v;1]

[0025] Based on the depth distribution (zb) of the lidar point cloud in the camera coordinate system, the depth value zs corresponding to the pixel point (u,v) along the ray direction is calculated using linear interpolation. This is then combined with the inverse matrix (R) of the extrinsic parameter matrix. -1 ,-R -1 T), converting pixels into 3D coordinates (xa', ya', za') in the LiDAR coordinate system.

[0026] Preferably, the dynamic compensation module analysis process is as follows:

[0027] Under standard testing conditions, the deviation between the actual ranging value and the true distance of the lidar under different reflection intensities is collected to construct an error sample set (reflection intensity F, distance error Δr).

[0028] A linear regression algorithm was used to fit the error sample set to establish a correlation model between distance error and reflection intensity: Δr=m×F+n;

[0029] Where m and n are model parameters (obtained through sample fitting), the model can calculate the distance error under different reflection intensities in real time;

[0030] During the bidirectional registration process, for each point cloud data in the 3D point cloud data acquired by the lidar, its reflection intensity F is read, the distance error Δr is calculated through the association model, the original distance r is corrected: Rb=r-Δr, and the point cloud coordinates are recalculated based on the corrected distance value Rb.

[0031] Preferably, the analysis process of the point cloud density interpolation unit is as follows:

[0032] After forward registration of 3D point cloud data to RGB image, the initial point cloud data in the lidar coordinate system is projected onto the ground plane (z=0) to form a BEV image, and the BEV image is divided into a uniform grid.

[0033] The number of point clouds within each grid cell is counted, and the point cloud count is processed. Grid cells with a point cloud count greater than or equal to a preset point cloud count threshold are designated as dense regions, while grid cells with a point cloud count less than the preset point cloud count threshold are designated as sparse regions.

[0034] For sparse grids, a pre-set Kriging interpolation method is used to calculate the supplementary point cloud coordinates within the sparse grid based on the point cloud coordinates of adjacent dense grids.

[0035] Preferably, the analysis process of the efficiency tracking module is as follows:

[0036] The distance between two coordinate points is obtained based on the coordinates (ua, va) and (ua', va'). The distance between the points is then processed to determine whether the signal is qualified or unqualified.

[0037] The straight-line distance between two coordinates (xa', ya', za') and (xa, ya, za) is obtained, and the straight-line distance is processed to obtain a precise positioning signal or an optimized positioning signal.

[0038] Preferably, the ratio between the number of qualified signals generated under forward registration analysis and the total number of registrations within a preset time period and the ratio between the number of accurate positioning signals generated under reverse registration analysis and the total number of registrations are obtained, and these ratios are set as forward registration efficiency and reverse registration efficiency, respectively. The forward registration efficiency and reverse registration efficiency are then processed to obtain an alarm signal or a stable signal.

[0039] The beneficial effects of this invention are as follows:

[0040] (1) This invention constructs a lookup table by pre-calculating trigonometric functions, which reduces the amount of real-time calculation and conversion time, thereby improving the efficiency of lidar point cloud preprocessing and meeting the real-time perception requirements;

[0041] (2) This invention realizes forward and reverse registration of 3D point cloud and 2D image by constructing a bidirectional mechanism of "forward registration analysis - reverse registration analysis". Based on the distance error-reflection intensity correlation model, it compensates for distance measurement error to complete the registration error correction. At the same time, it effectively controls the positioning error of bidirectional registration to meet the high-precision positioning requirements of autonomous driving.

[0042] (3) This invention reduces the area of ​​“holes” in the point cloud by interpolating the point cloud density, solves the problem of incomplete mapping of sparse point clouds, improves the accuracy of target detection and semantic segmentation in complex scenes, and monitors the continuous stability efficiency of bidirectional registration operation so as to make targeted adjustments to the errors in forward or reverse registration, so as to ensure the accuracy of forward or reverse registration and thus improve the positioning accuracy of the target. Attached Figure Description

[0043] The invention will now be further described with reference to the accompanying drawings;

[0044] Figure 1 This is a flowchart of the system of the present invention;

[0045] Figure 2 This is a partial reference diagram of Embodiment 2 of the present invention;

[0046] Figure 3 This is a partial reference diagram of Embodiment 3 of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments;

[0049] Example 1:

[0050] Please see Figures 1 to 3 As shown, the present invention is a two-way registration system for three-dimensional sensing radar based on multimodal fusion analysis, including a two-way registration center, a data acquisition module, a joint calibration module, a conversion processing module, a two-way registration module, a dynamic compensation module, a point cloud density interpolation unit, an efficiency tracking module, and a registration management module;

[0051] The bidirectional registration center has a bidirectional communication connection with the data acquisition module. The bidirectional registration center has a unidirectional communication connection with both the joint calibration module and the conversion processing module. The joint calibration module and the conversion processing module have a unidirectional communication connection with the bidirectional registration module. The bidirectional registration module has a unidirectional communication connection with the dynamic compensation module, the point cloud density interpolation unit, and the efficiency tracking module. The dynamic compensation module, the point cloud density interpolation unit, and the efficiency tracking module have a unidirectional communication connection with the registration management module.

[0052] S1: The data acquisition module is used to simultaneously acquire 3D point cloud data of the lidar (including azimuth angle, vertical angle, distance and reflection intensity information of each point) and image data of the RGB camera and send them to the bidirectional registration center for storage;

[0053] S2: The joint calibration module is used to obtain the intrinsic parameter matrix (including focal length, principal point coordinates, etc.) and extrinsic parameter matrix (including rotation matrix R and translation vector T) of the lidar and RGB camera.

[0054] That is, a checkerboard-reflective point composite calibration plate is used. The checkerboard is used for camera intrinsic parameter calibration (number of corner points ≥ 12×10), and the reflective points are used for lidar extrinsic parameter calibration (reflective point diameter ≥ 5mm, reflectivity ≥ 80%).

[0055] Using Zhang Zhengyou's calibration method, image data of the calibration board at different angles were collected, and the camera intrinsic parameter matrix (including focal length, principal point coordinates, etc.) was calculated.

[0056] By using the hand-eye calibration method, the 3D coordinates of the reflective points on the calibration board scanned by the lidar and the 2D coordinates of the calibration board captured by the camera are collected, and the transformation relationship between the lidar coordinate system and the camera coordinate system is established (external parameter matrix, including rotation matrix R and translation vector T).

[0057] S3: The transformation processing module pre-calculates the sine / cosine values ​​of the vertical angle and constructs a lookup table (LUT) to achieve efficient transformation from polar coordinate point clouds to Cartesian coordinates. Specifically, this includes:

[0058] Vertical angle parameter extraction: Read the hardware parameters of the lidar to obtain the vertical angle Li of the line laser beam, where i represents the number of scan lines. The vertical angle is a fixed parameter of the lidar at the factory and does not need to be measured in real time.

[0059] LUT building unit: pre-calculate the sine value sinLi and the cosine value cosLi of the vertical angle of each line, and store (Li, sine value sinLi, cosine value cosLi) as an array of LUTs in system memory;

[0060] For each polar coordinate point (azimuth φ, vertical angle Li, distance Rb) acquired by the lidar, sinLi and cosLi are obtained by querying the LUT, and the Cartesian coordinates (x, y, z) are directly calculated:

[0061] x = Rb × cosLi × cosφ;

[0062] y = Rb × cosLi × sinφ;

[0063] z = Rb × sinLi;

[0064] This conversion method avoids the real-time calculation of trigonometric functions in traditional methods, reduces the computational load and conversion time of polar coordinate transformation, and meets real-time requirements.

[0065] Example 2:

[0066] S4: The bidirectional registration module, based on the intrinsic and extrinsic parameter matrices output by the joint calibration module, constructs a bidirectional mechanism of "forward registration analysis - inverse registration analysis" to obtain the pixel coordinates and 3D coordinates corresponding to the point cloud, thereby achieving forward and inverse registration between the 3D point cloud and the 2D image. Specifically, it includes:

[0067] The forward registration analysis process is as follows:

[0068] The coordinates (xa, ya, za) of target point A in the lidar coordinate system and the coordinates (ua, va) in the pixel coordinate system are obtained.

[0069] The 3D point cloud from the LiDAR is transformed to the camera coordinate system and then mapped to the pixel coordinate system, achieving initial alignment between the 3D point cloud and the 2D image.

[0070] Based on the extrinsic parameter matrix (R, T) and the query LUT, the coordinates (xa, ya, za) of point A in the LiDAR coordinate system are converted to coordinates B (xb, yb, zb) in the camera coordinate system, i.e., [xa; ya; za] = R × [xb; yb; zb] + T;

[0071] Obtain the camera intrinsic parameter matrix K and project the coordinates B (xb, yb, zb) in the camera coordinate system to the pixel coordinate system (ua', va').

[0072] [u; v; 1] = K × [xb; yb; zb];

[0073] Where K is a 3×3 intrinsic parameter matrix, containing the camera focal length (f_x, f_y), i.e. the focal lengths along the x and y axes, and the principal point coordinates (u0, v0).

[0074] Reverse registration analysis process:

[0075] Based on inverse transform and depth interpolation, 2D pixels of the RGB image are located to the 3D point cloud of the LiDAR, realizing the back projection of the target detection box, specifically including:

[0076] For any pixel (u, v) in the image, according to the inverse matrix K of the camera intrinsic parameter matrix K, -1 Calculate the ray direction vector (x_r, y_r, z_r) in the camera coordinate system:

[0077] [x_r;y_r;z_r]=K -1 ×[u;v;1]

[0078] Based on the depth distribution (zb) of the lidar point cloud in the camera coordinate system, the depth value zs corresponding to the pixel point (u,v) along the ray direction is calculated using linear interpolation. This is then combined with the inverse matrix (R) of the extrinsic parameter matrix. -1 ,-R -1T), converting pixels into 3D coordinates (xa', ya', za') in the LiDAR coordinate system;

[0079] [xa'; ya'; za'] = R -1 ×(zs×[x_r;y_r;z_r]-T).

[0080] Example 3:

[0081] S5: The dynamic compensation module is used to establish a correlation model between distance error and reflection intensity, and to dynamically compensate for the lidar ranging data. The dynamic compensation analysis process is as follows:

[0082] Under standard testing conditions, the deviation between the actual ranging value and the true distance of the lidar under different reflection intensities is collected to construct an error sample set (reflection intensity F, distance error Δr).

[0083] A linear regression algorithm was used to fit the error sample set to establish a correlation model between distance error and reflection intensity: Δr=m×F+n;

[0084] Where m and n are model parameters (obtained through sample fitting), the model can calculate the distance error under different reflection intensities in real time;

[0085] During the bidirectional registration process, for each point cloud data in the 3D point cloud data acquired by the lidar, its reflection intensity F is read, the distance error Δr is calculated through the association model, the original distance r is corrected: Rb=r-Δr, and the point cloud coordinates are recalculated based on the corrected distance value Rb to complete the registration error correction.

[0086] S6: The point cloud density interpolation unit is based on the bird's-eye view (BEV) grid and fills sparse regions with density:

[0087] Simultaneously, after forward registration of 3D point cloud data to RGB image, the initial point cloud data in the lidar coordinate system is projected onto the ground plane (z=0) to form a BEV image, and the BEV image is divided into a uniform grid.

[0088] The number of point clouds within each grid cell is counted, and the point cloud count is processed. Grid cells with a point cloud count greater than or equal to a preset point cloud count threshold are designated as dense regions, while grid cells with a point cloud count less than the preset point cloud count threshold are designated as sparse regions.

[0089] For sparse grids, a pre-set Kriging interpolation method is used to calculate the supplementary point cloud coordinates within the sparse grid based on the point cloud coordinates of adjacent dense grids.

[0090] The registration management module responds to the supplementary point cloud coordinates and immediately supplements the sparse grid with point cloud to achieve uniform point cloud density, while ensuring the integrity and edge consistency of the mapping between the point cloud and the image.

[0091] In the processing of point clouds for 3D sensing radar, the lack of point clouds in sparse grids (such as radar scanning blind spots and low-density areas at long distances) can lead to a decrease in registration accuracy and incomplete environmental perception. Kriging, as a geostatistical interpolation method based on spatial correlation, can accurately calculate the coordinates of supplementary point clouds in sparse grids by using the spatial distribution characteristics of point clouds in adjacent dense grids, thereby achieving uniform point cloud density across the entire region.

[0092] S7: The efficiency tracking module is used to perform continuous stability efficiency analysis on bidirectional registration operations. The specific process is as follows:

[0093] The distance between two points is obtained based on the coordinates (ua, va) and (ua', va').

[0094] The distance between points is judged. If the distance between points is less than the preset distance threshold, a qualified signal is generated. If the distance between points is greater than or equal to the preset distance threshold, an unqualified signal is generated. The registration management module responds to the unqualified signal and reminds the operation and management personnel to manage the reverse registration to ensure the accuracy of the reverse registration.

[0095] The straight-line distance between the two coordinates (xa', ya', za') and (xa, ya, za) is obtained, and the straight-line distance is processed. If the straight-line distance is less than the preset straight-line distance threshold, a positioning accuracy signal is generated. If the straight-line distance is greater than or equal to the preset straight-line distance threshold, an optimization signal is generated. The registration management module responds to the optimization signal to remind the operation and management personnel to manage the forward registration to ensure the accuracy of the forward registration.

[0096] The ratio of the number of qualified signals generated under forward registration analysis to the total number of registrations within a preset time period and the ratio of the number of accurate positioning signals generated under reverse registration analysis to the total number of registrations are obtained and set as forward registration efficiency and reverse registration efficiency, respectively. The forward registration efficiency and reverse registration efficiency are judged. If the forward registration efficiency is less than the preset forward registration efficiency threshold, or the reverse registration efficiency is less than the preset reverse registration efficiency threshold, an alarm signal is generated. If the forward registration efficiency is greater than or equal to the preset forward registration efficiency threshold and the reverse registration efficiency is greater than or equal to the preset reverse registration efficiency threshold, a stable signal is generated.

[0097] The registration management module responds to alarm signals or stabilization signals and immediately performs the preset early warning operations corresponding to the alarm signals or stabilization signals. Based on the feedback information, it can intuitively understand whether the entire bidirectional registration is continuously stable, so as to make targeted adjustments to the errors existing in the forward or reverse registration, so as to ensure the accuracy of forward or reverse registration and thus improve the positioning accuracy of the target.

[0098] It effectively improves the accuracy and efficiency of multimodal data registration in 3D perception systems, providing reliable multimodal data support for scenarios such as autonomous driving and environmental perception;

[0099] In summary, this invention reduces real-time computation and conversion time by constructing lookup tables through pre-calculated trigonometric functions, thereby improving the efficiency of LiDAR point cloud preprocessing and meeting real-time perception requirements. Simultaneously, by constructing a bidirectional mechanism of "forward registration analysis - inverse registration analysis," it achieves forward and inverse registration of 3D point clouds with 2D images. Furthermore, based on a distance error-reflection intensity correlation model, it compensates for distance measurement errors to correct registration errors, effectively controlling the positioning error of bidirectional registration to meet the high-precision positioning requirements of autonomous driving. Additionally, by using point cloud density interpolation, it reduces the area of ​​"holes" in the point cloud, solving the problem of incomplete mapping of sparse point clouds and improving the accuracy of target detection and semantic segmentation in complex scenes. Furthermore, it monitors the continuous stability and efficiency of bidirectional registration operations to make targeted adjustments to errors in forward or inverse registration, ensuring the accuracy of both and thus improving target positioning accuracy.

[0100] The threshold is set for comparative analysis of results to determine whether they are good or bad. The value of the threshold is determined by a combination of large-scale model analysis of sample data and human experience. It can also be adjusted appropriately based on seasonal or common-sense influencing factors.

[0101] The size of the coefficient is a specific value obtained by quantifying each parameter to facilitate subsequent comparison. The size of the coefficient depends on the amount of sample data and the corresponding operating coefficient initially set by those skilled in the art for each set of sample data; as long as it does not affect the proportional relationship between the parameter and the quantified value.

[0102] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A two-way registration system for three-dimensional sensing radar based on multimodal fusion analysis, characterized in that, It includes a two-way registration center, a data acquisition module, a joint calibration module, a conversion processing module, a two-way registration module, a dynamic compensation module, a point cloud density interpolation unit, an efficiency tracking module, and a registration management module; The data acquisition module is used to simultaneously acquire 3D point cloud data from the lidar and image data from the RGB camera and send them to the bidirectional registration center for storage. The joint calibration module is used to obtain the intrinsic parameter matrix (including focal length and principal point coordinates) and extrinsic parameter matrix (including rotation matrix R and translation vector T) of the lidar and RGB camera. The conversion processing module constructs a lookup table (LUT) by pre-calculating the sine / cosine values ​​of the vertical angle. The bidirectional registration module constructs a bidirectional mechanism of "forward registration analysis - reverse registration analysis" based on the intrinsic and extrinsic parameter matrices output by the joint calibration module, and obtains the pixel coordinates and 3D coordinates corresponding to the point cloud. The dynamic compensation module is used to establish a correlation model between distance error and reflection intensity: Δr=m×F+n; The point cloud density interpolation unit is based on the bird's-eye view (BEV) grid and fills sparse areas with density to obtain supplementary point cloud coordinates; The efficiency tracking module is used to perform continuous stability efficiency analysis on the bidirectional registration operation and obtain alarm signals or stability signals.

2. The three-dimensional sensing radar bidirectional registration system based on multimodal fusion analysis according to claim 1, characterized in that, The analysis process of the conversion processing module is as follows: Vertical angle parameter extraction: Read the hardware parameters of the lidar to obtain the vertical angle Li of the line laser beam, where i represents the number of scan lines. The vertical angle is a fixed parameter of the lidar at the factory and does not need to be measured in real time. LUT building unit: pre-calculate the sine value sinLi and the cosine value cosLi of the vertical angle of each line, and store (Li, sine value sinLi, cosine value cosLi) as an array of LUTs; For each polar coordinate point (azimuth φ, vertical angle Li, distance Rb) acquired by the lidar, sinLi and cosLi are obtained by querying the LUT, and the Cartesian coordinates (x, y, z) are directly calculated.

3. The three-dimensional sensing radar bidirectional registration system based on multimodal fusion analysis according to claim 1, characterized in that, The forward registration analysis process is as follows: The coordinates (xa, ya, za) of target point A in the lidar coordinate system and the coordinates (ua, va) in the pixel coordinate system are obtained. Based on the extrinsic parameter matrix (R, T) and the query LUT, the coordinates (xa, ya, za) of point A in the LiDAR coordinate system are converted to coordinates B (xb, yb, zb) in the camera coordinate system, i.e., [xa; ya; za] = R × [xb; yb; zb] + T; Obtain the camera intrinsic parameter matrix K, and project the coordinates B (xb, yb, zb) in the camera coordinate system to the pixel coordinate system (ua', va').

4. The three-dimensional sensing radar bidirectional registration system based on multimodal fusion analysis according to claim 1, characterized in that, The reverse registration analysis process is as follows: For any pixel (u, v) in the image, according to the inverse matrix K of the camera intrinsic parameter matrix K, -1 Calculate the ray direction vector (x_r, y_r, z_r) in the camera coordinate system: [x_r;y_r;z_r]=K -1 ×[u;v;1] Based on the depth distribution (zb) of the lidar point cloud in the camera coordinate system, the depth value zs corresponding to the pixel point (u,v) along the ray direction is calculated using linear interpolation. This is then combined with the inverse matrix (R) of the extrinsic parameter matrix. -1 ,-R -1 T), converting pixels into 3D coordinates (xa', ya', za') in the LiDAR coordinate system.

5. The three-dimensional sensing radar bidirectional registration system based on multimodal fusion analysis according to claim 1, characterized in that, The analysis process of the dynamic compensation module is as follows: Under standard testing conditions, the deviation between the actual ranging value and the true distance of the lidar under different reflection intensities is collected to construct an error sample set (reflection intensity F, distance error Δr). A linear regression algorithm was used to fit the error sample set to establish a correlation model between distance error and reflection intensity: Δr=m×F+n; Where m and n are model parameters (obtained through sample fitting), the model can calculate the distance error under different reflection intensities in real time; During the bidirectional registration process, for each point cloud data in the 3D point cloud data acquired by the lidar, its reflection intensity F is read, the distance error Δr is calculated through the association model, the original distance r is corrected: Rb=r-Δr, and the point cloud coordinates are recalculated based on the corrected distance value Rb.

6. The three-dimensional sensing radar bidirectional registration system based on multimodal fusion analysis according to claim 1, characterized in that, The analysis process of the point cloud density interpolation unit is as follows: After forward registration of 3D point cloud data to RGB image, the initial point cloud data in the lidar coordinate system is projected onto the ground plane (z=0) to form a BEV image, and the BEV image is divided into a uniform grid. The number of point clouds in each grid is counted and processed. Grids with a number of point clouds greater than or equal to a preset threshold are set as dense regions, and grids with a number of point clouds less than the preset threshold are set as sparse regions. For sparse grids, a pre-set Kriging interpolation method is used to calculate the supplementary point cloud coordinates within the sparse grid based on the point cloud coordinates of adjacent dense grids.

7. The three-dimensional sensing radar bidirectional registration system based on multimodal fusion analysis according to claim 1, characterized in that, The analysis process of the efficiency tracking module is as follows: The distance between two coordinate points is obtained based on the coordinates (ua, va) and (ua', va'). The distance between the points is then processed to determine whether the signal is qualified or unqualified. The straight-line distance between two coordinates (xa', ya', za') and (xa, ya, za) is obtained, and the straight-line distance is processed to obtain a precise positioning signal or an optimized positioning signal.

8. The three-dimensional sensing radar bidirectional registration system based on multimodal fusion analysis according to claim 7, characterized in that, The ratio of the number of qualified signals generated under forward registration analysis to the total number of registrations within a preset time period and the ratio of the number of accurate positioning signals generated under reverse registration analysis to the total number of registrations are obtained and set as forward registration efficiency and reverse registration efficiency, respectively. The forward registration efficiency and reverse registration efficiency are then processed to obtain alarm signals or stable signals.

Images