Radar and camera depth fusion calibration method and system based on high-precision map

Abstract

The invention relates to a radar and camera depth fusion calibration method and system based on a high-precision map, and the method comprises the steps: M1, enabling a vehicle to run on a road, obtaining the data information of a lane line in real time based on the high-precision map, obtaining the point cloud data information of the road in real time based on a vehicle-mounted laser radar, the method comprises the following steps: acquiring image data information of a road in real time based on a vehicle-mounted camera, extracting image features of a lane line by adopting a lane line image feature extraction algorithm based on a Sobel operator and a Roberts operator to obtain image feature data information of the lane line, and matching the image feature data information with the data information of the lane line, and obtaining image feature data information of the matched lane line. According to the method, the high-precision map lane line true value is used as a calibration reference, the calibration precision and reliability are improved, automatic online calibration of deep fusion of the laser radar and the camera is realized, and the sensing precision and safety of an automatic driving system are improved.

Description

Technical Field

[0001] The present invention relates to the field of laser radar and camera calibration technology, and in particular to a radar and camera deep fusion calibration method and system based on high-precision maps. Background Art

[0002] With the rapid development of autonomous driving technology, mass-produced intelligent driving vehicles such as highway and urban NOA have gradually entered the market, bringing great convenience to people's travel. However, autonomous driving vehicles face many challenges in practical applications, among which sensor calibration is a key link to ensure the perception accuracy and reliability of autonomous driving systems. As the main perception sensors of autonomous driving vehicles, the accurate alignment and fusion of lidar and camera data is crucial to the decision-making of autonomous driving systems.

[0003] Traditional sensor calibration methods mostly rely on manually set calibration plates or static scenes, which are complex to operate and difficult to adapt to the complex and changeable road environment of autonomous vehicles in actual driving. In addition, the calibration process often requires the participation of professional technicians, which increases costs and time. At the same time, the storage, analysis and optimization of calibration data also face many challenges, especially in scenarios where the amount of data is huge and real-time processing is required. Summary of the invention

[0004] In view of the above problems, the present invention provides a radar and camera deep fusion calibration method and system based on high-precision maps, which not only uses the true value of the lane line in the high-precision map as a calibration reference, thereby improving the calibration accuracy and reliability, but also realizes automatic online calibration of the deep fusion of lidar and camera, thereby improving the perception accuracy and safety of the autonomous driving system.

[0005] In order to achieve the above-mentioned purpose and other related purposes, the technical solution provided by the present invention is as follows:

[0006] A radar and camera deep fusion calibration method based on high-precision maps, the method comprising:

[0007] M1. When a vehicle is driving on a road, it obtains lane line data information in real time based on a high-precision map, obtains road point cloud data information in real time based on a vehicle-mounted laser radar, and obtains road image data information in real time based on a vehicle-mounted camera;

[0008] M2. Based on the image data information of the road, the lane line image feature extraction algorithm based on the Sobel operator and the Roberts operator is used to extract the image features of the lane line to obtain the image feature data information of the lane line, and the image feature data information of the lane line is matched with the data information of the lane line to obtain the matched image feature data information of the lane line;

[0009] M3. Based on the point cloud data information of the road, the point cloud features of the lane line are extracted using an improved point cloud feature extraction algorithm based on deep learning to obtain data information of the point cloud features of the lane line;

[0010] M4. Based on the point cloud feature data information of the lane line and the image feature data information of the matched lane line, a deep fusion calibration algorithm of point cloud and image based on gradient neural network optimization is used to jointly calibrate the laser radar and camera to obtain data information of the joint calibration parameters of the laser radar and camera.

[0011] Furthermore, in step M2, the extracting of the image features of the lane line using a lane line image feature extraction algorithm based on a Sobel operator and a Roberts operator includes:

[0012] M21. Based on the image data information of the road, a gradient function Q of the pixel points in the image is established,

[0013]

[0014] Where r is the image data information of the road, α1, α2 and α3 are the weight coefficients of the pixels in the image, and the gradient value of each pixel in the image is calculated to obtain the data information of the gradient value of the pixel of the road image;

[0015] M22. Based on the data information of the gradient value of the pixel points of the road image, establish the image feature extraction function W of the lane line,

[0016]

[0017] Among them, x is the data information of the gradient value of the pixel point of the road image, f is the edge feature extraction function of the Sobel operator, g is the edge feature extraction function of the Roberts operator, β1, β2 and β3 are the image feature extraction factors of the lane line;

[0018] M23. Based on the lane line image feature extraction function W, the image features of the lane line are extracted to obtain the image feature data information of the lane line.

[0019] Furthermore, the Sobel operator edge feature extraction function f is,

[0020]

[0021] The Roberts operator edge feature extraction function g is:

[0022]

[0023] Wherein, x is the data information of the gradient value of the pixel point of the road image.

[0024] Furthermore, the lane line image feature extraction factors β1, β2 and β3 are:

[0025]

[0026]

[0027] Wherein, x is the data information of the gradient value of the pixel point of the road image.

[0028] Furthermore, in step M3, extracting the point cloud features of the lane line using the improved point cloud feature extraction algorithm based on deep learning includes:

[0029] M31. Based on the point cloud data information of the road, construct a point cloud data set of the road;

[0030] M32. Input the point cloud data set of the road into the point cloud feature extraction model of deep learning for training and learning, and determine the feature extraction function R of the model,

[0031]

[0032] Among them, y is the point cloud dataset of the road, δ1, δ2 and δ3 are the feature reward factors of the point cloud of the road, and the trained deep learning point cloud feature extraction model is obtained;

[0033] M33. Based on the trained deep learning point cloud feature extraction model, the point cloud dataset of the road is input, the point cloud features of the lane line are extracted, and the data information of the point cloud features of the lane line is obtained.

[0034] Furthermore, the constraints of the feature reward factors δ1, δ2 and δ3 of the road point cloud are:

[0035]

[0036] Furthermore, in step M4, the joint calibration of the laser radar and the camera using a deep fusion calibration algorithm of point cloud and image based on gradient neural network optimization includes:

[0037] M41. Based on the point cloud feature data information of the lane line and the image feature data information of the matched lane line, a multimodal fusion function P of the lane line is established.

[0038]

[0039] Among them, z1 is the point cloud feature data information of the lane line, z2 is the image feature data information of the matched lane line, γ1, γ2 and γ3 are the multimodal fusion factors of the lane line, and the features of the point cloud and image of the lane line are fused to obtain the fused feature data information of the lane line;

[0040] M42. Based on the characteristic data information of the fused lane line, a joint calibration function S of the laser radar and the camera is established.

[0041]

[0042] Wherein, h is the characteristic data information of the fused lane line, η1, η2 and η3 are the joint calibration factors of the laser radar and the camera, and the joint calibration factors of the laser radar and the camera are input into the gradient neural network for optimization;

[0043] M43. Based on the joint calibration function S of the laser radar and the camera, the laser radar and the camera are jointly calibrated to obtain data information of the joint calibration parameters of the laser radar and the camera.

[0044] Furthermore, the inputting the joint calibration factor of the laser radar and the camera into the gradient neural network for optimization is to input the joint calibration factor of the laser radar and the camera into the gradient neural network for training and learning, and construct an optimization function G of the network,

[0045]

[0046] Among them, q is the joint calibration factor of the lidar and the camera, λ1, λ2 and λ3 are optimization factors, and the joint calibration factor of the lidar and the camera is optimized to obtain the optimized joint calibration factor of the lidar and the camera.

[0047] In order to achieve the above-mentioned purpose and other related purposes, the present invention also provides a radar and camera deep fusion calibration system based on high-precision maps, including a computer device, which is programmed or configured to execute any one of the steps of the radar and camera deep fusion calibration method based on high-precision maps.

[0048] In order to achieve the above-mentioned objectives and other related objectives, the present invention also provides a computer-readable storage medium, which stores a computer program programmed or configured to execute any one of the high-precision map-based radar and camera deep fusion calibration methods.

[0049] The present invention has the following positive effects:

[0050] 1. The present invention extracts the image features of lane lines by adopting a lane line image feature extraction algorithm based on the Sobel operator and the Roberts operator, and extracts the point cloud features of lane lines by combining an improved point cloud feature extraction algorithm based on deep learning. It not only uses the true value of the lane line in the high-precision map as a calibration reference to improve the accuracy and reliability of calibration, but also realizes the deep fusion of laser radar and camera for automatic online calibration, thereby improving the perception accuracy and safety of the autonomous driving system.

[0051] 2. The present invention jointly calibrates the lidar and camera by adopting a deep fusion calibration algorithm of point cloud and image based on gradient neural network optimization, which not only realizes the storage, analysis and optimization of calibration data, provides data support for further optimization of the autonomous driving system, but also reduces the complexity and cost of the calibration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0053] Figure 2 It is a flow chart of the lane line image feature extraction algorithm based on the Sobel operator and the Roberts operator of the present invention;

[0054] Figure 3 It is a flowchart of the improved point cloud feature extraction algorithm based on deep learning of the present invention;

[0055] Figure 4 It is a flow chart of the deep fusion calibration algorithm of point cloud and image based on gradient neural network optimization of the present invention;

[0056] Figure 5 Schematic diagram of the system framework of the present invention. DETAILED DESCRIPTION

[0057] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0058] Example 1: Figure 1 As shown, a radar and camera deep fusion calibration method based on high-precision maps, the method comprising:

[0059] M1. When a vehicle is driving on a road, it obtains lane line data information in real time based on a high-precision map, obtains road point cloud data information in real time based on a vehicle-mounted laser radar, and obtains road image data information in real time based on a vehicle-mounted camera;

[0060] M2. Based on the image data information of the road, the lane line image feature extraction algorithm based on the Sobel operator and the Roberts operator is used to extract the image features of the lane line to obtain the image feature data information of the lane line, and the image feature data information of the lane line is matched with the data information of the lane line to obtain the matched image feature data information of the lane line;

[0061] M3. Based on the point cloud data information of the road, the point cloud features of the lane line are extracted using an improved point cloud feature extraction algorithm based on deep learning to obtain data information of the point cloud features of the lane line;

[0062] M4. Based on the point cloud feature data information of the lane line and the image feature data information of the matched lane line, a deep fusion calibration algorithm of point cloud and image based on gradient neural network optimization is used to jointly calibrate the laser radar and camera to obtain data information of the joint calibration parameters of the laser radar and camera.

[0063] In this embodiment, if Figure 2 As shown, in step M2, the lane line image feature extraction algorithm based on the Sobel operator and the Roberts operator is used to extract the image features of the lane line, including:

[0064] M21. Based on the image data information of the road, a gradient function Q of the pixel points in the image is established,

[0065]

[0066] Where r is the image data information of the road, α1, α2 and α3 are the weight coefficients of the pixels in the image, and the gradient value of each pixel in the image is calculated to obtain the data information of the gradient value of the pixel of the road image;

[0067] M22. Based on the data information of the gradient value of the pixel points of the road image, establish the image feature extraction function W of the lane line,

[0068]

[0069] Among them, x is the data information of the gradient value of the pixel point of the road image, f is the edge feature extraction function of the Sobel operator, g is the edge feature extraction function of the Roberts operator, β1, β2 and β3 are the image feature extraction factors of the lane line;

[0070] M23. Based on the lane line image feature extraction function W, the image features of the lane line are extracted to obtain the image feature data information of the lane line.

[0071] In this embodiment, the Sobel operator edge feature extraction function f is:

[0072]

[0073] The Roberts operator edge feature extraction function g is:

[0074]

[0075] Wherein, x is the data information of the gradient value of the pixel point of the road image.

[0076] In this embodiment, the lane line image feature extraction factors β1, β2 and β3 are:

[0077]

[0078] Wherein, x is the data information of the gradient value of the pixel point of the road image.

[0079] In this embodiment, if Figure 3 As shown, in step M3, the step of extracting the point cloud features of the lane line using the improved point cloud feature extraction algorithm based on deep learning includes:

[0080] M31. Based on the point cloud data information of the road, construct a point cloud data set of the road;

[0081] M32. Input the point cloud data set of the road into the point cloud feature extraction model of deep learning for training and learning, and determine the feature extraction function R of the model,

[0082]

[0083] Among them, y is the point cloud dataset of the road, δ1, δ2 and δ3 are the feature reward factors of the point cloud of the road, and the trained deep learning point cloud feature extraction model is obtained;

[0084] M33. Based on the trained deep learning point cloud feature extraction model, the point cloud dataset of the road is input, the point cloud features of the lane line are extracted, and the data information of the point cloud features of the lane line is obtained.

[0085] In this embodiment, the constraints of the feature reward factors δ1, δ2 and δ3 of the road point cloud are:

[0086]

[0087] Example 2: Based on the radar and camera deep fusion calibration method based on high-precision map in Example 1, the present invention is further illustrated and described below.

[0088] like Figure 1 As shown, a radar and camera deep fusion calibration method based on high-precision maps, the method comprising:

[0089] M1. When a vehicle is driving on a road, it obtains lane line data information in real time based on a high-precision map, obtains road point cloud data information in real time based on a vehicle-mounted laser radar, and obtains road image data information in real time based on a vehicle-mounted camera;

[0090] M2. Based on the image data information of the road, the lane line image feature extraction algorithm based on the Sobel operator and the Roberts operator is used to extract the image features of the lane line to obtain the image feature data information of the lane line, and the image feature data information of the lane line is matched with the data information of the lane line to obtain the matched image feature data information of the lane line;

[0091] M3. Based on the point cloud data information of the road, the point cloud features of the lane line are extracted using an improved point cloud feature extraction algorithm based on deep learning to obtain data information of the point cloud features of the lane line;

[0092] M4. Based on the point cloud feature data information of the lane line and the image feature data information of the matched lane line, a deep fusion calibration algorithm of point cloud and image based on gradient neural network optimization is used to jointly calibrate the laser radar and camera to obtain data information of the joint calibration parameters of the laser radar and camera.

[0093] In this embodiment, if Figure 4 As shown, in step M4, the laser radar and camera are jointly calibrated using a deep fusion calibration algorithm of point cloud and image based on gradient neural network optimization, including:

[0094] M41. Based on the point cloud feature data information of the lane line and the image feature data information of the matched lane line, a multimodal fusion function P of the lane line is established.

[0095]

[0096] Among them, z1 is the point cloud feature data information of the lane line, z2 is the image feature data information of the matched lane line, γ1, γ2 and γ3 are the multimodal fusion factors of the lane line, and the features of the point cloud and image of the lane line are fused to obtain the fused feature data information of the lane line;

[0097] M42. Based on the characteristic data information of the fused lane line, a joint calibration function S of the laser radar and the camera is established.

[0098]

[0099] Wherein, h is the characteristic data information of the fused lane line, η1, η2 and η3 are the joint calibration factors of the laser radar and the camera, and the joint calibration factors of the laser radar and the camera are input into the gradient neural network for optimization;

[0100] M43. Based on the joint calibration function S of the laser radar and the camera, the laser radar and the camera are jointly calibrated to obtain data information of the joint calibration parameters of the laser radar and the camera.

[0101] In this embodiment, the inputting the joint calibration factor of the laser radar and the camera into the gradient neural network for optimization is to input the joint calibration factor of the laser radar and the camera into the gradient neural network for training and learning, and construct an optimization function G of the network.

[0102]

[0103] Among them, q is the joint calibration factor of the lidar and the camera, λ1, λ2 and λ3 are optimization factors, and the joint calibration factor of the lidar and the camera is optimized to obtain the optimized joint calibration factor of the lidar and the camera.

[0104] In this embodiment, the present invention provides a radar and camera deep fusion calibration system based on high-precision maps, including a computer device that is programmed or configured to execute any step of the radar and camera deep fusion calibration method based on high-precision maps.

[0105] In this embodiment, if Figure 5 As shown, HD map lane line information extraction: extract the lane line information of the current driving section from the HD map database, including the location, shape, direction and other detailed information of the lane line, as the real reference benchmark for calibration;

[0106] LiDAR and camera data collection: The on-board LiDAR and camera synchronously collect road environment data, including point cloud data and image data. These data will be used in the subsequent calibration process;

[0107] HMI calibration interface design: Design an intuitive and convenient HMI calibration interface to display the true value of HD map lane lines, data collected by radar and camera, calibration results and other information. Users can perform calibration operations through the HMI interface, such as selecting the calibration section, starting the calibration process, and viewing the calibration results;

[0108] Lane feature extraction and matching: Use image processing algorithms to extract lane features from camera images and match them with the true lane values ​​in the HD map. At the same time, use point cloud processing algorithms to extract lane features from LiDAR data and match them with the lane features extracted by the camera to achieve preliminary alignment of LiDAR and camera data.

[0109] Calibration parameter optimization: Based on the matching results, the optimization algorithm is used to optimize the calibration parameters of the lidar and camera, including rotation matrix, translation vector, etc., so that the data collected by the lidar and camera can more accurately reflect the lane line information in the high-precision map;

[0110] Calibration result verification and feedback: Apply the optimized calibration parameters to the data fusion of the LiDAR and the camera, display the calibration results through the HMI interface, and compare and verify with the true value of the lane line in the HD map. At the same time, users can also fine-tune the calibration parameters according to the verification results until a satisfactory calibration effect is achieved;

[0111] Upload calibration data to the cloud: Upload calibration results and related data (such as HD map lane information, raw data collected by LiDAR and cameras, optimized calibration parameters, etc.) to the cloud server for storage, analysis and further optimization. The cloud server can use its powerful computing power to process and analyze large amounts of calibration data, extracting valuable information to improve the perception accuracy and safety of the autonomous driving system.

[0112] Cloud data analysis and optimization: On the cloud server, machine learning or deep learning algorithms are used to further analyze and optimize the uploaded calibration data. By analyzing the distribution, characteristics and other information of the calibration data, the key factors affecting the calibration accuracy can be extracted, and the calibration algorithm can be improved and optimized accordingly. At the same time, the cloud server can also summarize and analyze the calibration data of multiple vehicles to form a global calibration database, providing data support for further optimization of the autonomous driving system.

[0113] Calibration result feedback and update: The cloud server can feed back the optimized calibration results and related data to the user so that the user can update the vehicle calibration parameters and algorithms in a timely manner. This can ensure that the autonomous driving vehicle always maintains high accuracy and reliability during actual driving.

[0114] In this embodiment, the present invention provides a computer-readable storage medium, which stores a computer program programmed or configured to execute any one of the high-precision map-based radar and camera deep fusion calibration methods.

[0115] Any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0116] In summary, the present invention not only uses the true value of the lane line in the high-precision map as a calibration reference, thereby improving the accuracy and reliability of the calibration, but also realizes the automatic online calibration of the deep fusion of the lidar and the camera, thereby improving the perception accuracy and safety of the autonomous driving system.

[0117] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A radar and camera deep fusion calibration method based on high-precision maps, characterized in that: The method comprises: M1. When a vehicle is driving on a road, it obtains lane line data information in real time based on a high-precision map, obtains road point cloud data information in real time based on a vehicle-mounted laser radar, and obtains road image data information in real time based on a vehicle-mounted camera; M2. Based on the image data information of the road, the lane line image feature extraction algorithm based on the Sobel operator and the Roberts operator is used to extract the image features of the lane line to obtain the image feature data information of the lane line, and the image feature data information of the lane line is matched with the data information of the lane line to obtain the matched image feature data information of the lane line; M3. Based on the point cloud data information of the road, the point cloud features of the lane line are extracted using an improved point cloud feature extraction algorithm based on deep learning to obtain data information of the point cloud features of the lane line; M4. Based on the point cloud feature data information of the lane line and the image feature data information of the matched lane line, a deep fusion calibration algorithm of point cloud and image based on gradient neural network optimization is used to jointly calibrate the laser radar and camera to obtain data information of the joint calibration parameters of the laser radar and camera.

2. The radar and camera deep fusion calibration method based on high-precision map according to claim 1 is characterized in that: In step M2, the extracting of the image features of the lane line using the lane line image feature extraction algorithm based on the Sobel operator and the Roberts operator includes: M21. Based on the image data information of the road, a gradient function Q of the pixel points in the image is established, Where r is the image data information of the road, α1, α2 and α3 are the weight coefficients of the pixels in the image, and the gradient value of each pixel in the image is calculated to obtain the data information of the gradient value of the pixel of the road image; M22. Based on the data information of the gradient value of the pixel points of the road image, establish the image feature extraction function W of the lane line, Among them, x is the data information of the gradient value of the pixel point of the road image, f is the edge feature extraction function of the Sobel operator, g is the edge feature extraction function of the Roberts operator, β1, β2 and β3 are the image feature extraction factors of the lane line; M23. Based on the lane line image feature extraction function W, the image features of the lane line are extracted to obtain the image feature data information of the lane line.

3. The radar and camera deep fusion calibration method based on high-precision map according to claim 2 is characterized by: The Sobel operator edge feature extraction function f is: The Roberts operator edge feature extraction function g is: Wherein, x is the data information of the gradient value of the pixel point of the road image.

4. The radar and camera deep fusion calibration method based on high-precision map according to claim 2 is characterized by: The image feature extraction factors β1, β2 and β3 of the lane line are: Wherein, x is the data information of the gradient value of the pixel point of the road image.

5. The radar and camera deep fusion calibration method based on high-precision map according to claim 1 is characterized in that: In step M3, extracting the point cloud features of the lane line using the improved point cloud feature extraction algorithm based on deep learning includes: M31. Based on the point cloud data information of the road, construct a point cloud data set of the road; M32. Input the point cloud data set of the road into the point cloud feature extraction model of deep learning for training and learning, and determine the feature extraction function R of the model, Among them, y is the point cloud dataset of the road, δ1, δ2 and δ3 are the feature reward factors of the point cloud of the road, and the trained deep learning point cloud feature extraction model is obtained; M33. Based on the trained deep learning point cloud feature extraction model, the point cloud dataset of the road is input, the point cloud features of the lane line are extracted, and the data information of the point cloud features of the lane line is obtained.

6. The radar and camera depth fusion calibration method based on high-precision maps according to claim 5 is characterized by: The constraints of the feature reward factors δ1, δ2 and δ3 of the road point cloud are:

7. The radar and camera depth fusion calibration method based on high-precision map according to claim 1 is characterized in that: In step M4, the laser radar and camera are jointly calibrated using a point cloud and image deep fusion calibration algorithm based on gradient neural network optimization, including: M41. Based on the point cloud feature data information of the lane line and the image feature data information of the matched lane line, a multimodal fusion function P of the lane line is established. Among them, z1 is the point cloud feature data information of the lane line, z2 is the image feature data information of the matched lane line, γ1, γ2 and γ3 are the multimodal fusion factors of the lane line, and the features of the point cloud and image of the lane line are fused to obtain the fused feature data information of the lane line; M42. Based on the characteristic data information of the fused lane line, a joint calibration function S of the laser radar and the camera is established. Wherein, h is the characteristic data information of the fused lane line, η1, η2 and η3 are the joint calibration factors of the laser radar and the camera, and the joint calibration factors of the laser radar and the camera are input into the gradient neural network for optimization; M43. Based on the joint calibration function S of the laser radar and the camera, the laser radar and the camera are jointly calibrated to obtain data information of the joint calibration parameters of the laser radar and the camera.

8. The radar and camera deep fusion calibration method based on high-precision map according to claim 7 is characterized by: The inputting the joint calibration factor of the laser radar and the camera into the gradient neural network for optimization is to input the joint calibration factor of the laser radar and the camera into the gradient neural network for training and learning, and construct an optimization function G of the network, Among them, q is the joint calibration factor of the lidar and the camera, λ1, λ2 and λ3 are optimization factors, and the joint calibration factor of the lidar and the camera is optimized to obtain the optimized joint calibration factor of the lidar and the camera.

9. A radar and camera deep fusion calibration system based on high-precision maps, including a computer device, characterized in that: The computer device is programmed or configured to execute the steps of the radar and camera deep fusion calibration method based on high-precision maps as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that is programmed or configured to execute the radar and camera deep fusion calibration method based on high-precision maps as described in any one of claims 1 to 8.

Images