A method and system for joint external parameter calibration of vehicle lidar and camera

Abstract

This invention discloses a method and system for joint extrinsic parameter calibration of vehicle LiDAR and camera. By acquiring LiDAR point cloud information, video information, radar point cloud information, differential positioning information, navigation information, and vehicle attitude information during the target vehicle's operation, the method performs signal-level time synchronization and information alignment based on the gPTP protocol, as well as information type fusion, attribute fusion, topology fusion, and logical reasoning to obtain target parameter information. Finally, the target parameter information is used as a truth control group and compared with the initial extrinsic parameter calibration parameters of the target vehicle to filter out invalid values ​​in the initial extrinsic parameter calibration parameters, thus obtaining the joint extrinsic parameter calibration parameters. This invention enables joint extrinsic parameter calibration of vehicle LiDAR and camera during driving. When the position of intelligent driving sensors on the target vehicle shifts, offset compensation can be performed through seamless real-time extrinsic parameter calibration, thereby improving the user's driving experience.

Description

Technical Field

[0001] This invention relates to the field of vehicle calibration technology, specifically to a method and system for calibrating the combined external parameters of a vehicle's lidar and camera. Background Technology

[0002] At present, before a vehicle leaves the factory or before the intelligent assisted driving function of a smart car is used for the first time, it is necessary to calibrate the external parameters of the sensors related to intelligent assisted driving.

[0003] During vehicle use, factors such as vehicle aging and collisions can cause discrepancies between the actual external parameter calibration parameters of the intelligent driving sensors and the initial external parameter calibration parameters. This can lead to a decline in the performance of intelligent assisted driving functions or render the functions unusable. Since users cannot perform external parameter calibration of the intelligent driving sensors, they need to go to a repair shop for vehicle external parameter calibration.

[0004] Therefore, there is still a problem that the actual external parameter calibration parameters of the vehicle differ from the initial external parameter calibration parameters due to objective factors, resulting in a decrease in the performance of intelligent assisted driving functions or the unavailability of the functions. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for joint external parameter calibration of vehicle lidar and camera, which aims to solve the problem that in the prior art, the actual external parameter calibration parameters of the vehicle differ from the initial external parameter calibration parameters due to objective factors, resulting in a decline in the performance of intelligent assisted driving functions or the unavailability of functions.

[0006] A first aspect of the present invention provides a method for joint extrinsic parameter calibration of a vehicle lidar and a camera, the method comprising:

[0007] During the driving process of the target vehicle, laser point cloud information of the target road surface is acquired by lidar.

[0008] The camera acquires video information of the target road surface.

[0009] The radar point cloud information of the target road surface is obtained by forward millimeter-wave radar;

[0010] The carrier phase collected by the base station is sent to the on-board receiver of the target vehicle through real-time dynamic carrier phase differential, and the coordinates are calculated by difference to obtain differential positioning information.

[0011] The navigation information of the target vehicle is obtained through a navigation satellite system, and the navigation information is sent to the vehicle-mounted receiver.

[0012] The vehicle attitude information of the target vehicle is obtained through an inertial measurement unit, and the vehicle attitude information is sent to the on-board receiver.

[0013] Based on the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle attitude information, time synchronization and information alignment at the signal level are performed based on the gPTP protocol, and visual perception processing, signal classification and signal fusion processing are performed to obtain multiple lane elements.

[0014] By performing information type fusion, attribute fusion, topology fusion, and logical reasoning on multiple lane elements, target parameter information is obtained.

[0015] The target parameter information is compared with the initial extrinsic calibration parameters of the target vehicle. Invalid values ​​in the initial extrinsic calibration parameters are screened out using the target parameter information as a true value control group, and the joint extrinsic calibration parameters are output.

[0016] According to one aspect of the above technical solution, the steps of performing signal-level time synchronization and information alignment based on the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information, and vehicle attitude information using the gPTP protocol include:

[0017] The laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information, and vehicle attitude information are compared with the corresponding standard reference values ​​according to the time of transmission.

[0018] The difference in the comparison is fed back synchronously so that subsequent systems that acquire the above information can obtain more accurate information.

[0019] According to one aspect of the above technical solution, various lane elements include lane line type, lane line width, lane line color, lane line fork point, lane line connection point, lane line merging point, lane line convergence point, lane line appearance point, lane line disappearance point, lane line curvature, lane line longitude, lane line dimension, lane line grade, lane line length, and road speed limit.

[0020] According to one aspect of the above technical solution, the steps of fusing information types, attributes, topology, and performing logical reasoning on multiple lane elements to obtain target parameter information include:

[0021] All lane elements are classified using an information type fusion method to determine lane line information, position information, and vehicle attitude information;

[0022] The parameter information of all lane elements is determined by attribute fusion and converted according to a unified unit;

[0023] The information is collected in segments using a topology fusion method, and multiple segments of sub-information are merged into a complete segment of topology information.

[0024] It also uses logical reasoning to determine the validity of all lane elements and removes invalid information.

[0025] According to one aspect of the above technical solution, the method further includes:

[0026] After each power-on of the target vehicle, determine whether there are joint external parameter calibration parameters for the lidar and camera.

[0027] If so, the joint extrinsic parameter calibration parameters are recorded and stored, and the vehicle travel time of the target vehicle from the current time point to the time when the joint extrinsic parameter calibration parameters were generated is detected;

[0028] When the vehicle travel time exceeds a preset time threshold, real-time joint external parameter dynamic calibration of LiDAR and camera is performed.

[0029] If the real-time joint extrinsic parameter calibration parameters are obtained, and there are multiple consecutive differences from the initial extrinsic parameter calibration parameters when the vehicle is powered on, and the vehicle's driving time exceeds a preset time threshold, then the initial extrinsic parameter calibration parameters are replaced according to the joint extrinsic parameter calibration parameters.

[0030] According to one aspect of the above technical solution, if real-time joint extrinsic parameter calibration parameters are obtained, and there are consecutive differences between these parameters and the initial extrinsic parameter calibration parameters when the vehicle is powered on, and the vehicle's driving time exceeds a preset time threshold, then the step of replacing the initial extrinsic parameter calibration parameters with the joint extrinsic parameter calibration parameters includes:

[0031] If the joint extrinsic parameter calibration parameter differs from the initial extrinsic parameter calibration parameter when the vehicle is powered on for 5 consecutive times, and the vehicle driving time exceeds a preset time threshold, then the initial extrinsic parameter calibration parameter is replaced according to the joint extrinsic parameter calibration parameter.

[0032] A second aspect of the present invention is to provide a joint extrinsic parameter calibration system for a vehicle lidar and camera, applied to the method described in the above-mentioned technical solution, the system comprising:

[0033] The first information acquisition module is used to acquire laser point cloud information of the target road surface through lidar during the driving process of the target vehicle.

[0034] The second information acquisition module is used to acquire video information of the target road surface through the camera;

[0035] The third information acquisition module is used to acquire radar point cloud information of the target road surface through forward millimeter-wave radar;

[0036] The fourth information acquisition module is used to send the carrier phase collected by the reference station to the on-board receiver of the target vehicle through real-time dynamic carrier phase differential, perform differential calculation to obtain differential positioning information;

[0037] The fifth information acquisition module is used to acquire the navigation information of the target vehicle through the navigation satellite system and send the navigation information to the vehicle-mounted receiver;

[0038] The sixth information acquisition module is used to acquire the vehicle attitude information of the target vehicle through the inertial measurement unit and send the vehicle attitude information to the vehicle receiver.

[0039] The first information processing module is used to perform time synchronization and information alignment at the signal level based on the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle attitude information, and to perform visual perception processing, signal classification and signal fusion processing to obtain multiple lane elements.

[0040] The second information processing module is used to perform information type fusion, attribute fusion, topology fusion and logical reasoning on multiple lane elements to obtain target parameter information.

[0041] The calibration parameter output module is used to compare the target parameter information with the initial extrinsic calibration parameters of the target vehicle, use the target parameter information as a true value control group to screen out invalid values ​​in the initial extrinsic calibration parameters, and output the joint extrinsic calibration parameters.

[0042] According to one aspect of the above technical solution, various lane elements include lane line type, lane line width, lane line color, lane line fork point, lane line connection point, lane line merging point, lane line convergence point, lane line appearance point, lane line disappearance point, lane line curvature, lane line longitude, lane line dimension, lane line grade, lane line length, and road speed limit.

[0043] A third aspect of the present invention is to provide a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in the above-described technical solutions.

[0044] A third aspect of the present invention is to provide an automobile, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the methods described in the above-described technical solutions.

[0045] Compared with existing technologies, the advantages of the vehicle lidar and camera combined extrinsic parameter calibration method and system shown in this invention are as follows:

[0046] The method described in this invention acquires laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information, and vehicle attitude information during the target vehicle's movement. Then, based on the gPTP protocol, it performs signal-level time synchronization and information alignment, as well as information type fusion, attribute fusion, topology fusion, and logical reasoning to obtain target parameter information. Finally, it compares this target parameter information with the initial extrinsic parameter calibration parameters of the target vehicle as a truth control group, filtering out invalid values ​​from the initial extrinsic parameter calibration parameters, thus outputting real-time joint extrinsic parameter calibration parameters. This method enables joint extrinsic parameter calibration of the vehicle's LiDAR and cameras during the target vehicle's movement. When the position of the intelligent driving sensors on the target vehicle shifts, offset compensation can be performed seamlessly through real-time extrinsic parameter calibration, thereby improving the user's driving experience. Attached Figure Description

[0047] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0048] Figure 1 This is a flowchart illustrating a method for joint external parameter calibration of vehicle lidar and camera in one embodiment of the present invention.

[0049] Figure 2 This is a structural block diagram of a vehicle lidar and camera combined extrinsic parameter calibration system according to an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Several embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of the present invention will be more thorough and complete.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0052] Example 1

[0053] Please see Figure 1 The first embodiment of the present invention provides a method for joint extrinsic parameter calibration of a vehicle lidar and a camera, the method comprising steps S10-S90:

[0054] Step S10: During the driving process of the target vehicle, the laser point cloud information of the target road surface is acquired by the lidar.

[0055] The target vehicle is an intelligent car equipped with intelligent driving sensors, including LiDAR, cameras, and millimeter-wave radar, which acquire environmental information through intelligent driving sensors and then make driving decisions based on the environmental information, thereby outputting driving strategy information, such as controlling the target vehicle to follow other vehicles or avoid obstacles.

[0056] Specifically, the intelligent driving sensors in the aforementioned radar system include lidar and millimeter-wave radar, with the millimeter-wave radar typically being a forward-facing millimeter-wave radar to acquire environmental information in front of the vehicle. During the vehicle's movement, lidar acquires and stores laser point cloud information of the road surface the vehicle has already traversed, in the form of laser point clouds.

[0057] Step S20: Acquire video information of the target road surface through the camera.

[0058] Similarly, during the driving of the target vehicle, video information of the target road surface will be acquired in the form of video signals through the camera of the camera system. The video information records the real scene that occurs on the target road surface.

[0059] Step S30: Obtain radar point cloud information of the target road surface using forward millimeter-wave radar.

[0060] Similarly, during the movement of the target vehicle, the radar system's forward-facing millimeter-wave radar will also acquire radar point cloud information of the target road surface in the form of radar point cloud.

[0061] Step S40: The carrier phase collected by the base station is sent to the on-board receiver of the target vehicle through real-time dynamic carrier phase differential, and the coordinates are calculated by difference to obtain differential positioning information.

[0062] In this embodiment, the carrier phase collected by the base station is sent to the on-board receiver of the target vehicle where the user is located through RTK (Real-time kinematic). Then, the coordinates are calculated based on the difference of the carrier phase to obtain differential positioning information.

[0063] Step S50: Obtain the navigation information of the target vehicle through the navigation satellite system and send the navigation information to the vehicle-mounted receiver.

[0064] In this embodiment, the navigation information of the target vehicle is obtained through GNSS (Global Navigation Satellite System), including the target vehicle's three-dimensional coordinates, real-time speed, and time information, and then the navigation information is sent to the on-board receiver of the target vehicle where the user is located.

[0065] Step S60: Obtain the vehicle attitude information of the target vehicle through the inertial measurement unit, and send the vehicle attitude information to the vehicle-mounted receiver.

[0066] In this embodiment, the vehicle attitude information of the target vehicle during driving is obtained through an IMU (Inertial Measurement Unit), and then the vehicle attitude information is sent to the on-board receiver of the target vehicle where the user is located.

[0067] The above provides information on laser point cloud, video, radar point cloud, differential positioning, navigation, and vehicle attitude.

[0068] Step S70: Based on the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle attitude information, time synchronization and information alignment at the signal level are performed according to the gPTP protocol, and visual perception processing, signal classification and signal fusion processing are performed to obtain multiple lane elements.

[0069] In this embodiment, the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle attitude information generated in steps S10-S60 are synchronized at the signal level using the gPTP protocol (generalized Precision Time Protocol, which is an Ethernet-based time synchronization protocol defined by the IEEE 802.1AS standard).

[0070] The steps of performing signal-level time synchronization and information alignment based on the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information, and vehicle attitude information using the gPTP protocol include:

[0071] The laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information, and vehicle attitude information are compared with the corresponding standard reference values ​​according to the time of transmission.

[0072] The difference in the comparison is fed back synchronously so that subsequent systems that acquire the above information can obtain more accurate information.

[0073] Specifically, the goal of time synchronization is to compare all signals with a standard reference value according to their transmission time, and to synchronously feed back the difference in the comparison so that subsequent systems using these signals can obtain accurate information.

[0074] Specifically, based on the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information, and vehicle attitude information generated in steps S10-S60 above, visual perception processing, signal classification, and signal fusion processing are performed on the above information to obtain various lane elements.

[0075] Specifically, various lane elements include lane line type, lane line width, lane line color, lane line fork point, lane line connection point, lane line merging point, lane line convergence point, lane line appearance point, lane line disappearance point, lane line curvature, lane line longitude, lane line dimension, lane line level, lane line length, and road speed limit.

[0076] Step S80: Information type fusion, attribute fusion, topology fusion and logical reasoning are performed on multiple lane elements to obtain target parameter information.

[0077] In this embodiment, the steps of fusing information types, attributes, topology, and logical reasoning from multiple lane elements to obtain target parameter information include:

[0078] All lane elements are classified using an information type fusion method to determine lane line information, position information, and vehicle attitude information;

[0079] The parameter information of all lane elements is determined by attribute fusion and converted according to a unified unit;

[0080] The information is collected in segments using a topology fusion method, and multiple segments of sub-information are merged into a complete segment of topology information.

[0081] It also uses logical reasoning to determine the validity of all lane elements and removes invalid information.

[0082] Specifically, the various lane elements generated in step S70 are fused using information type fusion to determine which are lane line information, which are position information, and which are vehicle attitude information. Then, attribute fusion is performed to clarify which parameters are included in each specific information type and to convert them using a unified unit. Next, topology fusion is performed, merging multiple segments of information into a single segment of topological information through segmented information collection. Finally, logical reasoning is performed to determine which information is valid and which is invalid, and invalid information is removed. Through the above information type fusion, attribute fusion, topology fusion, and logical reasoning, the target parameter information is obtained.

[0083] Step S90: Compare the target parameter information with the initial extrinsic calibration parameters of the target vehicle, use the target parameter information as a true value control group to screen out invalid values ​​in the initial extrinsic calibration parameters, and output the joint extrinsic calibration parameters.

[0084] Specifically, the target parameter information generated in step S90 is compared with the initial extrinsic calibration parameters of the target vehicle. The target parameter information is used as a true value control group to filter out invalid values ​​from the initial extrinsic calibration parameters, and finally the real-time joint extrinsic calibration parameters are output.

[0085] Compared with existing technologies, the combined extrinsic parameter calibration method for vehicle lidar and camera shown in this embodiment has the following advantages:

[0086] The method described in this embodiment acquires laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information, and vehicle attitude information during the target vehicle's movement. Then, based on the gPTP protocol, it performs signal-level time synchronization and information alignment, as well as information type fusion, attribute fusion, topology fusion, and logical reasoning to obtain target parameter information. Finally, it compares the target parameter information with the initial extrinsic parameter calibration parameters of the target vehicle as a truth control group, filtering out invalid values ​​from the initial extrinsic parameter calibration parameters, and outputting real-time joint extrinsic parameter calibration parameters. This method can achieve joint extrinsic parameter calibration of the vehicle's LiDAR and camera during the target vehicle's movement. When the position of the intelligent driving sensors on the target vehicle shifts, offset compensation can be performed through seamless real-time extrinsic parameter calibration, thereby improving the user's driving experience.

[0087] Example 2

[0088] The second embodiment of the present invention also provides a joint extrinsic parameter calibration method for vehicle lidar and camera. The joint extrinsic parameter calibration method shown in this embodiment is basically the same as the joint extrinsic parameter calibration method shown in the first embodiment, except that:

[0089] In this embodiment, the method further includes:

[0090] After each power-on of the target vehicle, determine whether there are joint external parameter calibration parameters for the lidar and camera.

[0091] If so, the joint extrinsic parameter calibration parameters are recorded and stored, and the vehicle travel time of the target vehicle from the current time point to the time when the joint extrinsic parameter calibration parameters were generated is detected;

[0092] When the vehicle travel time exceeds a preset time threshold, real-time joint external parameter dynamic calibration of LiDAR and camera is performed.

[0093] If the real-time joint extrinsic parameter calibration parameters are obtained, and there are multiple consecutive differences from the initial extrinsic parameter calibration parameters when the vehicle is powered on, and the vehicle's driving time exceeds a preset time threshold, then the initial extrinsic parameter calibration parameters are replaced according to the joint extrinsic parameter calibration parameters.

[0094] The step of replacing the initial extrinsic calibration parameters with the joint extrinsic calibration parameters based on the joint extrinsic calibration parameters, if the real-time joint extrinsic calibration parameters are obtained and there are multiple consecutive differences from the initial extrinsic calibration parameters when the vehicle is powered on, and the vehicle's driving time exceeds a preset time threshold, includes:

[0095] If the joint extrinsic parameter calibration parameter differs from the initial extrinsic parameter calibration parameter when the vehicle is powered on for 5 consecutive times, and the vehicle driving time exceeds a preset time threshold, then the initial extrinsic parameter calibration parameter is replaced according to the joint extrinsic parameter calibration parameter.

[0096] Specifically, the method shown in this embodiment first determines whether there are joint extrinsic parameter calibration parameters for the LiDAR and camera after each power-on (start-up) of the target vehicle. That is, the joint extrinsic parameter calibration parameters generated by the user during driving. If such parameters exist, the previously generated joint extrinsic parameter calibration parameters are recorded and stored. The method also detects the vehicle's driving time from the current time point to the time when the joint extrinsic parameter calibration parameters were generated. This time can be a natural duration, such as a number of days, or the actual driving time of the target vehicle, accurate to a number of hours and minutes. Then, it determines whether the vehicle's driving time is greater than a preset time threshold. If it is greater than the preset time threshold, real-time joint extrinsic parameter dynamic calibration of the LiDAR and camera is performed. If the real-time joint extrinsic parameter calibration parameters are obtained, and there are differences from the initial extrinsic parameter calibration parameters when the vehicle is powered on for 5 consecutive times, and the vehicle's driving time is greater than the preset time threshold, then the initial extrinsic parameter calibration parameters are replaced according to the joint extrinsic parameter calibration parameters.

[0097] In other words, in this embodiment, after each power-on of the target vehicle, a condition for whether to perform joint external parameter calibration will be determined, thereby ensuring that joint external parameter calibration is performed when actually needed.

[0098] Example 3

[0099] Please see Figure 2 A second embodiment of the present invention provides a joint extrinsic parameter calibration system for vehicle lidar and camera, applied to the method described in any of the above embodiments, the system comprising:

[0100] The first information acquisition module is used to acquire laser point cloud information of the target road surface through lidar during the driving process of the target vehicle.

[0101] The second information acquisition module is used to acquire video information of the target road surface through the camera;

[0102] The third information acquisition module is used to acquire radar point cloud information of the target road surface through forward millimeter-wave radar;

[0103] The fourth information acquisition module is used to send the carrier phase collected by the reference station to the on-board receiver of the target vehicle through real-time dynamic carrier phase differential, perform differential calculation to obtain differential positioning information;

[0104] The fifth information acquisition module is used to acquire the navigation information of the target vehicle through the navigation satellite system and send the navigation information to the vehicle-mounted receiver;

[0105] The sixth information acquisition module is used to acquire the vehicle attitude information of the target vehicle through the inertial measurement unit and send the vehicle attitude information to the vehicle receiver.

[0106] The first information processing module is used to perform time synchronization and information alignment at the signal level based on the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle attitude information, and to perform visual perception processing, signal classification and signal fusion processing to obtain multiple lane elements.

[0107] The second information processing module is used to perform information type fusion, attribute fusion, topology fusion and logical reasoning on multiple lane elements to obtain target parameter information.

[0108] The calibration parameter output module is used to compare the target parameter information with the initial extrinsic calibration parameters of the target vehicle, use the target parameter information as a true value control group to screen out invalid values ​​in the initial extrinsic calibration parameters, and output the joint extrinsic calibration parameters.

[0109] Specifically, various lane elements include lane line type, lane line width, lane line color, lane line fork point, lane line connection point, lane line merging point, lane line convergence point, lane line appearance point, lane line disappearance point, lane line curvature, lane line longitude, lane line dimension, lane line level, lane line length, and road speed limit.

[0110] Compared with existing technologies, the combined extrinsic parameter calibration system of vehicle lidar and camera shown in this embodiment has the following advantages:

[0111] The system described in this embodiment acquires laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information, and vehicle attitude information during the target vehicle's movement. Then, based on the gPTP protocol, it performs signal-level time synchronization and information alignment, as well as information type fusion, attribute fusion, topology fusion, and logical reasoning to obtain target parameter information. Finally, it compares the target parameter information with the initial extrinsic parameter calibration parameters of the target vehicle as a truth control group, filtering out invalid values ​​from the initial extrinsic parameter calibration parameters, and outputting real-time joint extrinsic parameter calibration parameters. This system can achieve joint extrinsic parameter calibration of the vehicle's LiDAR and camera while the target vehicle is in motion. When the position of the intelligent driving sensors on the target vehicle shifts, offset compensation can be performed through seamless real-time extrinsic parameter calibration, thereby improving the user's driving experience.

[0112] Example 4

[0113] A fourth embodiment of the present invention provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in the above embodiments.

[0114] Example 5

[0115] A fifth embodiment of the present invention provides an automobile, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the methods described in the above embodiments.

[0116] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0117] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A method for joint extrinsic parameter calibration of a vehicle's lidar and camera, characterized in that, The method includes: During the driving process of the target vehicle, laser point cloud information of the target road surface is acquired by lidar. The camera acquires video information of the target road surface. The radar point cloud information of the target road surface is obtained by forward millimeter-wave radar; The carrier phase collected by the base station is sent to the on-board receiver of the target vehicle through real-time dynamic carrier phase differential, and the coordinates are calculated by difference to obtain differential positioning information. The navigation information of the target vehicle is obtained through a navigation satellite system, and the navigation information is sent to the vehicle-mounted receiver. The vehicle attitude information of the target vehicle is obtained through an inertial measurement unit, and the vehicle attitude information is sent to the on-board receiver. Based on the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle attitude information, time synchronization and information alignment at the signal level are performed based on the gPTP protocol, and visual perception processing, signal classification and signal fusion processing are performed to obtain multiple lane elements. By performing information type fusion, attribute fusion, topology fusion, and logical reasoning on multiple lane elements, target parameter information is obtained. The target parameter information is compared with the initial extrinsic calibration parameters of the target vehicle. Invalid values ​​in the initial extrinsic calibration parameters are screened out using the target parameter information as a truth control group, and the joint extrinsic calibration parameters are output. Based on the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information, and vehicle attitude information, the steps for signal-level time synchronization and information alignment based on the gPTP protocol include: The laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information, and vehicle attitude information are compared with the corresponding standard reference values ​​according to the time of transmission. The comparison difference will be fed back synchronously so that subsequent systems that acquire the above information will obtain more accurate information. Various lane elements include lane line type, lane line width, lane line color, lane line fork point, lane line connection point, lane line merging point, lane line convergence point, lane line appearance point, lane line disappearance point, lane line curvature, lane line longitude, lane line dimension, lane line level, lane line length, and road speed limit; The steps for fusing information types, attributes, topology, and performing logical reasoning on multiple lane elements to obtain target parameter information include: All lane elements are classified using an information type fusion method to determine lane line information, position information, and vehicle attitude information; The parameter information of all lane elements is determined by attribute fusion and converted according to a unified unit; The information is collected in segments using a topology fusion method, and multiple segments of sub-information are merged into a complete segment of topology information. It also uses logical reasoning to determine the validity of all lane elements and removes invalid information.

2. The method for joint extrinsic parameter calibration of vehicle lidar and camera according to claim 1, characterized in that, The method further includes: After each power-on of the target vehicle, determine whether there are joint external parameter calibration parameters for the lidar and camera. If so, the joint extrinsic parameter calibration parameters are recorded and stored, and the vehicle travel time of the target vehicle from the current time point to the time when the joint extrinsic parameter calibration parameters were generated is detected; When the vehicle travel time exceeds a preset time threshold, real-time joint external parameter dynamic calibration of LiDAR and camera is performed. If the real-time joint extrinsic parameter calibration parameters are obtained, and there are multiple consecutive differences from the initial extrinsic parameter calibration parameters when the vehicle is powered on, and the vehicle's driving time exceeds a preset time threshold, then the initial extrinsic parameter calibration parameters are replaced according to the joint extrinsic parameter calibration parameters.

3. The method for joint extrinsic parameter calibration of vehicle lidar and camera according to claim 2, characterized in that, If the real-time joint extrinsic parameter calibration parameters are obtained, and there are multiple consecutive differences between these and the initial extrinsic parameter calibration parameters when the vehicle is powered on, and the vehicle's driving time exceeds a preset time threshold, then the step of replacing the initial extrinsic parameter calibration parameters according to the joint extrinsic parameter calibration parameters includes: If the joint extrinsic parameter calibration parameter differs from the initial extrinsic parameter calibration parameter when the vehicle is powered on for 5 consecutive times, and the vehicle driving time exceeds a preset time threshold, then the initial extrinsic parameter calibration parameter is replaced according to the joint extrinsic parameter calibration parameter.

4. A joint extrinsic parameter calibration system for vehicle lidar and camera, characterized in that, The system, applicable to the method of any one of claims 1-3, comprises: The first information acquisition module is used to acquire laser point cloud information of the target road surface through lidar during the driving process of the target vehicle. The second information acquisition module is used to acquire video information of the target road surface through the camera; The third information acquisition module is used to acquire radar point cloud information of the target road surface through forward millimeter-wave radar; The fourth information acquisition module is used to send the carrier phase collected by the reference station to the on-board receiver of the target vehicle through real-time dynamic carrier phase differential, perform differential calculation to obtain differential positioning information; The fifth information acquisition module is used to acquire the navigation information of the target vehicle through the navigation satellite system and send the navigation information to the vehicle-mounted receiver; The sixth information acquisition module is used to acquire the vehicle attitude information of the target vehicle through the inertial measurement unit and send the vehicle attitude information to the vehicle receiver. The first information processing module is used to perform time synchronization and information alignment at the signal level based on the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle attitude information, and to perform visual perception processing, signal classification and signal fusion processing to obtain multiple lane elements. The second information processing module is used to perform information type fusion, attribute fusion, topology fusion and logical reasoning on multiple lane elements to obtain target parameter information. The calibration parameter output module is used to compare the target parameter information with the initial extrinsic calibration parameters of the target vehicle, use the target parameter information as a true value control group to screen out invalid values ​​in the initial extrinsic calibration parameters, and output the joint extrinsic calibration parameters.

5. A readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 3.

6. A vehicle comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 3.

Images