Vehicle laser radar and camera combined external parameter calibration method and system

Abstract

The invention discloses a vehicle laser radar and camera combined external parameter calibration method and system, and the method comprises the steps: obtaining laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle attitude information in the driving process of a target vehicle; the information is subjected to time synchronization and information alignment of a signal level based on a gPTP protocol, information type fusion, attribute fusion, topology fusion and logical reasoning are carried out, target parameter information is obtained, finally, the target parameter information serves as a true value control group to be compared with initial external parameter calibration parameters of a target vehicle, and the target parameter information is obtained. Invalid values in the initial external parameter calibration parameters are screened out, and the combined external parameter calibration parameters are obtained. According to the method, the combined external parameter calibration of the vehicle laser radar and the camera can be realized in the driving process, and when the position of the intelligent driving sensor on the target vehicle deviates, deviation compensation can be carried out through non-inductive real-time external parameter calibration, so that the vehicle use experience of a user is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle calibration, and in particular to a vehicle laser radar and camera combined external parameter calibration method and system. Background Art

[0002] At present, before a vehicle leaves the factory or the intelligent assisted driving function of a smart car is used for the first time, it is necessary to perform external parameter calibration on the sensors related to intelligent assisted driving.

[0003] During the user's use of the vehicle, there are factors such as vehicle aging and vehicle collision, which will cause a certain difference between the actual external parameter calibration parameters of the intelligent driving sensor and the initial external parameter calibration parameters, thereby causing the performance of the intelligent assisted driving function to decline or the function to be unavailable. The user is unable to perform the external parameter calibration of the intelligent driving sensor and needs to go to a maintenance organization to perform vehicle external parameter calibration.

[0004] Therefore, there is still a problem that due to objective factors, the actual extrinsic calibration parameters of the vehicle differ from the initial extrinsic calibration parameters, resulting in a decline in the performance of the intelligent assisted driving function or the function being unavailable. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method and system for joint external parameter calibration of vehicle lidar and camera, aiming to solve the problem in the prior art that due to objective factors, the actual external parameter calibration parameters of the vehicle are different from the initial external parameter calibration parameters, resulting in the performance of the intelligent assisted driving function being reduced or the function being unavailable.

[0006] A first aspect of the present invention provides a joint extrinsic parameter calibration method for a vehicle laser radar and a camera, the method comprising:

[0007] During the driving process of the target vehicle, the laser point cloud information of the target road surface is obtained through the laser radar;

[0008] Acquiring video information of the target road surface through the camera;

[0009] Acquiring radar point cloud information of the target road surface by using a forward millimeter-wave radar;

[0010] The carrier phase collected by the reference station is sent to the on-board receiver of the target vehicle through real-time dynamic carrier phase difference, and the coordinates are calculated to obtain differential positioning information;

[0011] Acquire navigation information of the target vehicle through a navigation satellite system, and send the navigation information to the vehicle-mounted receiver;

[0012] Acquire vehicle posture information of the target vehicle through an inertial measurement unit, and send the vehicle posture information to the vehicle-mounted receiver;

[0013] According to the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information, time synchronization and information alignment at the signal level are performed based on the gPTP protocol, and visual perception processing, signal classification and signal fusion processing are performed to obtain multiple lane elements;

[0014] Perform information type fusion, attribute fusion, topology fusion and logical reasoning on multiple lane elements to obtain target parameter information;

[0015] The target parameter information is compared with the initial external parameter calibration parameters of the target vehicle, invalid values ​​in the initial external parameter calibration parameters are screened out using the target parameter information as a true value control group, and a combined external parameter calibration parameter is output.

[0016] According to one aspect of the above technical solution, the steps of performing time synchronization and information alignment at the signal level based on the gPTP protocol according to the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information include:

[0017] Comparing the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information with corresponding standard reference values ​​according to the time of issuance;

[0018] The comparison difference is synchronously fed back so that the system that subsequently obtains the above information can obtain more accurate information.

[0019] According to one aspect of the above technical solution, the various lane elements include lane line type, lane line width, lane line color, lane line bifurcation point, lane line connection point, lane line merging point, lane line merging point, lane line appearance point, lane line vanishing point, lane line curvature, lane line longitude, lane line dimension, lane line grade, lane line length and road speed limit.

[0020] According to one aspect of the above technical solution, the steps of performing information type fusion, attribute fusion, topology fusion and logical reasoning on multiple lane elements to obtain target parameter information include:

[0021] All lane elements are classified by information type fusion to determine lane line information, position information and vehicle body posture information;

[0022] The attribute fusion method is used to clarify the parameter information of all lane elements and convert them into unified units;

[0023] The information is collected in segments by using topological fusion, and multiple segments of information are fused into a whole segment of topological information;

[0024] And use logical reasoning to judge the validity of all lane elements and eliminate invalid information.

[0025] According to one aspect of the above technical solution, the method further includes:

[0026] After each power-on of the target vehicle, determine whether there are joint external calibration parameters of the laser radar and camera;

[0027] If yes, the joint external parameter calibration parameters are recorded and stored, and the vehicle driving time of the target vehicle from the current time node to the time when the joint external parameter calibration parameters are generated is detected;

[0028] When the driving time of the vehicle is greater than a preset time threshold, real-time joint external parameter dynamic calibration of the laser radar and the camera is performed;

[0029] If real-time joint external parameter calibration parameters are obtained, and there are differences with the initial external parameter calibration parameters when the vehicle is powered on for multiple consecutive times, and the vehicle driving time is greater than the preset time threshold, the initial external parameter calibration parameters are replaced according to the joint external parameter calibration parameters.

[0030] According to one aspect of the above technical solution, if the real-time joint external parameter calibration parameters are obtained, and there are differences with the initial external parameter calibration parameters when the vehicle is powered on for multiple consecutive times, and the driving time of the vehicle is greater than the preset time threshold, the step of replacing the initial external parameter calibration parameters according to the joint external parameter calibration parameters includes:

[0031] When the combined external parameter calibration parameters differ from the initial external parameter calibration parameters when the vehicle is powered on for five consecutive times, and the vehicle driving time is greater than a preset time threshold, the initial external parameter calibration parameters are replaced according to the combined external parameter calibration parameters.

[0032] A second aspect of the present invention is to provide a joint external parameter calibration system of a vehicle laser radar and a camera, which is applied to the method described in the above technical solution, and the system comprises:

[0033] The first information acquisition module is used to acquire laser point cloud information of the target road surface traveled by the target vehicle through the laser radar during the driving process;

[0034] A second information acquisition module, used to acquire video information of the target road surface through the camera;

[0035] A third information acquisition module, used to acquire radar point cloud information of the target road surface through a forward millimeter-wave radar;

[0036] The fourth information acquisition module is used to send the carrier phase collected by the reference station to the on-board receiver of the target vehicle through real-time dynamic carrier phase difference, perform difference calculation on the coordinates, and obtain differential positioning information;

[0037] A fifth information acquisition module, used to acquire navigation information of the target vehicle through a navigation satellite system, and send the navigation information to the vehicle-mounted receiver;

[0038] a sixth information acquisition module, configured to acquire vehicle posture information of the target vehicle through an inertial measurement unit, and send the vehicle posture information to the vehicle-mounted receiver;

[0039] A first information processing module is used to perform time synchronization and information alignment at the signal level based on the gPTP protocol according to the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information, and perform visual perception processing, signal classification and signal fusion processing to obtain multiple lane elements;

[0040] The second information processing module is used to perform information type fusion, attribute fusion, topology fusion and logical reasoning on multiple lane elements to obtain target parameter information;

[0041] The calibration parameter output module is used to compare the target parameter information with the initial external parameter calibration parameters of the target vehicle, screen out invalid values ​​in the initial external parameter calibration parameters using the target parameter information as a true value control group, and output a combined external parameter calibration parameter.

[0042] According to one aspect of the above technical solution, the various lane elements include lane line type, lane line width, lane line color, lane line bifurcation point, lane line connection point, lane line merging point, lane line merging point, lane line appearance point, lane line vanishing point, lane line curvature, lane line longitude, lane line dimension, lane line grade, lane line length and road speed limit.

[0043] A third aspect of the present invention is to provide a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the above technical solution.

[0044] The third aspect of the present invention is to provide a car, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the above technical solution when executing the computer program.

[0045] Compared with the prior art, the vehicle laser radar and camera combined external parameter calibration method and system shown in the present invention have the following beneficial effects:

[0046] The method shown in the present invention obtains laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle posture information during the driving process of the target vehicle, and then performs signal-level time synchronization and information alignment based on the gPTP protocol, as well as information type fusion, attribute fusion, topology fusion and logical reasoning on the above information to obtain target parameter information. Finally, the target parameter information is used as a true value control group to compare with the initial external parameter calibration parameters of the target vehicle, and the invalid values ​​in the initial external parameter calibration parameters are screened out, and the real-time joint external parameter calibration parameters can be output. The method shown in the present invention can realize the joint external parameter calibration of the vehicle laser radar and the camera during the driving process of the target vehicle. When the position of the intelligent driving sensor on the target vehicle is offset, the offset can be compensated by non-sensing real-time external parameter calibration, thereby improving the user's car experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0048] Figure 1 It is a flow chart of a method for calibrating a joint external parameter of a vehicle laser radar and a camera in one embodiment of the present invention;

[0049] Figure 2 This is a structural block diagram of a vehicle laser radar and camera joint external parameter calibration system in one embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. Several embodiments of the present invention are shown in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0052] Embodiment 1

[0053] See also Figure 1 The first embodiment of the present invention provides a joint external parameter calibration method for a vehicle laser radar and a camera, the method comprising steps S10 to S90:

[0054] Step S10, during the driving process of the target vehicle, laser point cloud information of the target road surface that the vehicle has driven on is obtained through a laser radar.

[0055] Among them, the target vehicle is a smart car, which is equipped with intelligent driving sensors of radar system and camera system, including lidar, camera and millimeter wave radar. It obtains environmental information through intelligent driving sensors, and then makes driving decisions based on the environmental information, thereby outputting driving strategy information, such as controlling the target vehicle to follow the vehicle, avoid the vehicle, etc.

[0056] Specifically, the intelligent driving sensors of the above radar system include laser radar and millimeter wave radar, of which the millimeter wave radar is usually a forward millimeter wave radar, so as to obtain the environmental information in front of the vehicle. During the driving process of the target vehicle, the laser radar will obtain the laser point cloud information of the target road surface that the target vehicle has passed in the form of laser point cloud and store it.

[0057] Step S20: acquiring video information of the target road surface through the camera.

[0058] Similarly, during the driving process of the target vehicle, the video information of the target road surface will be obtained in the form of a video signal through the camera of the camera system, and the video information records the real scene occurring on the target road surface.

[0059] Step S30, acquiring radar point cloud information of the target road surface through a forward millimeter-wave radar.

[0060] Similarly, during the driving process of the target vehicle, the radar point cloud information of the target road surface will also be obtained in the form of radar point cloud through the forward millimeter-wave radar of the radar system.

[0061] Step S40, the carrier phase collected by the reference station is sent to the on-board receiver of the target vehicle through real-time dynamic carrier phase difference, and the coordinates are calculated by difference to obtain differential positioning information.

[0062] In this embodiment, the carrier phase collected by the base station is sent to the on-board receiver of the target vehicle where the user is located through RTK (Real-time kinematic, real-time dynamic carrier phase difference), and then the coordinates are solved based on the carrier phase to obtain differential positioning information.

[0063] Step S50, obtaining navigation information of the target vehicle through a navigation satellite system, and sending the navigation information to the vehicle-mounted receiver.

[0064] In this embodiment, the navigation information of the target vehicle, including the three-dimensional coordinates, real-time speed and time information of the target vehicle, is obtained through GNSS (Global Navigation Satellite System), and then the navigation information is sent to the on-board receiver of the target vehicle where the user is located.

[0065] Step S60, obtaining vehicle posture information of the target vehicle through an inertial measurement unit, and sending the vehicle posture information to the vehicle-mounted receiver.

[0066] In this embodiment, the vehicle posture information of the target vehicle during driving is obtained through an IMU (Inertial Measurement Unit), and then the vehicle posture information is sent to the on-board receiver of the target vehicle where the user is located.

[0067] From the above, we can obtain laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle posture information.

[0068] Step S70, based on the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information, perform time synchronization and information alignment at the signal level based on the gPTP protocol, and perform visual perception processing, signal classification and signal fusion processing to obtain multiple lane elements.

[0069] In this embodiment, according to the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information generated by the above steps S10 to S60, the above information is synchronized at the signal level through the gPTP protocol (generalized Precision Time Protocol, which is an Ethernet-based time synchronization protocol defined by the IEEE 802.1AS standard).

[0070] Among them, the steps of performing time synchronization and information alignment at the signal level based on the gPTP protocol according to the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information include:

[0071] Comparing the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information with corresponding standard reference values ​​according to the time of issuance;

[0072] The comparison difference is synchronously fed back so that the system that subsequently obtains the above information can obtain more accurate information.

[0073] Specifically, the goal of time synchronization is to compare all signals with a standard reference value according to the time they are emitted, and to synchronously feed back the comparison differences so that subsequent systems that use these signals can obtain accurate information.

[0074] Among them, according to the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information generated by the above steps S10 to S60, visual perception processing, signal classification and signal fusion processing are performed on the above information to obtain multiple lane elements.

[0075] Specifically, the various lane elements include lane line type, lane line width, lane line color, lane line bifurcation point, lane line connection point, lane line merging point, lane line merging point, lane line appearance point, lane line vanishing point, lane line curvature, lane line longitude, lane line dimension, lane line grade, lane line length and road speed limit.

[0076] Step S80, performing information type fusion, attribute fusion, topology fusion and logical reasoning on multiple lane elements to obtain target parameter information.

[0077] In this embodiment, the steps of performing information type fusion, attribute fusion, topology fusion and logical reasoning on multiple lane elements to obtain target parameter information include:

[0078] All lane elements are classified by information type fusion to determine lane line information, position information and vehicle body posture information;

[0079] The attribute fusion method is used to clarify the parameter information of all lane elements and convert them into unified units;

[0080] The information is collected in segments by using topological fusion, and multiple segments of information are fused into a whole segment of topological information;

[0081] And use logical reasoning to judge the validity of all lane elements and eliminate invalid information.

[0082] Specifically, the various lane elements generated in step S70 are fused by information type to determine which are lane line information, which are position information, and which are vehicle body posture information; then attribute fusion is performed to clarify which parameters are included in the specific information type and convert them into a unified unit; and topological fusion is performed to collect information in segments and fuse multiple sub-segments of information into a whole segment of topological information, and finally logical reasoning is performed to determine which information is valid and which is invalid through reasoning, and the invalid information is eliminated. Through the above-mentioned information type fusion, attribute fusion, topological fusion and logical reasoning, the target parameter information is obtained.

[0083] Step S90, comparing the target parameter information with the initial external parameter calibration parameters of the target vehicle, using the target parameter information as a true value control group to filter out invalid values ​​in the initial external parameter calibration parameters, and outputting a combined external parameter calibration parameter.

[0084] Specifically, the target parameter information generated in step S90 is compared with the initial extrinsic calibration parameters of the target vehicle, and the target parameter information is used as a true value control group. The initial extrinsic calibration parameters are screened again to extract invalid values, and finally real-time joint extrinsic calibration parameters are output.

[0085] Compared with the prior art, the joint external parameter calibration method of the vehicle laser radar and the camera shown in this embodiment has the following beneficial effects:

[0086] The method shown in this embodiment obtains laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle posture information during the driving process of the target vehicle, and then performs signal-level time synchronization and information alignment based on the gPTP protocol, as well as information type fusion, attribute fusion, topology fusion and logical reasoning on the above information to obtain target parameter information. Finally, the target parameter information is used as a true value control group to compare with the initial external parameter calibration parameters of the target vehicle, and invalid values ​​in the initial external parameter calibration parameters are screened out to output real-time joint external parameter calibration parameters. The method shown in this embodiment can realize the joint external parameter calibration of the vehicle laser radar and the camera during the driving process of the target vehicle. When the position of the intelligent driving sensor on the target vehicle is offset, the offset can be compensated by non-sensitive real-time external parameter calibration, thereby improving the user's car experience.

[0087] Embodiment 2

[0088] The second embodiment of the present invention also provides a joint extrinsic parameter calibration method for a vehicle laser radar and a camera. The joint extrinsic parameter calibration method shown in this embodiment is basically the same as the joint extrinsic parameter calibration method shown in the first embodiment, except that:

[0089] In this embodiment, the method further includes:

[0090] After each power-on of the target vehicle, determine whether there are joint external calibration parameters of the laser radar and camera;

[0091] If yes, the joint external parameter calibration parameters are recorded and stored, and the vehicle driving time of the target vehicle from the current time node to the time when the joint external parameter calibration parameters are generated is detected;

[0092] When the driving time of the vehicle is greater than a preset time threshold, real-time joint external parameter dynamic calibration of the laser radar and the camera is performed;

[0093] If real-time joint external parameter calibration parameters are obtained, and there are differences with the initial external parameter calibration parameters when the vehicle is powered on for multiple consecutive times, and the vehicle driving time is greater than the preset time threshold, the initial external parameter calibration parameters are replaced according to the joint external parameter calibration parameters.

[0094] Wherein, if the real-time joint external parameter calibration parameters are obtained, and there are differences with the initial external parameter calibration parameters when the vehicle is powered on for multiple consecutive times, and the driving time of the vehicle is greater than the preset time threshold, the step of replacing the initial external parameter calibration parameters according to the joint external parameter calibration parameters includes:

[0095] When the combined external parameter calibration parameters differ from the initial external parameter calibration parameters when the vehicle is powered on for five consecutive times, and the vehicle driving time is greater than a preset time threshold, the initial external parameter calibration parameters are replaced according to the combined external parameter calibration parameters.

[0096] Specifically, after each power-on (start-up) of the target vehicle, the method shown in this embodiment will first determine whether there are joint external parameter calibration parameters of the laser radar and the camera, that is, the joint external parameter calibration parameters generated by the user during driving. If such parameters exist, the joint external parameter calibration parameters generated in the early stage will be recorded and stored, and the vehicle driving time of the target vehicle from the current time node to the time when the joint external parameter calibration parameters are generated will be detected. The time can be a natural time, such as a number of days, or it can be the actual driving time of the target vehicle, accurate to a number of hours and minutes, and then determine whether the vehicle driving time is greater than a preset time threshold. If it is greater than the preset time threshold, real-time joint external parameter dynamic calibration of the laser radar and the camera is performed. If the real-time joint external parameter calibration parameters are obtained, and there are differences with the initial external parameter calibration parameters when the vehicle is powered on for 5 consecutive times, and the vehicle driving time is greater than the preset time threshold, the initial external parameter calibration parameters are replaced according to the joint external parameter calibration parameters.

[0097] That is to say, in this embodiment, after each power-on of the target vehicle, a determination will be made as to whether to perform the joint external parameter calibration condition, thereby ensuring that the joint external parameter calibration is performed when actually needed.

[0098] Embodiment 3

[0099] See also Figure 2 A second embodiment of the present invention provides a joint external parameter calibration system of a vehicle laser radar and a camera, which is applied to the method described in any of the above embodiments, and the system includes:

[0100] The first information acquisition module is used to acquire laser point cloud information of the target road surface traveled by the target vehicle through the laser radar during the driving process;

[0101] A second information acquisition module, used to acquire video information of the target road surface through the camera;

[0102] A third information acquisition module, used to acquire radar point cloud information of the target road surface through a forward millimeter-wave radar;

[0103] The fourth information acquisition module is used to send the carrier phase collected by the reference station to the on-board receiver of the target vehicle through real-time dynamic carrier phase difference, perform difference calculation on the coordinates, and obtain differential positioning information;

[0104] A fifth information acquisition module, used to acquire navigation information of the target vehicle through a navigation satellite system, and send the navigation information to the vehicle-mounted receiver;

[0105] a sixth information acquisition module, configured to acquire vehicle posture information of the target vehicle through an inertial measurement unit, and send the vehicle posture information to the vehicle-mounted receiver;

[0106] A first information processing module is used to perform time synchronization and information alignment at the signal level based on the gPTP protocol according to the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information, and perform visual perception processing, signal classification and signal fusion processing to obtain multiple lane elements;

[0107] The second information processing module is used to perform information type fusion, attribute fusion, topology fusion and logical reasoning on multiple lane elements to obtain target parameter information;

[0108] The calibration parameter output module is used to compare the target parameter information with the initial external parameter calibration parameters of the target vehicle, screen out invalid values ​​in the initial external parameter calibration parameters using the target parameter information as a true value control group, and output a combined external parameter calibration parameter.

[0109] Specifically, the various lane elements include lane line type, lane line width, lane line color, lane line bifurcation point, lane line connection point, lane line merging point, lane line merging point, lane line appearance point, lane line vanishing point, lane line curvature, lane line longitude, lane line dimension, lane line grade, lane line length and road speed limit.

[0110] Compared with the prior art, the combined external parameter calibration system of the vehicle laser radar and the camera shown in this embodiment has the following beneficial effects:

[0111] The system shown in this embodiment obtains laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle posture information during the driving process of the target vehicle, and then performs signal-level time synchronization and information alignment based on the gPTP protocol, as well as information type fusion, attribute fusion, topology fusion and logical reasoning on the above information to obtain target parameter information. Finally, the target parameter information is used as a true value control group to compare with the initial external parameter calibration parameters of the target vehicle, and invalid values ​​in the initial external parameter calibration parameters are screened out to output real-time joint external parameter calibration parameters. The system shown in this embodiment can realize the joint external parameter calibration of the vehicle laser radar and the camera during the driving process of the target vehicle. When the position of the intelligent driving sensor on the target vehicle is offset, the offset can be compensated by non-sensitive real-time external parameter calibration, thereby improving the user's car experience.

[0112] Embodiment 4

[0113] A fourth embodiment of the present invention provides a readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method described in the above embodiment is implemented.

[0114] Embodiment 5

[0115] A fifth embodiment of the present invention provides a car, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the above embodiments when executing the computer program.

[0116] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0117] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A joint external parameter calibration method for a vehicle laser radar and a camera, characterized in that: The method comprises: During the driving process of the target vehicle, the laser point cloud information of the target road surface is obtained through the laser radar; Acquiring video information of the target road surface through the camera; Acquiring radar point cloud information of the target road surface by using a forward millimeter-wave radar; The carrier phase collected by the reference station is sent to the on-board receiver of the target vehicle through real-time dynamic carrier phase difference, and the coordinates are calculated to obtain differential positioning information; Acquire navigation information of the target vehicle through a navigation satellite system, and send the navigation information to the vehicle-mounted receiver; Acquire vehicle posture information of the target vehicle through an inertial measurement unit, and send the vehicle posture information to the vehicle-mounted receiver; According to the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information, time synchronization and information alignment at the signal level are performed based on the gPTP protocol, and visual perception processing, signal classification and signal fusion processing are performed to obtain multiple lane elements; Perform information type fusion, attribute fusion, topology fusion and logical reasoning on multiple lane elements to obtain target parameter information; The target parameter information is compared with the initial external parameter calibration parameters of the target vehicle, invalid values ​​in the initial external parameter calibration parameters are screened out using the target parameter information as a true value control group, and a combined external parameter calibration parameter is output.

2. The joint external parameter calibration method of vehicle laser radar and camera according to claim 1 is characterized in that: The steps of performing time synchronization and information alignment at the signal level based on the gPTP protocol according to the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information include: Comparing the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information with corresponding standard reference values ​​according to the time of issuance; The comparison difference is synchronously fed back so that the system that subsequently obtains the above information can obtain more accurate information.

3. The joint external parameter calibration method of vehicle laser radar and camera according to claim 1 is characterized in that: Various lane elements include lane type, lane width, lane color, lane bifurcation point, lane connection point, lane merging point, lane merging point, lane appearance point, lane vanishing point, lane curvature, lane longitude, lane dimension, lane grade, lane length and road speed limit.

4. The joint external parameter calibration method of vehicle laser radar and camera according to claim 3 is characterized in that: The steps of performing information type fusion, attribute fusion, topology fusion and logical reasoning on multiple lane elements to obtain target parameter information include: All lane elements are classified by information type fusion to determine lane line information, position information and vehicle body posture information; The attribute fusion method is used to clarify the parameter information of all lane elements and convert them into unified units; The information is collected in segments by using topological fusion, and multiple segments of information are fused into a whole segment of topological information; And use logical reasoning to judge the validity of all lane elements and eliminate invalid information.

5. The joint external parameter calibration method of a vehicle laser radar and a camera according to any one of claims 1 to 4, characterized in that: The method further comprises: After each power-on of the target vehicle, determine whether there are joint external calibration parameters of the laser radar and camera; If yes, the joint external parameter calibration parameters are recorded and stored, and the vehicle driving time of the target vehicle from the current time node to the time when the joint external parameter calibration parameters are generated is detected; When the driving time of the vehicle is greater than a preset time threshold, real-time joint external parameter dynamic calibration of the laser radar and the camera is performed; If real-time joint external parameter calibration parameters are obtained, and there are differences with the initial external parameter calibration parameters when the vehicle is powered on for multiple consecutive times, and the vehicle driving time is greater than the preset time threshold, the initial external parameter calibration parameters are replaced according to the joint external parameter calibration parameters.

6. The joint external parameter calibration method of vehicle laser radar and camera according to claim 5 is characterized in that: If the real-time joint external parameter calibration parameters are obtained, and there are differences with the initial external parameter calibration parameters when the vehicle is powered on for multiple consecutive times, and the driving time of the vehicle is greater than the preset time threshold, the step of replacing the initial external parameter calibration parameters according to the joint external parameter calibration parameters includes: When the combined external parameter calibration parameters differ from the initial external parameter calibration parameters when the vehicle is powered on for five consecutive times, and the vehicle driving time is greater than a preset time threshold, the initial external parameter calibration parameters are replaced according to the combined external parameter calibration parameters.

7. A joint external parameter calibration system for vehicle laser radar and camera, characterized in that: The method applied to any one of claims 1 to 6, wherein the system comprises: The first information acquisition module is used to acquire laser point cloud information of the target road surface traveled by the target vehicle through the laser radar during the driving process; A second information acquisition module, used to acquire video information of the target road surface through the camera; A third information acquisition module, used to acquire radar point cloud information of the target road surface through a forward millimeter-wave radar; The fourth information acquisition module is used to send the carrier phase collected by the reference station to the on-board receiver of the target vehicle through real-time dynamic carrier phase difference, perform difference calculation on the coordinates, and obtain differential positioning information; A fifth information acquisition module, used to acquire navigation information of the target vehicle through a navigation satellite system, and send the navigation information to the vehicle-mounted receiver; a sixth information acquisition module, configured to acquire vehicle posture information of the target vehicle through an inertial measurement unit, and send the vehicle posture information to the vehicle-mounted receiver; A first information processing module is used to perform time synchronization and information alignment at the signal level based on the gPTP protocol according to the laser point cloud information, video information, radar point cloud information, differential positioning information, navigation information and vehicle body posture information, and perform visual perception processing, signal classification and signal fusion processing to obtain multiple lane elements; The second information processing module is used to perform information type fusion, attribute fusion, topology fusion and logical reasoning on multiple lane elements to obtain target parameter information; The calibration parameter output module is used to compare the target parameter information with the initial external parameter calibration parameters of the target vehicle, screen out invalid values ​​in the initial external parameter calibration parameters using the target parameter information as a true value control group, and output a combined external parameter calibration parameter.

8. The vehicle laser radar and camera joint external parameter calibration system according to claim 7, characterized in that: Various lane elements include lane type, lane width, lane color, lane bifurcation point, lane connection point, lane merging point, lane merging point, lane appearance point, lane vanishing point, lane curvature, lane longitude, lane dimension, lane grade, lane length and road speed limit.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. An automobile, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

Images