A multi-sensor fusion perception calibration method under vehicle-road cooperative conditions

Abstract

The present invention relates to the field of multi-sensor fusion of vehicle cooperation, and specifically discloses a multi-sensor fusion perception calibration method under vehicle-road cooperation conditions, including the following steps: selecting a vehicle-road cooperation intersection according to corresponding road traffic safety factors; judging the intersection type of the vehicle-road cooperation intersection according to the selected vehicle-road cooperation intersection, and determining the intersection perception range corresponding to the vehicle-road cooperation intersection; determining the corresponding calibration method according to the intersection type, and calibrating the vehicle-road cooperation intersection to obtain corresponding calibration data; matching the calibration data of the previous moment at the current moment according to the obtained calibration data, and using the calibration algorithm according to the calibration data of the previous moment to generate the corresponding calibration result of the current moment; comparing the corresponding calibration result of the current moment with the real result, if the two are consistent, then completing this calibration and performing the calibration of the next moment. This scheme can achieve the unification of the spatiotemporal position of the detection target of different sensors, and the accuracy of the calibration result obtained is also higher.

Description

Technical Field

[0001] The present invention relates to the field of multi-sensor fusion for vehicle-road collaboration, and in particular to a multi-sensor fusion perception calibration method under vehicle-road collaboration conditions. Background Art

[0002] In the vehicle's advanced driver assistance system, positioning and mapping are indispensable technologies. For example, when GPS is weak or there is no GPS, the vehicle needs to rely on SLAM technology for navigation and path planning. The advanced driver assistance system uses a variety of sensors installed on the vehicle, such as millimeter-wave radar, lidar, camera, and satellite navigation, to sense the surrounding environment at any time during the driving process of the car, collect data, identify, detect and track static and dynamic objects, and combine the navigation map data to perform systematic calculations and analysis, so as to make the driver aware of possible dangers in advance, effectively increasing the comfort and safety of car driving.

[0003] The basic principle of multi-sensor fusion is just like the process of comprehensive information processing by the human brain. It processes various sensors through multi-level and multi-space information complementarity and optimal combination, and finally produces a consistent interpretation of the observed environment. Summary of the invention

[0004] The present invention aims to provide a multi-sensor fusion perception calibration method under vehicle-road collaborative conditions, which can achieve the unification of the temporal and spatial positions of targets detected by different sensors, and at the same time, the accuracy of the calibration results obtained is higher.

[0005] To achieve the above object, the present invention adopts the following technical solution: a multi-sensor fusion perception calibration method under vehicle-road cooperative conditions, comprising the following steps:

[0006] S1, select the vehicle-road cooperative intersection according to the corresponding road traffic safety factors;

[0007] S2, judging the intersection type of the selected vehicle-road cooperative intersection, and determining the intersection sensing range corresponding to the vehicle-road cooperative intersection according to the corresponding intersection type;

[0008] S3, determining a calibration method corresponding to the vehicle-road cooperative intersection according to the intersection type of the vehicle-road cooperative intersection;

[0009] S4, calibrating the vehicle-road cooperative intersection according to the determined calibration method, and obtaining corresponding calibration data; the calibration data includes millimeter wave radar point data, camera point data and RTK point data;

[0010] S5, matching the calibration data of the previous moment at the current moment according to the acquired calibration data, and using the calibration algorithm according to the calibration data of the previous moment to generate the corresponding calibration result of the current moment;

[0011] S6, compare the corresponding calibration result at the current moment with the actual result. If the two are consistent, the current calibration is completed and the calibration at the next moment is carried out. Otherwise, S4 is carried out.

[0012] The principle and advantages of this scheme are: in this scheme, first of all, the vehicle-road cooperative intersection will be selected according to the road traffic safety factors, so that the selected vehicle-road cooperative intersection can play its due technical value when the equipment is installed later, and can also play its due social benefits. Then the intersection perception range of the selected vehicle-road cooperative intersection will be determined. The perception range corresponding to different intersection types is also different, which can be more humane and accurate. After that, the calibration method is selected. The calibration method corresponding to different intersection types is also different. In the selection of the calibration method, not only the accuracy of the collection should be considered, but also the safety of the collection and the time spent on the collection should be considered. The determination of the calibration method can make the collection of the entire calibration data more reasonable and safe, and greatly improve the efficiency of the calibration data collection. Of course, after the corresponding calibration data is collected, the calibration algorithm will be used to generate the calibration result of the current moment according to the calibration data of the previous moment. First, the calibration algorithm is used to achieve the unification of the time and space positions of different sensors, that is, the point data of the millimeter wave radar, camera and RTK are converted into the coordinates corresponding to the geodetic coordinate system. Secondly, the calibration result at the current moment generated in this way can not only make full use of the historical calibration data and enhance the utilization rate of the historical calibration data, but also further increase the accuracy of the calibration result.

[0013] Preferably, as an improvement, S5 includes:

[0014] S50, matching the millimeter-wave radar point data, camera point data, and RTK point data corresponding to the previous moment at the current moment according to the acquired millimeter-wave radar point data, camera point data, and RTK point data;

[0015] S51, based on the matched millimeter-wave radar point data, camera point data and RTK point data of the previous moment, the Actor-Critic algorithm is used to predict the calibration result of the millimeter-wave radar in the geodetic coordinate system and the calibration result of the camera in the geodetic coordinate system at the current moment. If the corresponding calibration result is predicted, proceed to S6, otherwise proceed to the next step;

[0016] S52, based on the matched millimeter-wave radar point data, camera point data and RTK point data of the previous moment, use perspective transformation to calculate the calibration result of the millimeter-wave radar in the geodetic coordinate system and the calibration result of the camera in the geodetic coordinate system at the current moment.

[0017] Beneficial effects: In this scheme, two methods are first used to generate the calibration results at the current moment. One is to use the Actor-Critic algorithm to predict the calibration results at the current moment, and the other is to calculate the calibration results at the current moment when the former cannot predict. The setting of these two methods greatly increases the success rate of generating the corresponding calibration results, and at the same time makes the entire calibration results more reasonable and accurate, and makes great use of historical calibration data.

[0018] Preferably, as an improvement, the calibration method includes a dynamic calibration method and a static calibration method.

[0019] Beneficial effects: By setting two different calibration methods, the entire data collection process can adapt to more road sections with different conditions, greatly increasing the diversity and efficiency of data collection.

[0020] Preferably, as an improvement, the dynamic calibration method is:

[0021] Install the front-end acquisition sensor equipment on the corresponding acquisition vehicle, and adjust the angle of the front-end acquisition sensor equipment according to the perception range of the intersection; the front-end acquisition sensor equipment includes a camera, a millimeter wave radar and RTK;

[0022] Drive through each lane of the intersection within the perception range of the intersection one by one, and collect data in the process to generate corresponding millimeter-wave radar point data, camera point data, and RTK point data; each time data is collected, the camera, millimeter-wave radar, and RTK must be collected simultaneously while the vehicle is driving;

[0023] The GPS points generated by the millimeter-wave radar and camera data are collected to generate corresponding millimeter-wave radar GPS point data and camera GPS point data.

[0024] Beneficial effects: This dynamic calibration can better adapt to roads with more complex vehicle conditions, which not only greatly saves calibration time but is also safer.

[0025] Preferably, as an improvement, the static calibration method is:

[0026] Select 6 points at the intersection within the sensing range of the intersection, hold the front-end collection sensor device, first collect points from the left and then from the right, from near to far, keep the order of collecting points for each device consistent, and collect the corresponding millimeter wave radar point data, camera point data and RTK point data in turn;

[0027] The GPS points generated by the millimeter-wave radar and camera data are collected to generate corresponding millimeter-wave radar GPS point data and camera GPS point data.

[0028] Beneficial effects: The static calibration method can calibrate some road sections that are inconvenient for vehicles to pass through, making the corresponding calibration sections more diverse.

[0029] Preferably, as an improvement, the steps between S5 and S6 further include:

[0030] According to the millimeter-wave radar point data, camera point data, and their corresponding GPS point data at the current moment, the calibration result at the current moment calculated by the calibration algorithm is compared with the corresponding GPS point data for verification. If the requirements are met, S6 is performed. If not, the millimeter-wave radar point data and camera point data are adjusted until the requirements are met.

[0031] Beneficial effect: By comparing the calibration results at the current moment with the corresponding GPS point data, the calibration results are automatically corrected, making the final judgment more accurate and greatly improving the accuracy of the calibration.

[0032] Preferably, as an improvement, the calibration result at the current moment calculated by the calibration algorithm according to the millimeter-wave radar point data, the camera point data, and the corresponding GPS point data are compared with the corresponding GPS point data for verification. If the requirements are met, S6 is performed. If not, the millimeter-wave radar point data and the camera point data are adjusted until the requirements are met, including:

[0033] S53, matching the millimeter wave radar point data and the camera point data at the current moment, and obtaining the GPS point data corresponding to the millimeter wave radar and the camera at the current moment;

[0034] S54, calculating the corresponding calibration result of the millimeter-wave radar in the geodetic coordinate system and the calibration result of the camera in the geodetic coordinate system by using perspective transformation according to the millimeter-wave radar point data and the camera point data at the current moment;

[0035] S54, comparing the GPS point data corresponding to the millimeter wave radar and the camera at the current moment with their respective corresponding calibration results, to determine whether there is a deviation between the acquired GPS point data and the calculated calibration result; if so, reversely calculating the corresponding sampling point data from the calibration result using the calibration algorithm, and calculating the corresponding deviation amount through the reverse calculated sampling point data and the previously collected sampling point data; otherwise, the calibration is completed and S6 is performed;

[0036] S55, adjusting the collected sampling point data according to the corresponding deviation amount, generating corresponding new millimeter wave radar sampling point data and camera sampling point data, and then proceeding to S54.

[0037] Beneficial effect: The calibration algorithm is used to calculate the corresponding calibration result in the forward direction, and then the accuracy of the calibration result is determined based on the calibration result and the corresponding GPS point data. If the accuracy is not satisfied, the calibration algorithm is used to calculate the corresponding equipment point data in the reverse direction. By adjusting the point data, the calibration result corresponding to the forward calculation of the calibration algorithm is consistent with the GPS point data, so that the final calibration result is accurate. In this way, the calibration result is automatically corrected, and the whole process is logical and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic structural diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following is further described in detail through specific implementation methods:

[0040] The embodiment is basically as shown in the attached Figure 1 As shown: A multi-sensor fusion perception calibration method under vehicle-road cooperative conditions, comprising the following steps:

[0041] S1. Select a vehicle-road cooperative intersection based on corresponding road traffic safety factors. In this embodiment, road traffic safety factors include mixed traffic conflicts among traffic participants, road blind spots, frequent accidents, traffic congestion, road occupancy, road structuring, traffic safety facilities, road weather, road lighting and road friction coefficient.

[0042] S2, judging the intersection type of the vehicle-road cooperative intersection according to the selected vehicle-road cooperative intersection, and determining the intersection sensing range corresponding to the vehicle-road cooperative intersection according to the corresponding intersection type;

[0043] S3, determining a calibration method corresponding to the vehicle-road cooperative intersection according to the intersection type of the vehicle-road cooperative intersection; the calibration method includes a dynamic calibration method and a static calibration method.

[0044] The dynamic calibration method is:

[0045] Install the front-end acquisition sensor equipment on the corresponding acquisition vehicle, and adjust the angle of the front-end acquisition sensor equipment according to the perception range of the intersection; the front-end acquisition sensor equipment includes a camera, a millimeter wave radar and RTK;

[0046] Drive through each lane of the intersection within the perception range of the intersection one by one, and collect data in the process to generate corresponding millimeter-wave radar point data, camera point data, and RTK point data; each time data is collected, the camera, millimeter-wave radar, and RTK must be collected simultaneously while the vehicle is driving;

[0047] The GPS points generated by the millimeter-wave radar and camera data are collected to generate corresponding millimeter-wave radar GPS point data and camera GPS point data.

[0048] The static calibration method is:

[0049] Select 6 points at the intersection within the sensing range of the intersection, hold the front-end collection sensor device, first collect points from the left and then from the right, from near to far, keep the order of collecting points for each device consistent, and collect the corresponding millimeter wave radar point data, camera point data and RTK point data in turn;

[0050] The GPS points generated by the millimeter-wave radar and camera data are collected to generate corresponding millimeter-wave radar GPS point data and camera GPS point data.

[0051] S4, calibrating the vehicle-road cooperative intersection according to the determined calibration method, and obtaining corresponding calibration data; the calibration data includes millimeter wave radar point data, camera point data and RTK point data;

[0052] S5, matching the calibration data at the previous moment at the current moment according to the acquired calibration data, and generating the corresponding calibration result at the current moment by using an algorithm according to the calibration data at the previous moment;

[0053] The S5 includes:

[0054] S50, matching the millimeter-wave radar point data, camera point data, and RTK point data corresponding to the previous moment at the current moment according to the acquired millimeter-wave radar point data, camera point data, and RTK point data;

[0055] S51, based on the matched millimeter-wave radar point data, camera point data and RTK point data of the previous moment, the Actor-Critic algorithm is used to predict the calibration result of the millimeter-wave radar in the geodetic coordinate system and the calibration result of the camera in the geodetic coordinate system at the current moment. If the corresponding calibration result is predicted, proceed to S6, otherwise proceed to the next step;

[0056] S52, based on the matched millimeter-wave radar point data, camera point data and RTK point data of the previous moment, use perspective transformation to calculate the calibration result of the millimeter-wave radar in the geodetic coordinate system and the calibration result of the camera in the geodetic coordinate system at the current moment.

[0057] The S5 and S6 also include:

[0058] According to the millimeter-wave radar point data, camera point data, and their corresponding GPS point data at the current moment, the calibration result at the current moment calculated by the calibration algorithm is compared with the corresponding GPS point data for verification. If the requirements are met, S6 is performed. If not, the millimeter-wave radar point data and camera point data are adjusted until the requirements are met.

[0059] Specifically, including:

[0060] S53, matching the millimeter wave radar point data and the camera point data at the current moment, and obtaining the GPS point data corresponding to the millimeter wave radar and the camera at the current moment;

[0061] S54, calculating the corresponding calibration result of the millimeter-wave radar in the geodetic coordinate system and the calibration result of the camera in the geodetic coordinate system by using perspective transformation according to the millimeter-wave radar point data and the camera point data at the current moment;

[0062] S54, comparing the GPS point data corresponding to the millimeter wave radar and the camera at the current moment with their respective corresponding calibration results, to determine whether there is a deviation between the acquired GPS point data and the calculated calibration result; if so, reversely calculating the corresponding sampling point data from the calibration result using the calibration algorithm, and calculating the corresponding deviation amount through the reverse calculated sampling point data and the previously collected sampling point data; otherwise, the calibration is completed and S6 is performed;

[0063] S55, adjusting the collected sampling point data according to the corresponding deviation amount, generating corresponding new millimeter wave radar sampling point data and camera sampling point data, and then proceeding to S54.

[0064] For example, taking the camera point data as an example, assume that the camera point data point A (x1, y1) has been obtained before calibration, and the corresponding GPS point B (x2, y2) has been collected, and the calibration algorithm is used to obtain the GPS point C (x3, y3) corresponding to point A. At this time, the deviation between point C and point B can be obtained:

[0065]

[0066] Point C is calibrated by the calibration algorithm to obtain the calibrated pixel position D (x4, y4). At this time, the deviation between point A and point D can be obtained:

[0067]

[0068] According to the gradient descent algorithm, the adjustment amount of point A needs to be determined, so:

[0069] G=L2-L1

[0070] The final adjustment amount of point A is:

[0071]

[0072]

[0073] Just add X and Y to the corresponding horizontal and vertical coordinates of point A.

[0074] S6, compare the corresponding calibration result at the current moment with the actual result. If the two are consistent, the current calibration is completed and the calibration at the next moment is carried out. Otherwise, S4 is carried out.

[0075] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions and / or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical solution of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A multi-sensor fusion perception calibration method under vehicle-road cooperative conditions, characterized by: The following steps are involved: S1, select the vehicle-road cooperative intersection according to the corresponding road traffic safety factors; S2, judging the intersection type of the vehicle-road cooperative intersection according to the selected vehicle-road cooperative intersection, and determining the intersection sensing range corresponding to the vehicle-road cooperative intersection according to the corresponding intersection type; S3, determining a calibration method corresponding to the vehicle-road cooperative intersection according to the intersection type of the vehicle-road cooperative intersection; S4, calibrating the vehicle-road cooperative intersection according to the determined calibration method, and obtaining corresponding calibration data; the calibration data includes millimeter wave radar point data, camera point data and RTK point data; S5, matching the calibration data of the previous moment at the current moment according to the acquired calibration data, and using the calibration algorithm according to the calibration data of the previous moment to generate the corresponding calibration result of the current moment; S6, compare the calibration result at the current moment with the real result. If the two are consistent, the current calibration is completed and the calibration at the next moment is carried out. Otherwise, S4 is carried out. The S5 includes: S50, matching the millimeter-wave radar point data, camera point data, and RTK point data corresponding to the previous moment at the current moment according to the acquired millimeter-wave radar point data, camera point data, and RTK point data; S51, based on the matched millimeter-wave radar point data, camera point data and RTK point data of the previous moment, the Actor-Critic algorithm is used to predict the calibration result of the millimeter-wave radar in the geodetic coordinate system and the calibration result of the camera in the geodetic coordinate system at the current moment. If the corresponding calibration result is predicted, proceed to S6, otherwise proceed to the next step; S52, according to the matched millimeter-wave radar point data, camera point data and RTK point data of the previous moment, use perspective transformation to calculate the calibration result of the millimeter-wave radar in the geodetic coordinate system and the calibration result of the camera in the geodetic coordinate system at the current moment.

2. The multi-sensor fusion perception calibration method under vehicle-road cooperative conditions according to claim 1 is characterized by: The calibration method includes a dynamic calibration method and a static calibration method.

3. The multi-sensor fusion perception calibration method under vehicle-road cooperative conditions according to claim 2 is characterized by: The dynamic calibration method is: Install the front-end acquisition sensor equipment on the corresponding acquisition vehicle, and adjust the angle of the front-end acquisition sensor equipment according to the perception range of the intersection; the front-end acquisition sensor equipment includes a camera, a millimeter wave radar and RTK; Drive through each lane of the intersection within the perception range of the intersection one by one, and collect data in the process to generate corresponding millimeter-wave radar point data, camera point data, and RTK point data; each time data is collected, the camera, millimeter-wave radar, and RTK must be collected simultaneously while the vehicle is driving; The GPS points generated by the millimeter-wave radar and camera data are collected to generate corresponding millimeter-wave radar GPS point data and camera GPS point data.

4. The multi-sensor fusion perception calibration method under vehicle-road cooperative conditions according to claim 3 is characterized by: The static calibration method is: Select 6 points at the intersection within the sensing range of the intersection, hold the front-end collection sensor device, first collect points from the left and then from the right, from near to far, keep the order of collecting points for each device consistent, and collect the corresponding millimeter wave radar point data, camera point data and RTK point data in turn; The GPS points generated by the millimeter-wave radar and camera data are collected to generate corresponding millimeter-wave radar GPS point data and camera GPS point data.

5. The multi-sensor fusion perception calibration method under vehicle-road cooperative conditions according to claim 1 is characterized by: The S5 and S6 also include: According to the millimeter-wave radar point data, camera point data, and their corresponding GPS point data at the current moment, the calibration result at the current moment calculated by the calibration algorithm is compared with the corresponding GPS point data for verification. If the requirements are met, S6 is performed. If not, the millimeter-wave radar point data and camera point data are adjusted until the requirements are met.

6. The multi-sensor fusion perception calibration method under vehicle-road cooperative conditions according to claim 5 is characterized by: The calibration result at the current moment calculated by the calibration algorithm according to the millimeter-wave radar point data, the camera point data, and the corresponding GPS point data are compared with the corresponding GPS point data for verification. If the requirements are met, S6 is performed. If not, the millimeter-wave radar point data and the camera point data are adjusted until the requirements are met, including: S53, matching the millimeter wave radar point data and the camera point data at the current moment, and obtaining the GPS point data corresponding to the millimeter wave radar and the camera at the current moment; S54, calculating the corresponding calibration result of the millimeter-wave radar in the geodetic coordinate system and the calibration result of the camera in the geodetic coordinate system by using perspective transformation according to the millimeter-wave radar point data and the camera point data at the current moment; S54, comparing the GPS point data corresponding to the millimeter wave radar and the camera at the current moment with their respective corresponding calibration results, to determine whether there is a deviation between the acquired GPS point data and the calculated calibration result; if so, reversely calculating the corresponding sampling point data from the calibration result using the calibration algorithm, and calculating the corresponding deviation amount through the reverse calculated sampling point data and the previously collected sampling point data; otherwise, the calibration is completed and S6 is performed; S55, adjusting the collected sampling point data according to the corresponding deviation amount, generating corresponding new millimeter wave radar sampling point data and camera sampling point data, and then proceeding to S54.

Images