Vehicle-mounted sensor sensing performance evaluation truth value system based on vehicle-infrastructure cooperation

Abstract

The invention relates to a vehicle-mounted sensor sensing performance evaluation truth value system based on vehicle-road cooperation, and the system comprises a vehicle-end sensing data obtaining module which is used for obtaining first point cloud data information of a road in real time based on a vehicle-mounted laser radar, obtaining first image data information of the road in real time based on a vehicle-mounted camera, and obtaining the first point cloud data information of the road in real time based on the vehicle-mounted laser radar; acquiring data information of speed, distance, horizontal angle and height of a target obstacle in real time based on a vehicle-mounted 4D millimeter wave radar; the roadside sensing data acquisition module is used for acquiring second point cloud data information of the road in real time based on a roadside laser radar and acquiring second image data information of the road in real time based on a roadside camera; and the vehicle end and road end sensing data fusion module is connected with the vehicle end sensing data acquisition module. The speed detection precision of the truth value system is improved, the influence of severe weather is reduced, the detection distance of the truth value system is prolonged, and the evaluation range is expanded.

Description

Technical Field

[0001] The present invention relates to the field of vehicle-mounted sensor technology, and in particular to a vehicle-road collaboration-based vehicle-mounted sensor perception performance evaluation truth value system. Background Art

[0002] A true value system is a data acquisition system composed of on-board sensors such as millimeter-wave radar, lidar, and high-precision combined inertial navigation, along with efficient data recording equipment. Because it processes data to produce data (true value) that is more reliable than the sensor being tested, true value systems are often used to evaluate the performance of the sensor being tested. Furthermore, based on the accuracy of data collection, after cleaning, labeling, and data mining of the true value data, a natural driving scene dataset can be generated, which can then be used to build a natural driving scene library.

[0003] With the rapid development of artificial intelligence and sensor technology, autonomous driving has made significant progress. As autonomous driving levels increase, the performance of autonomous vehicle sensors is bound to improve. However, actual performance in real-world road environments requires testing and verification. Traditional sensor testing methods primarily rely on single-product testing in closed scenarios, which poses challenges such as limited test scenario coverage and difficulty accurately reflecting actual road conditions. Summary of the Invention

[0004] In view of the above problems, the present invention provides a true value system for evaluating the perception performance of vehicle-mounted sensors based on vehicle-road collaboration, which not only improves the speed detection accuracy of the true value system and reduces the impact of bad weather, but also extends the detection distance of the true value system and expands the evaluation range.

[0005] In order to achieve the above-mentioned and other related purposes, the present invention provides the following technical solutions:

[0006] A vehicle-mounted sensor perception performance evaluation truth value system based on vehicle-road collaboration, the system comprising:

[0007] The vehicle-side perception data acquisition module is used to obtain the first point cloud data information of the road in real time based on the vehicle-mounted laser radar, the first image data information of the road in real time based on the vehicle-mounted camera, and the speed, distance, horizontal angle and height data information of the target obstacle in real time based on the vehicle-mounted 4D millimeter wave radar;

[0008] A roadside perception data acquisition module is used to acquire second point cloud data information of the road in real time based on a roadside laser radar, and to acquire second image data information of the road in real time based on a roadside camera;

[0009] a vehicle-side and road-side perception data fusion module, connected to the vehicle-side perception data acquisition module and the road test perception data acquisition module, for fusing the road and target obstacle data using a multimodal feature fusion algorithm based on a deep belief network with an attenuation factor based on the first point cloud data information of the road, the second point cloud data information of the road, the first image data information of the road, the second image data information of the road, and the speed, distance, horizontal angle, and height data information of the target obstacle, to obtain fused dynamic perception data;

[0010] The vehicle-side sensor perception performance evaluation true value module is connected to the vehicle-side and road-side perception data fusion module, and is used to construct the vehicle sensor perception performance evaluation true value function R based on the fused dynamic perception data, calculate the vehicle sensor perception performance evaluation true value, and obtain data information of the vehicle sensor perception performance evaluation true value.

[0011] Furthermore, the multimodal feature fusion algorithm based on the attenuation factor deep belief network is used to fuse the road and target obstacle data, including:

[0012] M1. Constructing a feature fusion function F of the image and point cloud of the road based on the first point cloud data information of the road, the second point cloud data information of the road, the first image data information of the road, and the second image data information of the road,

[0013] ,

[0014] Wherein, A1 is the first image data information of the road, A2 is the second image data information of the road, B1 is the first point cloud data information of the road, B2 is the second point cloud data information of the road, f1 is the image feature function of the road, f2 is the point cloud feature function of the road, ɑ1, ɑ2 and ɑ3 are the attenuation factors of the fusion of the image and point cloud of the road. The image and point cloud data of the road are fused to obtain the data information of the feature matrix after the fusion of the image and point cloud of the road;

[0015] M2. Input the data information of the speed, distance, horizontal angle and height of the target obstacle into the deep belief network for training and learning, extract the state feature matrix of the target obstacle, and obtain the data information of the state feature matrix of the target obstacle;

[0016] M3. Constructing a multimodal feature fusion function W based on the data information of the state feature matrix of the target obstacle and the data information of the feature matrix after the image and point cloud of the road are fused.

[0017] ,

[0018] Among them, x is the data information of the state feature matrix of the target obstacle, y is the data information of the feature matrix after the fusion of the road image and point cloud, β1, β2 and β3 are weight coefficients, and the data of the road and target obstacle are fused to obtain the fused dynamic perception data.

[0019] Furthermore, the image feature function f1 of the road is:

[0020] ,

[0021] Wherein, A1 is the first image data information of the road, and A2 is the second image data information of the road.

[0022] Furthermore, the point cloud feature function f2 of the road is:

[0023] ,

[0024] Among them, B1 is the first point cloud data information of the road, and B2 is the second point cloud data information of the road.

[0025] Furthermore, the attenuation factors ɑ1, ɑ2 and ɑ3 of the fusion of the road image and point cloud are:

[0026] ,

[0027] ,

[0028] ,

[0029] Among them, A1 is the first image data information of the road, A2 is the second image data information of the road, B1 is the first point cloud data information of the road, and B2 is the second point cloud data information of the road.

[0030] Furthermore, the constraints of the weight coefficients β1, β2 and β3 are:

[0031] .

[0032] Furthermore, the perception performance evaluation truth function R of the vehicle sensor is

[0033] ,

[0034] Among them, z is the fused dynamic perception data, δ, μ, and ω are the true value error factors of the perception performance evaluation of the vehicle sensor.

[0035] Furthermore, the true value error factors δ, μ and ω of the perception performance evaluation of the vehicle sensor are:

[0036] ,

[0037] ,

[0038] ,

[0039] Among them, z is the fused dynamic perception data.

[0040] Furthermore, the system also includes a display module, which is connected to the perception performance evaluation true value module of the vehicle-side sensor and is used to display the perception performance evaluation true value of the vehicle-side sensor in real time.

[0041] Furthermore, the system also includes a voice module, which is connected to the perception performance evaluation true value module of the vehicle-side sensor, and is used to obtain the perception performance evaluation true value of the vehicle-side sensor in real time and broadcast it.

[0042] The present invention has the following positive effects:

[0043] 1. The present invention fuses the road and target obstacle data by adopting a multimodal feature fusion algorithm based on a deep belief network with an attenuation factor to obtain fused dynamic perception data. Combined with the vehicle sensor's perception performance evaluation truth function R, the true value of the vehicle sensor's perception performance evaluation is inferred. This not only improves the speed detection accuracy of the truth value system and reduces the impact of bad weather, but also extends the detection distance of the truth value system and expands the evaluation range.

[0044] 2. By adding a true value system to the 4D millimeter-wave radar, this invention not only improves the system's overall detection accuracy but also allows for testing the sensor's extreme performance in inclement weather. Simultaneously, the use of vehicle-road collaborative sensing effectively addresses the long-distance sensor evaluation problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of the system framework of the present invention;

[0046] Figure 2 A schematic diagram of a drive test of the system of the present invention;

[0047] Figure 3 Schematic diagram of the process of the multimodal feature fusion algorithm of the deep belief network based on the attenuation factor of the present invention. DETAILED DESCRIPTION

[0048] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0049] Example 1: Figure 1 or Figure 2 As shown, a vehicle-mounted sensor perception performance evaluation truth value system based on vehicle-road collaboration includes:

[0050] The vehicle-side perception data acquisition module is used to obtain the first point cloud data information of the road in real time based on the vehicle-mounted laser radar, the first image data information of the road in real time based on the vehicle-mounted camera, and the speed, distance, horizontal angle and height data information of the target obstacle in real time based on the vehicle-mounted 4D millimeter wave radar;

[0051] A roadside perception data acquisition module is used to acquire second point cloud data information of the road in real time based on a roadside laser radar, and to acquire second image data information of the road in real time based on a roadside camera;

[0052] a vehicle-side and road-side perception data fusion module, connected to the vehicle-side perception data acquisition module and the road test perception data acquisition module, for fusing the road and target obstacle data using a multimodal feature fusion algorithm based on a deep belief network with an attenuation factor based on the first point cloud data information of the road, the second point cloud data information of the road, the first image data information of the road, the second image data information of the road, and the speed, distance, horizontal angle, and height data information of the target obstacle, to obtain fused dynamic perception data;

[0053] The vehicle-side sensor perception performance evaluation true value module is connected to the vehicle-side and road-side perception data fusion module, and is used to construct the vehicle sensor perception performance evaluation true value function R based on the fused dynamic perception data, calculate the vehicle sensor perception performance evaluation true value, and obtain data information of the vehicle sensor perception performance evaluation true value.

[0054] In this embodiment, if Figure 3 As shown, the multimodal feature fusion algorithm based on the deep belief network of the attenuation factor is used to fuse the data of the road and the target obstacle, including:

[0055] M1. Constructing a feature fusion function F of the image and point cloud of the road based on the first point cloud data information of the road, the second point cloud data information of the road, the first image data information of the road, and the second image data information of the road,

[0056] ,

[0057] Wherein, A1 is the first image data information of the road, A2 is the second image data information of the road, B1 is the first point cloud data information of the road, B2 is the second point cloud data information of the road, f1 is the image feature function of the road, f2 is the point cloud feature function of the road, ɑ1, ɑ2 and ɑ3 are the attenuation factors of the fusion of the image and point cloud of the road. The image and point cloud data of the road are fused to obtain the data information of the feature matrix after the fusion of the image and point cloud of the road;

[0058] M2. Input the data information of the speed, distance, horizontal angle and height of the target obstacle into the deep belief network for training and learning, extract the state feature matrix of the target obstacle, and obtain the data information of the state feature matrix of the target obstacle;

[0059] M3. Constructing a multimodal feature fusion function W based on the data information of the state feature matrix of the target obstacle and the data information of the feature matrix after the image and point cloud of the road are fused.

[0060] ,

[0061] Among them, x is the data information of the state feature matrix of the target obstacle, y is the data information of the feature matrix after the fusion of the road image and point cloud, β1, β2 and β3 are weight coefficients, and the data of the road and target obstacle are fused to obtain the fused dynamic perception data.

[0062] In this embodiment, the image feature function f1 of the road is:

[0063] ,

[0064] Wherein, A1 is the first image data information of the road, and A2 is the second image data information of the road.

[0065] In this embodiment, the point cloud feature function f2 of the road is:

[0066] ,

[0067] Among them, B1 is the first point cloud data information of the road, and B2 is the second point cloud data information of the road.

[0068] In this embodiment, the attenuation factors ɑ1, ɑ2 and ɑ3 of the fusion of the road image and point cloud are:

[0069] ,

[0070] ,

[0071] ,

[0072] Among them, A1 is the first image data information of the road, A2 is the second image data information of the road, B1 is the first point cloud data information of the road, and B2 is the second point cloud data information of the road.

[0073] In this embodiment, the constraints of the weight coefficients β1, β2 and β3 are:

[0074] .

[0075] In this embodiment, the perception performance evaluation truth function R of the vehicle sensor is:

[0076] ,

[0077] Among them, z is the fused dynamic perception data, δ, μ, and ω are the true value error factors of the perception performance evaluation of the vehicle sensor.

[0078] In this embodiment, the true value error factors δ, μ, and ω of the perception performance evaluation of the vehicle sensor are:

[0079] ,

[0080] ,

[0081] ,

[0082] Among them, z is the fused dynamic perception data.

[0083] Example 2: Based on the vehicle-road collaboration-based vehicle-mounted sensor perception performance evaluation truth value system of Example 1, the present invention is further illustrated and described below.

[0084] like Figure 1 or Figure 2 As shown, a vehicle-mounted sensor perception performance evaluation truth value system based on vehicle-road collaboration includes:

[0085] The vehicle-side perception data acquisition module is used to obtain the first point cloud data information of the road in real time based on the vehicle-mounted laser radar, the first image data information of the road in real time based on the vehicle-mounted camera, and the speed, distance, horizontal angle and height data information of the target obstacle in real time based on the vehicle-mounted 4D millimeter wave radar;

[0086] A roadside perception data acquisition module is used to acquire second point cloud data information of the road in real time based on a roadside laser radar, and to acquire second image data information of the road in real time based on a roadside camera;

[0087] a vehicle-side and road-side perception data fusion module, connected to the vehicle-side perception data acquisition module and the road test perception data acquisition module, for fusing the road and target obstacle data using a multimodal feature fusion algorithm based on a deep belief network with an attenuation factor based on the first point cloud data information of the road, the second point cloud data information of the road, the first image data information of the road, the second image data information of the road, and the speed, distance, horizontal angle, and height data information of the target obstacle, to obtain fused dynamic perception data;

[0088] The vehicle-side sensor perception performance evaluation true value module is connected to the vehicle-side and road-side perception data fusion module, and is used to construct the vehicle sensor perception performance evaluation true value function R based on the fused dynamic perception data, calculate the vehicle sensor perception performance evaluation true value, and obtain data information of the vehicle sensor perception performance evaluation true value.

[0089] In this embodiment, the system further includes a display module, which is connected to the perception performance evaluation true value module of the vehicle-side sensor and is used to display the perception performance evaluation true value of the vehicle-side sensor in real time.

[0090] In this embodiment, the system further includes a voice module connected to the perception performance evaluation true value module of the vehicle-side sensor, for obtaining the perception performance evaluation true value of the vehicle-side sensor in real time and reporting it.

[0091] LiDAR is susceptible to weather conditions. Rain, snow, and fog can scatter or absorb the laser beam, reducing detection accuracy and range. LiDAR can also be interfered with in direct sunlight or in highly reflective environments, affecting performance. Surfaces are easily contaminated, requiring regular cleaning and maintenance. With the advancement of millimeter-wave radar technology, 4D millimeter-wave radar not only achieves high-level perception but also outputs highly accurate speed detection. Millimeter-wave detection is only affected by reflections from materials like metal and glass, making it less susceptible to weather factors like rain, snow, and strong sunlight. Surface dirt also has no impact on performance. Therefore, adding a true value system to 4D millimeter-wave radar not only improves the system's overall detection accuracy but also allows for testing the sensor's performance limits in inclement weather.

[0092] For the evaluation of assisted driving sensors below Level 2, the traditional solution uses a combination of on-board lidar and cameras. The reliable range of on-board lidar is generally within 150 meters, which cannot be used to evaluate ordinary 3D millimeter-wave radar (detection range of 250 meters). Adding a 4D millimeter-wave radar has a maximum range of up to 350 meters, which can improve the detection range of the true value system to a certain extent.

[0093] However, with the continuous improvement of autonomous driving technology, 4D millimeter-wave radar will also be included in the evaluation of autonomous driving sensor configurations above L3 in the future. Furthermore, long-distance measurement accuracy is relatively poor for lidar. Therefore, the use of vehicle-road collaborative perception can effectively solve the long-distance sensor evaluation problem. Vehicle-road collaborative perception uses roadside lidar, cameras, and other sensors to accurately detect roadside targets. After packaging and processing by an industrial computer, the data is transmitted to the roadside unit (ROU). The RSU then transmits the perception data to the vehicle-side V2X module. The V2X module transmits it to the industrial computer of the vehicle-side truth system via transparent transmission. The vehicle-side perception data is then integrated with the roadside perception data to form high-precision, long-distance, real-time dynamic perception data.

[0094] In summary, the present invention not only improves the speed detection accuracy of the true value system and reduces the impact of bad weather, but also extends the detection distance of the true value system and expands the evaluation range.

[0095] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A true value system for evaluating vehicle-mounted sensor perception performance based on vehicle-road collaboration, characterized by: The system comprises: The vehicle-side perception data acquisition module is used to obtain the first point cloud data information of the road in real time based on the vehicle-mounted laser radar, the first image data information of the road in real time based on the vehicle-mounted camera, and the speed, distance, horizontal angle and height data information of the target obstacle in real time based on the vehicle-mounted 4D millimeter wave radar; A roadside perception data acquisition module is used to acquire second point cloud data information of the road in real time based on a roadside laser radar, and to acquire second image data information of the road in real time based on a roadside camera; a vehicle-side and road-side perception data fusion module, connected to the vehicle-side perception data acquisition module and the road test perception data acquisition module, for fusing the road and target obstacle data using a multimodal feature fusion algorithm based on a deep belief network with an attenuation factor based on the first point cloud data information of the road, the second point cloud data information of the road, the first image data information of the road, the second image data information of the road, and the speed, distance, horizontal angle, and height data information of the target obstacle, to obtain fused dynamic perception data; The vehicle-side sensor perception performance evaluation true value module is connected to the vehicle-side and road-side perception data fusion module, and is used to construct the vehicle sensor perception performance evaluation true value function R based on the fused dynamic perception data, calculate the vehicle sensor perception performance evaluation true value, and obtain data information of the vehicle sensor perception performance evaluation true value.

2. The vehicle-mounted sensor perception performance evaluation truth value system based on vehicle-road collaboration according to claim 1 is characterized by: The multimodal feature fusion algorithm based on the deep belief network of the attenuation factor is used to fuse the data of the road and the target obstacle, including: M1. Constructing a feature fusion function F of the image and point cloud of the road based on the first point cloud data information of the road, the second point cloud data information of the road, the first image data information of the road, and the second image data information of the road, ， Wherein, A1 is the first image data information of the road, A2 is the second image data information of the road, B1 is the first point cloud data information of the road, B2 is the second point cloud data information of the road, f1 is the image feature function of the road, f2 is the point cloud feature function of the road, ɑ1, ɑ2 and ɑ3 are the attenuation factors of the fusion of the image and point cloud of the road. The image and point cloud data of the road are fused to obtain the data information of the feature matrix after the fusion of the image and point cloud of the road; M2. Input the data information of the speed, distance, horizontal angle and height of the target obstacle into the deep belief network for training and learning, extract the state feature matrix of the target obstacle, and obtain the data information of the state feature matrix of the target obstacle; M3. Constructing a multimodal feature fusion function W based on the data information of the state feature matrix of the target obstacle and the data information of the feature matrix after the image and point cloud fusion of the road, ， Among them, x is the data information of the state feature matrix of the target obstacle, y is the data information of the feature matrix after the fusion of the road image and point cloud, β1, β2 and β3 are weight coefficients, and the data of the road and target obstacle are fused to obtain the fused dynamic perception data.

3. The vehicle-mounted sensor perception performance evaluation truth value system based on vehicle-road collaboration according to claim 2 is characterized by: The image feature function f1 of the road is: ， Wherein, A1 is the first image data information of the road, and A2 is the second image data information of the road.

4. The vehicle-mounted sensor perception performance evaluation truth value system based on vehicle-road collaboration according to claim 2 is characterized by: The point cloud feature function f2 of the road is: , Among them, B1 is the first point cloud data information of the road, and B2 is the second point cloud data information of the road.

5. The vehicle-mounted sensor perception performance evaluation truth value system based on vehicle-road collaboration according to claim 2 is characterized by: The attenuation factors ɑ1, ɑ2 and ɑ3 of the image and point cloud fusion of the road are: ， ， ， Among them, A1 is the first image data information of the road, A2 is the second image data information of the road, B1 is the first point cloud data information of the road, and B2 is the second point cloud data information of the road.

6. The vehicle-mounted sensor perception performance evaluation truth value system based on vehicle-road collaboration according to claim 2 is characterized by: The constraints of the weight coefficients β1, β2 and β3 are: 。 7. The vehicle-mounted sensor perception performance evaluation truth value system based on vehicle-road collaboration according to claim 1 is characterized by: The perception performance evaluation truth function R of the vehicle sensor, ， Among them, z is the fused dynamic perception data, δ, μ, and ω are the true value error factors of the perception performance evaluation of the vehicle sensor.

8. The vehicle-mounted sensor perception performance evaluation truth value system based on vehicle-road collaboration according to claim 7 is characterized by: The true value error factors δ, μ and ω of the perception performance evaluation of the vehicle sensor are: ， ， ， Among them, z is the fused dynamic perception data.

9. The vehicle-mounted sensor perception performance evaluation truth value system based on vehicle-road collaboration according to claim 1 is characterized in that: The system also includes a display module, which is connected to the perception performance evaluation true value module of the vehicle-side sensor and is used to display the perception performance evaluation true value of the vehicle-side sensor in real time.

10. The vehicle-mounted sensor perception performance evaluation truth value system based on vehicle-road collaboration according to claim 1 is characterized in that: The system also includes a voice module, which is connected to the perception performance evaluation true value module of the vehicle-side sensor, and is used to obtain the perception performance evaluation true value of the vehicle-side sensor in real time and broadcast it.

Images