Inspection test platform for vehicle-road cooperation roadside fusion sensing system

Abstract

The embodiment of the invention discloses an inspection test platform for a vehicle-road cooperation roadside fusion sensing system, and the inspection test platform communicates with a user side and a truth-value inspection vehicle, and is used for creating a test task for testing the vehicle-road cooperation roadside fusion sensing system, and transmitting the test task to the user side. And receiving test data generated by the truth-value inspection vehicle executing the test task in the test area according to the user guidance generated by the user side, and analyzing the test data to generate a test report. According to the invention, various types of sensors are installed on the real inspection vehicle, so that comprehensive test data can be obtained. Therefore, various evaluation indexes of the vehicle-road cooperation roadside fusion sensing system can be comprehensively tested, and three types of indexes including target identification precision, target detection precision and data frequency stability can be calculated; meanwhile, the flexibility of a mobile sensor method is achieved, truth value data and sensing data under the same time and space can be collected in real time, and a test report of the inspection task is generated after the inspection task is finished.

Description

Technical Field

[0001] The present application relates to the field of traffic testing technology, and more specifically, to an inspection and testing platform for a vehicle-road collaborative roadside fusion perception system. Background Art

[0002] In recent years, the country has vigorously promoted the integrated development strategy of vehicle, road and cloud. The Ministry of Industry and Information Technology and other five ministries have organized pilot projects for the application of "vehicle, road and cloud integration" for intelligent connected vehicles, and clarified the development path of "vehicle, road and cloud integration". my country's intelligent connected vehicles have officially entered a new stage of large-scale promotion at the city level and exploration of new business models. However, according to the "Summary Report on the Practice Activities of the Four Cross-border Pilot Applications of C-V2X in the Internet of Vehicles (2022)" and the summary of the testing experience of the Internet of Vehicles infrastructure in various places, the built roadside perception systems have problems such as inconsistency with standards and insufficient accuracy of perception data. There is a general lack of operation and maintenance systems for roadside perception facilities in various regions, and there is a lack of continuous accuracy assurance of perception data. The industry needs an automated, accurate and comprehensive inspection system to quickly evaluate and inspect vehicle-road-cloud integrated facilities.

[0003] The roadside perception system test is mainly aimed at two system stages: one is the acceptance test in the construction phase, which mainly tests the equipment performance and service capabilities after the system is built, in order to verify whether the roadside perception system has stable and efficient service capabilities; the other is the operation and maintenance monitoring in the operation phase, in order to continuously verify whether the roadside perception system can provide stable services.

[0004] The existing roadside perception system testing method is mainly achieved by comparing the differences between higher-precision heterogeneous true data and the data to be evaluated. The acquisition equipment carriers of heterogeneous true data are mainly divided into three types: one is the roadside poles, and the acquisition equipment is installed in the adjacent position of the sensor to be tested. This method is difficult to install, inconvenient to debug, and has low test efficiency; the second is the mobile acquisition station, which does not rely on the roadside poles and can improve the accuracy of the true data by fusing multiple sensors, but each test requires redeployment and calibration, making it difficult to achieve an automated process; the third is to install the sensor on the test vehicle. Most of the current tests only install RTK, which cannot achieve multi-target testing, and there is no inspection cloud platform to receive the test true data in real time, so it cannot be analyzed in real time, and the data cannot be uniformly classified and managed. Summary of the invention

[0005] This application provides an inspection test platform for a vehicle-road cooperative roadside fusion perception system to conduct a comprehensive and accurate test of the vehicle-road cooperative roadside fusion perception system. The specific scheme is as follows:

[0006] A patrol test platform for a vehicle-road cooperative roadside fusion perception system, the patrol test platform comprising a patrol cloud platform deployed on a cloud server, the patrol cloud platform communicating with a user terminal and a true value patrol vehicle;

[0007] The inspection cloud platform is used to create a test task for testing the vehicle-road cooperative roadside fusion perception system and send it to the user end;

[0008] The user terminal is used to execute the test task and generate user instructions to be sent to the true value inspection vehicle;

[0009] The true value inspection vehicle is used to perform the test task in the test area according to the user's instructions, thereby generating test data, and reporting the test data to the inspection cloud platform;

[0010] The inspection cloud platform is used to analyze the test data and generate a test report, wherein the test report includes evaluation indicators of the vehicle-road cooperative roadside fusion perception system.

[0011] Optionally, the true value inspection vehicle includes a timing device, multiple sensors, a switch, an acquisition controller and a network module; the timing device includes a time synchronization server for synchronizing the time of each device on the true value inspection vehicle; the multiple sensors are used to collect status data of traffic participants in the test area, and use the status data as test data and report it to the inspection cloud platform through the network module; the switch is used to exchange data between the acquisition controller and the network module; the acquisition controller is used to control the multiple sensors to collect data.

[0012] Optionally, the test data includes: true target data perceived by the true patrol vehicle in the same time and space, motion data of the true patrol vehicle, and target data to be tested perceived by the tested vehicle-road cooperative roadside fusion perception system.

[0013] Optionally, the inspection cloud platform is used to:

[0014] Performing truth value cleaning on the true value target data and the motion data to obtain cleaned target data and motion data;

[0015] Performing spatiotemporal alignment matching on the target data to be tested and the cleaned true value target data to obtain target data after spatiotemporal alignment processing;

[0016] Clearing the time-space aligned target data and the data not in the test area in the cleaned inspection data to obtain processed test data;

[0017] The processed test data is used to analyze the evaluation indicators of the vehicle-road cooperative roadside fusion perception system.

[0018] Optionally, the performing cleaning on the true value target data and the true value data to obtain cleaned true value target data and motion data includes:

[0019] Deleting the motion data whose precision is lower than the precision threshold in the motion data and deleting the target data whose confidence is lower than the confidence threshold in the true value target data, to obtain the initially cleaned motion data and the true value target data;

[0020] The reliability of the current frame data is inferred based on the previous and next frame data, and the outlier data that violates the physical relationship in the initially cleaned motion data and true value target data is removed to obtain the cleaned inspection data.

[0021] Optionally, performing spatiotemporal alignment matching on the target data to be tested and the cleaned true value target data to obtain the target data after spatiotemporal alignment processing includes:

[0022] Interpolate the cleaned true value target data and the cleaned motion data based on the time of the target data to be measured to obtain interpolated true value target data and motion data;

[0023] Calculating the trajectory matching degree between the interpolated true value target data and the target data to be measured, and obtaining the matching relationship between the interpolated true value target data and the target data to be measured based on the mutually exclusive principle of trajectory matching degree and target unique matching;

[0024] The matched true value target data and the target data to be tested are labeled with TP, the unmatched target data to be tested are labeled with FP, and the unmatched true value target data are labeled with FN;

[0025] The true value target data carrying the label and the data not in the test area in the target data to be tested are removed to obtain the target data after the spatiotemporal alignment processing.

[0026] Optionally, the analyzing the evaluation index of the vehicle-road cooperative roadside fusion perception system by using the processed test data includes:

[0027] The grid map in the test area is used to calculate the evaluation index of the vehicle-road cooperative roadside fusion perception system, and the evaluation index includes a rasterization index and an overall index. Both the rasterization index and the overall index include the target recognition accuracy, the target detection accuracy and the data frequency stability rate. If the target data carrying the TP tag in a grid does not reach the predetermined quantity index, the rasterization index is invalid. The overall index uses the number of samples of the target data carrying the TP tag in each grid as the weighted average value.

[0028] Optionally, the target recognition accuracy includes positioning accuracy, positioning standard deviation, speed accuracy, speed standard deviation, heading angle accuracy, heading angle standard deviation, size accuracy and size standard deviation; the target detection accuracy includes target tracking rate, detection rate, association rate, detection recall rate, detection accuracy rate, association recall rate, association accuracy rate and classification error rate; the data frequency stability rate includes frequency deviation, packet loss rate, number of disconnections, total disconnection duration, maximum time interval error and time deviation.

[0029] Optionally, an APP is provided on the user terminal, and the test task is displayed on the APP. The user views the test task and operates to generate user instructions which are then sent to the true value inspection vehicle.

[0030] Optionally, the inspection test platform includes a test analysis module, an analysis algorithm module, a management module, a manual review module and a terminal background support module; wherein the test analysis module is used to analyze the test data to generate a test report.

[0031] The innovative features of the embodiments of the present application include:

[0032] The embodiment of the present application discloses a patrol test platform for a vehicle-road cooperative roadside fusion perception system. The patrol test platform communicates with a user terminal and a true patrol vehicle, and is used to create a test task for testing the vehicle-road cooperative roadside fusion perception system, and sends it to the user terminal; and receives the test data generated by the true patrol vehicle executing the test task in the test area according to the user instructions generated by the user terminal, and analyzes the test data to generate a test report. The present application installs various types of sensors on a real patrol vehicle to obtain more comprehensive test data. Therefore, various evaluation indicators of the vehicle-road cooperative roadside fusion perception system can be comprehensively tested, and three major indicators of target recognition accuracy, target detection accuracy and data frequency stability can be calculated; at the same time, it has the flexibility of a mobile sensor method, and can collect true value data and perception data in the same time and space in real time, and generate a test report for this patrol task after the patrol task is completed. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work.

[0034] Figure 1 A schematic diagram of an inspection test platform for a vehicle-road cooperative roadside fusion perception system provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of a user terminal is provided for an embodiment of the present application;

[0036] Figure 3 A schematic diagram of a true value inspection vehicle provided in an embodiment of the present application;

[0037] Figure 4 A schematic diagram of a process for generating a test report provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0040] The embodiment of the present application discloses a patrol test platform for a vehicle-road cooperative roadside fusion perception system, which communicates with a user terminal and a true value patrol vehicle, and is used to create a test task for testing the vehicle-road cooperative roadside fusion perception system, and sends it to the user terminal; and receives test data generated by the true value patrol vehicle executing the test task in the test area according to the user instructions generated by the user terminal, and analyzes the test data to generate a test report, which includes evaluation indicators of the vehicle-road cooperative roadside fusion perception system. The present application installs multiple laser radars and RTK combined inertial navigation and other sensors on the true value patrol vehicle to obtain more comprehensive test data, so it can comprehensively test various evaluation indicators of the vehicle-road cooperative roadside fusion perception system, and can calculate three categories of indicators: target recognition accuracy, target detection accuracy, and data frequency stability; at the same time, it has the flexibility of a mobile sensor method, and can collect true value data and perception data in the same time and space in real time, and generate a test report for this patrol task after the patrol task is completed.

[0041] The embodiments of the present application are described in detail below.

[0042] Figure 1A schematic diagram of the inspection test platform for the vehicle-road collaborative roadside fusion perception system provided by the embodiment of the present application is shown. The test and analysis process of the inspection test platform is mainly: 1. Task creation, 2. Data collection, 3. Data analysis, 4. Report update. After the inspection cloud platform registers and accesses the true value inspection vehicle and obtains the reported test data in real time, the inspection test platform registers and accesses the vehicle-road collaborative roadside fusion perception system being tested, obtains the reported data in real time, and analyzes the frequency stability of the data in real time. The inspection test platform creates a test task, and the tester drives the true value inspection vehicle to complete the test action in the test area to obtain the test data. The test data includes the surrounding perception target data of the true value inspection vehicle in the same time and space, the high-precision trajectory data of the true value inspection vehicle itself, and the perception target data of the vehicle-road collaborative roadside fusion perception system being tested. The inspection cloud platform calculates the evaluation indicators of the vehicle-road collaborative roadside fusion perception system based on the grid map in the test area.

[0043] The present application provides an inspection test platform for a vehicle-road cooperative roadside fusion perception system, the inspection test platform comprising an inspection cloud platform deployed on a cloud server, the inspection cloud platform communicating with a user terminal and a true value inspection vehicle;

[0044] The inspection cloud platform is used to create test tasks for testing the vehicle-road collaborative roadside fusion perception system and send them to the user end; wherein, the inspection test platform includes a test analysis module, an analysis algorithm module, a management module, a manual review module and a terminal background support module; wherein, the test analysis module is used to analyze the test data and generate a test report.

[0045] refer to Figure 1 The inspection cloud platform is deployed on the cloud server. Its functions include a large screen for displaying test information, a module management system (data, equipment, reports and system management), test analysis and calculation, manual test review and proofreading, and backend support for user terminals. The platform provides users with a basic web interaction space. The interactive part supports ① monitoring the health information of all current maintenance points, ② managing asset information such as test points and test equipment, ③ creating tracking inspection test tasks, and ④ viewing test reports and review report contents.

[0046] The user terminal is used to execute the test task and generate user instructions to be sent to the true value inspection vehicle;

[0047] Among them, an APP is set on the user terminal, and the test task is displayed on the APP. The user views the test task and operates to generate user instructions which are sent to the true value inspection vehicle.

[0048] Both the user end and the real-value inspection vehicle of this application have the participation of users in the inspection test. The user operation of the inspection test is mainly divided into five steps:

[0049] In the asset management system of the patrol cloud platform, register the true value patrol vehicle and the tested vehicle-road cooperative roadside fusion perception system, confirm that the data of the true value patrol vehicle and the tested system are reported to the patrol cloud platform gateway normally, and the time synchronization status is normal; create a patrol test task in the patrol cloud platform, select the vehicle-road cooperative roadside fusion perception system to be tested in the task, and complete the task issuance; power on the true value patrol vehicle and drive it to the designated test area, bind the registered true value patrol vehicle to the test task created in the previous step in the user terminal APP and start test data collection; drive the true value patrol vehicle along the designated route to perform the test task. During the test, monitor the data status and test progress in real time through the user terminal APP, and complete the test task according to the user terminal instructions. Check and review the test results in the patrol cloud platform and generate a test report.

[0050] refer to Figure 2 The user terminal is an APP based on Android devices. The software is mainly used for test task execution, test information viewing and real-time test information interaction. It supports controlling the start and end of test tasks, supports viewing the test data reporting status, and viewing the time synchronization status. It can display the true value inspection vehicle position, its surrounding perception data and test progress heat map in real time during the test.

[0051] The true value inspection vehicle is used to perform the test task in the test area according to the user's instructions, thereby generating test data, and reporting the test data to the inspection cloud platform;

[0052] Among them, the true value inspection vehicle includes a timing device, multiple sensors, a switch, an acquisition controller and a network module; the timing device includes a time synchronization server for synchronizing the time of each device on the true value inspection vehicle, the multiple sensors are used to collect status data of traffic participants in the test area, and use the status data as test data, and report it to the inspection cloud platform through the network module; the switch is used to exchange data between the acquisition controller and the network module; the acquisition controller is used to control the multiple sensors to collect data.

[0053] In specific practice, the main working process of the true value inspection vehicle is:

[0054] It performs self-check at power on, corrects the local time based on the standard time source, and reports its own status information to the inspection cloud platform. It collects the high-precision trajectory data of the true inspection vehicle in real time through RTK combined inertial navigation, collects original point cloud data through lidar, and fuses multiple lidar point clouds based on calibration information to perform perception calculations on the fused point cloud in real time, and obtains the target data perceived by the true inspection vehicle all around it in combination with the high-precision trajectory data of its own vehicle. It reports the local time synchronization status to the inspection cloud platform at a frequency of 1Hz, reports the target data perceived by the true inspection vehicle all around it to the inspection cloud platform at a frequency of 10Hz, and reports the high-precision trajectory data of the true inspection vehicle itself to the inspection cloud platform at a frequency of 100Hz.

[0055] Combination Figure 3 and Figure 4 The true value inspection vehicle is modified with a passenger car as the carrier. In addition to the body, it also contains a time synchronization server, sensors, network modules and computing units. The sensors carried include multi-lidar and RTK combined inertial navigation. As a true value collection tool for the inspection system, the true value inspection vehicle collects high-precision time series data of traffic participants on the test section through its own sensors and perception computing capabilities, and reports it to the inspection cloud platform gateway in real time. The functions of the device include: ① Synchronizing device time based on the time synchronization server, ② Collecting RTK positioning data, ③ Collecting multi-lidar data and performing real-time fusion perception computing, and ④ Data reporting.

[0056] Combination Figure 3 and Figure 4 The true inspection vehicle is equipped with a high-precision combined inertial navigation and perception kit to collect vehicle motion data and data of traffic participants around the vehicle respectively. In order to meet the real-time and high-precision requirements of the inspection test, the perception computing model trained two sets of models, namely a small model with fast response and a large model with high precision and heavy resources. The small model with fast response is deployed on the true inspection vehicle to guide the tester to complete the test online. The perception model performs real-time perception calculation during the test, and reports the perception results (data of traffic participants around the vehicle) to the cloud platform in real time through 5G-CPE. The cloud platform roughly calculates the number of matching TP samples in each test grid based on the real-time perception data, and guides the tester in the user terminal to collect enough TP samples for analysis in each grid, so that the number of TP samples in each grid is not less than 5 / m 2 (The so-called rough calculation means omitting data cleaning, not interpolating data during time alignment, not using full data for spatial matching, and only using the data from the latest 5 seconds to improve calculation efficiency and meet real-time requirements)

[0057] The test data includes: the true value target data perceived by the true value patrol vehicle in the same time and space, the motion data of the true value patrol vehicle, and the target data to be tested perceived by the tested vehicle-road cooperative roadside fusion perception system.

[0058] It is worth noting that the collection of the motion data of the true inspection vehicle is determined by the high-precision combined inertial navigation and the vehicle's own attributes. The high-precision combined inertial navigation collects the vehicle's position information (longitude, latitude), heading angle information and vehicle speed information at 100hz. The appearance of the vehicle determines the size of the vehicle (length, width, height) and vehicle model information. The true target data is the data of traffic participants around the vehicle, which is generated by the perception kit. The original data is first captured by lidar and visual sensors, and then the target type, target position, target speed, target heading, target size and other information of traffic participants around the vehicle are obtained through fusion perception, and then the target identification id is obtained through the tracking algorithm (the same target in the previous and next frames should keep the same id). The target data to be tested is the traffic participant data generated by the tested vehicle-road cooperative roadside fusion perception system, and the data is obtained through the cloud-to-cloud protocol.

[0059] The inspection cloud platform is used to analyze the test data and generate a test report, wherein the test report includes evaluation indicators of the vehicle-road cooperative roadside fusion perception system.

[0060] The high-precision and resource-intensive large model is deployed on the patrol cloud platform to serve the final indicator calculation. After the test, the perception model re-perceives the original sensor data collected by the perception kit on the patrol real-value vehicle to generate high-precision perception results. Due to its high-precision requirements, the model has a large demand for computing resources and computing time, so it needs to be deployed on the patrol cloud platform and cannot be run in real time on the patrol real-value vehicle.

[0061] This application combines evaluation indicators to determine whether the operating status of the tested vehicle-road cooperative roadside fusion perception system is good, forms a test report, and outputs the health status.

[0062] The inspection test platform of the vehicle-road cooperative roadside fusion perception system in this application can inspect the vehicle-road cooperative roadside fusion perception system, and use a fast and highly automated method to detect the target recognition accuracy, target detection accuracy and data frequency stability of the perception system under test, determine whether its working operation status is good, and maintain the health information of the point.

[0063] In a specific embodiment of the present application, reference Figure 4 The inspection cloud platform is used to:

[0064] Step a, performing truth cleaning on the true value target data and the motion data to obtain cleaned target data and motion data;

[0065] Step b, performing spatiotemporal alignment matching on the target data to be tested and the true target data after cleaning to obtain the target data after spatiotemporal alignment processing;

[0066] Step c, clearing the time-space aligned target data and the data in the cleaned inspection data that are not in the test area to obtain processed test data; and using the processed test data to analyze the evaluation indicators of the vehicle-road cooperative roadside fusion perception system.

[0067] In a specific implementation of the present application, the true value target data and the true value data are cleaned to obtain the cleaned true value target data and motion data, including:

[0068] a1, deleting the motion data with precision lower than the precision threshold in the motion data and deleting the target data with confidence lower than the confidence threshold in the true target data, to obtain the initially cleaned motion data and the true target data;

[0069] This application deletes data with too low precision according to the data precision in the vehicle motion data (if there is none, it is derived based on the differential state), and retains data with precision < 0.05m, as shown in the following table:

[0070] Differential state explain Data accuracy Fixed solution Accurately resolve differential signals <0.03m Floating point solution Inaccurately resolved differential signal <0.1m Single point solution Unable to resolve differential signals <1m

[0071] Then, based on the perceived confidence of traffic participants around the vehicle, the data with too low confidence is deleted, and the data with confidence > 80% is retained.

[0072] a2, the reliability of the current frame data is estimated based on the previous and next frame data, and the outlier data that violates the physical relationship in the initially cleaned motion data and the true value target data is removed to obtain the cleaned inspection data.

[0073] This step calculates the reliability of the current frame data based on the previous and next frame data, removes outlier data that violates physical relationships, and calculates acceleration based on the second-order derivative of position, which should be less than 10m / s 2 Value; Calculate the angular acceleration based on the second-order derivative of the heading angle, which should be less than 1 rad / s 2 value.

[0074] In a specific implementation of the present application, the target data to be tested is subjected to spatiotemporal alignment matching with the cleaned true value target data, and the target data after spatiotemporal alignment processing is obtained, including:

[0075] b1, interpolating the cleaned true value target data and the cleaned motion data based on the time of the target data to be measured to obtain the interpolated true value target data and motion data;

[0076] Since both the true value and the data to be measured are discrete data, that is, the true value target data and the target data to be measured, time alignment is required, and the time series data of the true value is interpolated based on the data time to be measured, using cubic curve interpolation or Kalman filtering. After the speed information is interpolated, values ​​less than 0 need to be set to zero.

[0077] b2, calculating the trajectory matching degree between the interpolated true value target data and the target data to be measured, and obtaining the matching relationship between the interpolated true value target data and the target data to be measured based on the mutually exclusive principle of trajectory matching degree and target unique matching;

[0078] b3, label the matched true value target data and the target data to be tested with TP labels, label the unmatched target data to be tested with FP labels, and label the unmatched true value target data with FN labels;

[0079] It is necessary to match the true target and the target to be measured, and calculate the subsequent indicators based on the matching results. The matching method is the trajectory matching of ds demonstration. The trajectory matching degree is calculated based on the average positioning error, average speed error, average heading error and average size error of each target trajectory. Based on the mutually exclusive principle of trajectory matching degree and target unique matching, the matching relationship between each target is obtained. The matched true target and the target to be measured are recorded as TP labels, the unmatched target to be measured is recorded as FP labels, and the unmatched true target is recorded as FN labels.

[0080] b4, remove the true value target data carrying the label and the data that is not in the test area in the target data to be tested, and obtain the target data after spatiotemporal alignment processing.

[0081] According to the above spatial matching results and the test range ROI (polygon composed of multiple location points), all data that is not in the test area ROI and is not a TP tag is cleared. Here, the spatial matching information is combined, and the data outside the test range is not directly cleared to ensure that traffic participants will not cause indicator calculation errors due to data sampling errors when entering and leaving the test range.

[0082] In a specific implementation of the present application, the evaluation indexes of the vehicle-road cooperative roadside fusion perception system analyzed by using the processed test data include:

[0083] The grid map in the test area is used to calculate the evaluation index of the vehicle-road cooperative roadside fusion perception system, and the evaluation index includes a rasterization index and an overall index. Both the rasterization index and the overall index include the target recognition accuracy, the target detection accuracy and the data frequency stability rate. If the target data carrying the TP tag in a grid does not reach the predetermined quantity index, the rasterization index is invalid. The overall index uses the number of samples of the target data carrying the TP tag in each grid as the weighted average value.

[0084] It is worth noting that the test area ROI is grid-segmented based on the grid gap (usually 2m), and each true value test match pair (TP) is assigned to a grid based on its true value position. The number of TP samples in each grid needs to be greater than 5 / m2 , otherwise the indicator statistics of the grid are recorded as invalid. The evaluation indicators of target recognition accuracy and target detection accuracy are calculated separately for each grid. The overall indicator is the average value with the number of TP samples of each grid as the weight.

[0085] 1. The target recognition accuracy includes positioning accuracy, positioning standard deviation, speed accuracy, speed standard deviation, heading angle accuracy, heading angle standard deviation, size accuracy and size standard deviation, which are respectively expressed as:

[0086] (1) Positioning accuracy: The average value of the position distance of all matched true value target pairs (TP tags) in the grid / overall data, expressed as:

[0087]

[0088] Where n is the number of true value target pairs matched in the grid, d i is the distance of the i-th pair.

[0089] (2) Positioning standard deviation: The standard deviation of the position distance of all matched true value target pairs (TP tags) in the grid / overall data, expressed as:

[0090]

[0091] (3) Velocity accuracy: The average velocity error of all matched true target pairs (TP tags) in the grid / overall data.

[0092]

[0093] In the formula, v i is the velocity error of the i-th pair.

[0094] (4) Speed ​​standard deviation: The speed error standard deviation of all matched true value target pairs (TP tags) in the grid / overall data, expressed as:

[0095]

[0096] (5) Heading angle accuracy: The average heading angle error of all matched true value target pairs (TP tags) in the grid / overall data, expressed as:

[0097]

[0098] In the formula, θ i is the heading angle error of the i-th pair.

[0099] (6) Heading angle standard deviation: The heading error standard deviation of all matched true value target pairs (TP tags) in the grid / overall data, expressed as:

[0100]

[0101] (7) Size accuracy: The average size error of all matched true value target pairs (TP labels) in the grid / overall data, expressed as:

[0102]

[0103] In the formula, s i is the size error of the i-th pair.

[0104] (8) Size standard deviation: The size error standard deviation of all matched true value target pairs (TP labels) in the grid / overall data, expressed as:

[0105]

[0106] 2. The target detection accuracy will calculate the sub-indicators of each type according to the type of traffic participants, including motor vehicles, non-motor vehicles and pedestrians. When calculating the sub-type indicators, all traffic participant targets of non-types will be cleared, and only traffic participant targets of the current type will be retained. Including target tracking rate, detection rate, association rate, detection recall rate, detection accuracy rate, association recall rate, association accuracy rate and classification error rate, the sub-table is expressed as follows:

[0107] (1) HOTA (high-order multi-target tracking rate), expressed as:

[0108]

[0109]

[0110] Where c refers to a TP matching relationship, and TPA(c), FNA(c), and FPA(c) are defined as follows:

[0111] TPA(c)={k|k∈TP|prID(k)=prID(c)∧gtID(k)=gtID(c)};

[0112] FNA(c)={k∣k∈{TP∣prID(k)≠prID(c)∧gtID(k)

[0113] =gtID(c)}∪{FN∣gtID(k)=gtID(c)}}

[0114] FPA(c)={k∣k∈{TP∣prID(k)=prID(c)∧gtID(k)

[0115] ≠gtID(c)}∪{FP|prID(k)=prID(c)}}

[0116] TP is the number of true value pairs to be tested that are matched, FP is the number of unmatched test targets, and FN is the number of unmatched true value targets.

[0117] (2) DetA (detection rate), expressed as:

[0118]

[0119] (3) AssA (association rate), expressed as:

[0120]

[0121] (4) DetRe (detection recall rate), expressed as:

[0122]

[0123] (5) DetPr (detection accuracy), expressed as:

[0124]

[0125] (6) AssRe (association recall), expressed as:

[0126]

[0127] (7) AssPr (association accuracy), expressed as:

[0128]

[0129] (8) IER (classification error rate) is the proportion of type errors in all matched true value target pairs (TP labels) in the grid / overall data, expressed as:

[0130]

[0131] Where e is the number of type errors, and n is the total number of true value target pairs that are matched.

[0132] 3. The data frequency stability rate does not rely on the true value test, but is directly analyzed based on the test data reported by the tested vehicle-road cooperative roadside fusion perception system. It includes frequency deviation, packet loss rate, number of disconnections, total disconnection duration, maximum time interval error and time deviation, which are expressed as:

[0133] (1) Frequency deviation, expressed as:

[0134]

[0135] Δf=|f actual -f standard |

[0136] Among them, f standard is the system standard output frequency, f actual is the actual system output frequency, N actual The amount of data received by the actual system within the target time period, where T is the length of the target time period.

[0137] (2) Packet loss rate, expressed as:

[0138] N expected =f standard ×T

[0139]

[0140] Where N expected is the amount of data that the system expects to receive within the target time period.

[0141] (3) Disconnection times: the number of times the preceding and following frames of data are greater than the disconnection determination threshold (usually 10 seconds)

[0142]

[0143] The value is 1 if the condition is true, otherwise 0.

[0144] (4) Total disconnection duration: the total duration of the data frames before and after being greater than the disconnection judgment threshold (usually 10s), expressed as:

[0145]

[0146] (5) Maximum time interval error: The maximum value of the peak-to-peak value of the delay between the timing signal time interval output by the roadside perception system and the ideal time interval for a given observation interval τ = nτ0 during the measurement period with a total length of T. It represents the frequency stability and is expressed as:

[0147]

[0148] Among them, x i is the instantaneous time error, N is the total number of samples, and τ0 is the sampling interval.

[0149] (6) Time deviation is a measure of the change in the signal time interval of the roadside perception system's timing output information within a given integration time τ, which can provide information about the signal phase and spectral content. If the sampling time of the system is τ0 and the observation interval is τ = nτ0, then the corresponding definition of time deviation is:

[0150]

[0151] Among them, <·> is the set average (ensemble average). When the sampling time of the system is τ0 and the observation interval is τ=nτ0, the time deviation can be estimated according to the following formula:

[0152]

[0153]

[0154] Where [·] is rounded down, τ is the integration time, and n is the number of sampling intervals within the integration time.

[0155] The embodiment of the present application discloses a patrol test platform for a vehicle-road cooperative roadside fusion perception system. The patrol test platform communicates with a user terminal and a true patrol vehicle, and is used to create a test task for testing the vehicle-road cooperative roadside fusion perception system, and sends it to the user terminal; and receives the test data generated by the true patrol vehicle executing the test task in the test area according to the user instructions generated by the user terminal, and analyzes the test data to generate a test report. The present application installs various types of sensors on a real patrol vehicle to obtain more comprehensive test data. Therefore, various evaluation indicators of the vehicle-road cooperative roadside fusion perception system can be comprehensively tested, and three major indicators of target recognition accuracy, target detection accuracy and data frequency stability can be calculated; at the same time, it has the flexibility of a mobile sensor method, and can collect true value data and perception data in the same time and space in real time, and generate a test report for this patrol task after the patrol task is completed.

[0156] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present application.

[0157] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A patrol test platform for a vehicle-road cooperative roadside fusion perception system, characterized in that: The inspection test platform includes an inspection cloud platform deployed on a cloud server, and the inspection cloud platform communicates with a user terminal and a true value inspection vehicle; The inspection cloud platform is used to create a test task for testing the vehicle-road cooperative roadside fusion perception system and send it to the user end; The user terminal is used to execute the test task and generate user instructions to be sent to the true value inspection vehicle; The true value inspection vehicle is used to perform the test task in the test area according to the user's instructions, thereby generating test data, and reporting the test data to the inspection cloud platform; The inspection cloud platform is used to analyze the test data and generate a test report, wherein the test report includes evaluation indicators of the vehicle-road cooperative roadside fusion perception system.

2. The inspection test platform for the vehicle-road cooperative roadside fusion perception system according to claim 1 is characterized in that: The true value inspection vehicle includes a timing device, multiple sensors, a switch, an acquisition controller and a network module; the timing device includes a time synchronization server for synchronizing the time of each device on the true value inspection vehicle, and the multiple sensors are used to collect the status data of traffic participants in the test area, and use the status data as test data to report to the inspection cloud platform through the network module; The switch is used to exchange data between the acquisition controller and the network module; the acquisition controller is used to control the multiple sensors to collect data.

3. The inspection test platform for the vehicle-road cooperative roadside fusion perception system according to claim 1 is characterized in that: The test data includes: the true value target data perceived by the true value patrol vehicle in the same time and space, the motion data of the true value patrol vehicle, and the target data to be tested perceived by the tested vehicle-road cooperative roadside fusion perception system.

4. The inspection test platform for the vehicle-road cooperative roadside fusion perception system according to claim 3 is characterized in that: The inspection cloud platform is used for: Performing truth value cleaning on the true value target data and the motion data to obtain cleaned target data and motion data; Performing spatiotemporal alignment matching on the target data to be tested and the cleaned true value target data to obtain target data after spatiotemporal alignment processing; Clearing the time-space aligned target data and the data not in the test area in the cleaned inspection data to obtain processed test data; The processed test data is used to analyze the evaluation indicators of the vehicle-road cooperative roadside fusion perception system.

5. The inspection test platform for the vehicle-road cooperative roadside fusion perception system according to claim 4 is characterized in that: The true value target data and the true value data cleaning to obtain the cleaned true value target data and motion data include: Deleting the motion data whose precision is lower than the precision threshold in the motion data and deleting the target data whose confidence is lower than the confidence threshold in the true value target data, to obtain the initially cleaned motion data and the true value target data; The reliability of the current frame data is inferred based on the previous and next frame data, and the outlier data that violates the physical relationship in the initially cleaned motion data and true value target data is removed to obtain the cleaned inspection data.

6. The inspection test platform for the vehicle-road cooperative roadside fusion perception system according to claim 4 is characterized in that: Performing spatiotemporal alignment matching on the target data to be tested and the true target data after cleaning, and obtaining the target data after spatiotemporal alignment processing includes: Interpolate the cleaned true value target data and the cleaned motion data based on the time of the target data to be measured to obtain interpolated true value target data and motion data; Calculating the trajectory matching degree between the interpolated true value target data and the target data to be measured, and obtaining the matching relationship between the interpolated true value target data and the target data to be measured based on the mutually exclusive principle of trajectory matching degree and target unique matching; The matched true value target data and the target data to be tested are labeled with TP, the unmatched target data to be tested are labeled with FP, and the unmatched true value target data are labeled with FN; The true value target data carrying the label and the data not in the test area in the target data to be tested are removed to obtain the target data after the spatiotemporal alignment processing.

7. The inspection test platform for the vehicle-road cooperative roadside fusion perception system according to claim 4 is characterized in that: The evaluation index of analyzing the vehicle-road cooperative roadside fusion perception system using the processed test data includes: The grid map in the test area is used to calculate the evaluation index of the vehicle-road cooperative roadside fusion perception system, and the evaluation index includes a rasterization index and an overall index. Both the rasterization index and the overall index include target recognition accuracy, target detection accuracy and data frequency stability rate. If the target data carrying TP tags in a grid does not reach the predetermined quantity index, the rasterization index is invalid. The overall index uses the number of samples of target data carrying TP tags in each grid as the weighted average value.

8. The inspection test platform for the vehicle-road cooperative roadside fusion perception system according to claim 7 is characterized in that: The target recognition accuracy includes positioning accuracy, positioning standard deviation, speed accuracy, speed standard deviation, heading angle accuracy, heading angle standard deviation, size accuracy and size standard deviation; the target detection accuracy includes target tracking rate, detection rate, association rate, detection recall rate, detection accuracy rate, association recall rate, association accuracy rate and classification error rate; the data frequency stability rate includes frequency deviation, packet loss rate, number of disconnections, total disconnection duration, maximum time interval error and time deviation.

9. The inspection test platform for the vehicle-road cooperative roadside fusion perception system according to claim 1 is characterized in that: An APP is provided on the user terminal, and the test task is displayed on the APP. The user views the test task and operates to generate user instructions which are then sent to the true value inspection vehicle.

10. The inspection test platform for the vehicle-road cooperative roadside fusion perception system according to claim 1 is characterized in that: The inspection test platform includes a test analysis module, an analysis algorithm module, a management module, a manual review module and a terminal background support module; wherein the test analysis module is used to analyze the test data and generate a test report.

Images