Perception Test Method for Traffic Events by V2I-Based Roadside Perception System

Abstract

The present invention discloses a perception test method for traffic events by a roadside perception system based on V2I, belonging to the field of vehicle networking. The perception test method includes the following steps: S1: Test area design and selection; S2: Start the test process and data collection; S3: Conduct test data analysis and evaluation. Compared with the prior art, the beneficial effects of the present invention are as follows: In an open road, by designing different intersection types and test areas, using the data of the vehicle-mounted ground truth system as a benchmark, traffic events in the road (such as speeding, low speed, illegal lane change, illegal parking, reverse driving, etc.) are perceived, and the kinematic perception performance of the system to be tested can be evaluated with time error, which is more comprehensive and accurate.

Description

Technical Field

[0001] The present invention belongs to the field of vehicle networking, and particularly relates to a method for perceiving and testing traffic events by a roadside perception system based on V2I. Background Art

[0002] Vehicle-road cooperation adopts advanced wireless communication and new-generation Internet technologies to comprehensively implement dynamic real-time information interaction between vehicles and between vehicles and roads. Based on the collection and fusion of all-time and all-space dynamic traffic information, it conducts vehicle active safety control and road cooperation management, fully realizing the effective cooperation of people, vehicles, and roads, ensuring traffic safety, improving traffic efficiency, and thus forming a safe, efficient, and environmentally friendly road traffic system.

[0003] The Roadside Sensing System (RSS) is an important means to support connected autonomous driving, improve traffic operation efficiency, and relieve congestion. Providing information such as beyond-line-of-sight perception, blind spot warning, and driving intention for autonomous vehicles through the RSS system is one of the important technical means to make up for the perception limitations of single-vehicle autonomous driving.

[0004] In vehicle-road cooperation applications, the roadside perception system realizes the real-time vectorization and tracking of global goals, and its accurate perception ability is the key to the roadside perception system.

[0005] Based on this, there are various existing solutions for evaluating the performance of the roadside perception system in the prior art.

[0006] For example, the existing patent with the publication number CN114383649A discloses a method for testing a roadside perception system based on high-precision positioning, including a test system and a roadside perception system. The test system includes a mobile carrier. The test system obtains traffic participant information with the mobile carrier as a reference, and outputs the reference state information of the traffic participant information by processing the obtained traffic participant information; the roadside perception system obtains the to-be-tested state information of the traffic participant; compares and analyzes the reference state information and the to-be-tested state information, calculates the error between the reference state information and the to-be-tested state information, and gives a performance evaluation report of the roadside perception system according to the calculated error.

[0007] For another example, the existing patent with the publication number CN112816954A discloses a method for evaluating a roadside perception system based on ground truth, which includes the following steps: establishing a ground truth perception device group, and during a selected test time interval, synchronously collecting roadside perception data with the perception devices of the roadside perception system RSS to be tested; processing the original data transmitted back by the ground truth perception device group, completing target type recognition and target trajectory recognition, and completing perception data annotation; generating ground truth based on the annotated data, and the ground truth data includes traffic participant target types, positions, speeds, accelerations, and trajectories; during the selected test time interval, comparing the structured perception data output by the RSS to be tested and the ground truth data, and outputting statistical evaluation results of the perception performance.

[0008] For another example, the existing patent with the publication number CN112382079A discloses a method and system for simulating and emulating roadside perception for vehicle-road collaboration, which provides a virtual environment for simulation testing before actual road testing of roadside perception. With the aid of the simulation environment, the relationship between the sensor and the environment before actual installation can be analyzed, and the corresponding working effects can be visually displayed. In addition, the training data and test data required for machine learning can be directly output through the simulation system.

[0009] The above patents can all test and evaluate the traffic participant perception ability of the roadside perception system in vehicle-road collaboration. However, the object of their evaluation is the overall traffic participants, including traffic targets, traffic events, traffic flows, etc., and it is not specifically for the evaluation of the perception ability of traffic events. At the same time, the above patents also mainly use conventional roadside perception system testing methods, and the considerations for different intersection types and test areas are not detailed and perfect enough. Summary of the Invention

[0010] To solve the above problems, the primary object of the present invention is to provide a method for testing the perception of traffic events by a roadside perception system based on V2I. In an open road, by designing different intersection scenarios and test areas, using the data of the vehicle-mounted ground truth system as a benchmark, the kinematic perception performance of the system to be tested for traffic events can be evaluated more accurately and comprehensively.

[0011] To achieve the above object, the technical solution of the present invention is as follows:

[0012] A method for testing the perception of traffic events by a roadside perception system based on V2I, the perception testing method includes the following steps:

[0013] S1: Test area design and selection: In an open road, according to different test scenarios, the test scenarios are divided into two categories: long straight roads and holographic intersections, and different types of test areas are respectively set, and points are recorded and numbered within the test area; then a two-lane range is selected as the main event triggering area;

[0014] S2: Start the test process and data collection: Select a ground truth vehicle to conduct a driving test within the test area. The ground truth vehicle records its own ground truth data, and the system under test performs target perception on the ground truth vehicle in the test area and outputs structured perception data.

[0015] S3: Conduct test data analysis and evaluation.

[0016] Further, the long straight road in step S1 is also called the "ordinary intersection".

[0017] Further, in step S1, the set test area of the long straight road is a quadrilateral area, and 6 points are recorded and numbered, including 4 rectangle vertices and 2 cross-section points; there are two types of rectangle areas. One uses the road boundary as the area boundary, that is, it includes all lanes, and the other test area only includes some lanes; then select the two-lane range as the main event trigger area.

[0018] Further, in step S1, the set test area of the holographic intersection is a cross-shaped area, and 32 points are recorded and numbered. Considering the actual intersection construction situation, the number of these points can be determined by itself. If a separate cross-section is set, 8 additional points need to be recorded; then select the two-lane range as the main event trigger area.

[0019] Further, step S2 includes:

[0020] Step S21: The range of the main event trigger area. Record the longitude and latitude information of each point in the order of "southeast - east - northeast - north - northwest - west - southwest - south - southeast".

[0021] Step S22: Select at least 3 different types of ground truth vehicles, with at least 1 vehicle of each type, and all ground truth vehicles are equipped with RTK inertial navigation systems.

[0022] Step S23: Select a place outside the perception range of the system under test as the starting point of the ground truth vehicle. The ground truth vehicle starts the test from the starting point, and the test personnel start the ground truth vehicle's own data collection system to record the ground truth data of the vehicle's own driving.

[0023] Step S24: The ground truth vehicle executes various traffic events to complete the trigger test.

[0024] Step S25: The ground truth vehicle leaves the test area and stops the ground truth data collection of its own vehicle.

[0025] Step S26: Unify the positioning reference points of the roadside ground truth system and the system under test; then process the ground truth data collected by the ground truth vehicle and the perception data of the system under test on the ground truth vehicle, and both output structured data.

[0026] Further, in step S23, the truth vehicle is outside the sensing range of the system under test, and the distance from the position of the system under test is required to be more than 400 meters.

[0027] Further, in step S24, the truth vehicle enters from the starting point entrance direction along the lane and passes through the mainly defined event triggering area, and various traffic event operations are completed within this area; when the tester enters and leaves the mainly defined event triggering area, the self-vehicle data acquisition system records and marks the timestamps.

[0028] Further, in step S24, various traffic events include speeding, low speed, illegal lane change, illegal parking, reverse driving, etc.

[0029] Further, in step S24, the number of test times for each type of vehicle to complete each traffic event is more than 10 times.

[0030] Further, in step S25, after the truth vehicle leaves the test area, it needs to drive an additional distance of at least 100 meters, and then the tester stops the self-vehicle data acquisition of the truth vehicle to ensure the integrity of the truth data.

[0031] Further, in step S26, the structured data output by the truth vehicle:

[0032] Tstart, the start timestamp of acquisition;

[0033] T1 and T2, the timestamps of entering and leaving the mainly defined event triggering area;

[0034] Tend, the end timestamp of acquisition;

[0035] T△, the event duration timestamp, T△ = T2 - T1;

[0036] Typegt, the event type;

[0037] Correspondingly, the structured data output by the system under test is Dstart, D1 and D2, Dend, D△, Typedut.

[0038] Further, in step S3, based on the structured data output by the truth vehicle and the system under test, the following evaluations are carried out for the tests of each traffic event type:

[0039] (1) Condition 1: Compare whether the Type values are the same, that is, Typegt = Typedut;

[0040] (2) Condition 2: Compare whether T1 and D1 are the same, allowing a time error within 2s, that is, |T1 - D1| ≤ 1.5s;

[0041] (3) Condition 3: Compare whether T2 and D2 are consistent, allowing a time error within 2 s, i.e., |T2 - D2| ≤ 1.5 s;

[0042] (4) Condition 4: Compare whether T△ and D△ are consistent, allowing a time error within 2 s, i.e., |T△ - D△| ≤ 2 s;

[0043] Only when the above 4 conditions are met simultaneously can it be determined that the test of this event in this instance passes; otherwise, it is determined as failed, and the number of failed times is recorded.

[0044] Manually record the number of times Si for each test and the total number of tests S, and record the number of times Ci that each test of the system under test passes and the total number of passed tests C. Here, i represents the value of type, i.e., the event type.

[0045] According to the statistics, calculate the traffic event recognition detection rate:

[0046] For each item, αi = Ci / Si,

[0047] In total, α = C / S.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows: In an open road, by designing different intersection types and test areas, using the data of the vehicle-mounted ground truth system as a benchmark, traffic events (such as speeding, low speed, illegal lane change, illegal parking, reverse driving, etc.) on the road are perceived, and the kinematic perception performance of the system under test can be evaluated with a time error, which is more comprehensive and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is the test flow chart of the present invention.

[0050] Figure 2 is the test area diagram of the long straight road of the present invention.

[0051] Figure 3 is the test area diagram of the holographic intersection of the present invention.

[0052] Figure 4 is the diagram of the main event trigger area setting in the test scenario of the long straight road of the present invention.

[0053] Figure 5 is the diagram of the main event trigger area setting in the test scenario of the holographic intersection of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0055] Figures 1-5 As shown, the implementation of the present invention is as follows:

[0056] A method for sensing and testing traffic events by a roadside sensing system based on V2I, the sensing and testing method comprising the following steps:

[0057] S1: Test area design and selection: In an open road, according to different test scenarios, the test scenarios are divided into two categories: long straight roads and holographic intersections, and different types of test areas are set respectively. Points are recorded and numbered within the test area, and then a two-lane range is selected as the main event trigger area;

[0058] S2: Start the test process and data collection: Select a truth vehicle to conduct a driving test within the test area. The truth vehicle records the true data of its own vehicle, and the system to be tested performs target sensing on the truth vehicle in the test area and outputs structured sensing data;

[0059] S3: Conduct test data analysis and evaluation.

[0060] Further, the long straight road in step S1 is also called an "ordinary intersection".

[0061] Further, in step S1, the test area set for the long straight road is a quadrilateral area, 6 points are recorded and numbered, including 4 rectangle vertices and 2 cross-section points, and the cross-section points are on the blue dotted line; There are two types of rectangular areas. One uses the road boundary as the area boundary, that is, it includes all lanes, as shown in Figure 2 A(a) in, and the other test area only includes some lanes, as shown in Figure 2 A(b) in. Select a two-lane range as the main event trigger area, and the event trigger area is marked in the Figure 4 corresponding scenario.

[0062] Further, in step S1, the test area set for the holographic intersection is a cross-shaped area, 32 points are recorded and numbered. Considering the actual intersection construction situation, the number of these points recorded can be determined by oneself. If a separate cross-section is set, 8 additional points need to be recorded. As shown in Figure 4 shown. Select a two-lane range as the main event trigger area, and the event trigger area is marked in the Figure 5 corresponding scenario.

[0063] Further, step S2 includes:

[0064] Step S21: The range of the main event trigger area. Record the longitude and latitude information of each point in the order of "southeast - east - northeast - north - northwest - west - southwest - south - southeast";

[0065] Step S22: Select at least 3 different types of ground truth vehicles, with at least 1 vehicle of each type (such as economy cars, mid-size cars, and premium mid-size cars), and all ground truth vehicles are equipped with RTK inertial navigation systems;

[0066] Step S23: Select a location outside the sensing range of the system to be tested as the starting point of the ground truth vehicle. The ground truth vehicle starts from the starting point for testing, and the tester starts the on-vehicle data acquisition system of the ground truth vehicle to record the ground truth data of the vehicle's travel;

[0067] Step S24: The ground truth vehicle performs various traffic events to complete the trigger test;

[0068] Step S25: The ground truth vehicle leaves the test area and stops the ground truth data acquisition of the vehicle;

[0069] Step S26: Unify the positioning reference points of the roadside ground truth system and the system to be tested; then process the ground truth data collected by the ground truth vehicle and the perception data of the ground truth vehicle by the system to be tested, and both output structured data.

[0070] Further, in step S23, when the ground truth vehicle is outside the sensing range of the system to be tested, the distance from the position of the system to be tested is required to be more than 400 meters.

[0071] Further, in step S24, the ground truth vehicle enters along the lane from the starting point entrance direction and passes through the main event trigger area delimited. Various traffic event operations are completed within this area; when the tester enters and leaves the main event trigger area, the marking timestamps are recorded through the on-vehicle data acquisition system.

[0072] Further, in step S24, various traffic events include speeding, low speed, illegal lane change, illegal parking, reverse driving, etc.

[0073] Further, in step S24, the number of test times for each traffic event for each vehicle type is more than 10 times.

[0074] Further, in step S25, after the ground truth vehicle leaves the test area, it needs to drive an additional distance of at least 100 meters, and then the tester stops the on-vehicle data acquisition of the ground truth vehicle to ensure the integrity of the ground truth data.

[0075] Further, in step S26, the structured data output by the ground truth vehicle:

[0076] Tstart, the acquisition start timestamp;

[0077] T1 and T2, the timestamps for entering and leaving the main event trigger area respectively;

[0078] Tend, the acquisition end timestamp;

[0079] T△, the event duration timestamp, T△ = T2 - T1;

[0080] Typegt, the event type;

[0081] Correspondingly, the structured data output by the system under test is:

[0082] Dstart, the acquisition start timestamp;

[0083] D1 and D2 are the timestamps of entering and leaving the main trigger area of the event respectively;

[0084] Dend, the acquisition end timestamp;

[0085] D△, the event duration timestamp, T△ = T2 - T1;

[0086] Typedut, the event type.

[0087] Type gt and Type dut The value of, and the actual actions of the truth vehicle are as shown in the following table:

[0088]

[0089] Furthermore, in step S3, based on the structured data output by the truth vehicle and the system under test, the following evaluations are carried out for each type of traffic event test:

[0090] (1) Condition 1: Compare whether the Type values are the same, that is, Typegt = Typedut;

[0091] (2) Condition 2: Compare whether T1 and D1 are the same, allowing a time error within 2s, that is, |T1 - D1| ≤ 1.5s;

[0092] (3) Condition 3: Compare whether T2 and D2 are the same, allowing a time error within 2s, that is, |T2 - D2| ≤ 1.5s;

[0093] (4) Condition 4: Compare whether T△ and D△ are the same, allowing a time error within 2s, that is, |T△ - D△| ≤ 2s;

[0094] Only when all the above 4 conditions are met simultaneously can it be determined that the test of this event passes, otherwise it is determined as failed, and the number of failures is recorded.

[0095] Manually record the number of times Si of each test and the total number of tests S, and record the number of times Ci that each test of the system under test passes and the total number of passing tests C. Where i represents the value of type, that is, the event type.

[0096] According to the statistics, calculate the traffic event recognition detection rate:

[0097] Each αi = Ci / Si,

[0098] In total, α = C / S.

[0099] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for perceiving and testing traffic events by a roadside perception system based on V2I, characterized in that, The perception test method includes the following steps: S1: Test area design and selection: In an open road, according to different test scenarios, the test scenarios are divided into two categories: long straight roads and holographic intersections, and different types of test areas are set respectively. Points are recorded and numbered within the test area, and then a two-lane range is selected as the main event trigger area; S2: Start the test process and data collection: Select a ground truth vehicle to conduct a driving test within the test area. The ground truth vehicle records the ground truth data of its own vehicle, and the system under test perceives the ground truth vehicle in the test area and outputs structured perception data; S3: Conduct test data analysis and evaluation; The step S2 includes: Step S21: The range of the main event trigger area, record the latitude and longitude information of each point in the order of "southeast - east - northeast - north - northwest - west - southwest - south - southeast"; Step S22: Select at least 3 different types of ground truth vehicles, with at least 1 vehicle of each type, and all ground truth vehicles are equipped with RTK inertial navigation systems; Step S23: Select a place outside the perception range of the system under test as the starting point of the ground truth vehicle. The ground truth vehicle starts testing from the starting point, and the tester starts the ground truth vehicle's own vehicle data collection system to record the ground truth data of its own vehicle's travel; Step S24: The ground truth vehicle executes various traffic events to complete the trigger test; Step S25: The ground truth vehicle leaves the test area and stops the ground truth data collection of its own vehicle; Step S26: Unify the positioning reference points of the roadside ground truth system and the system under test; then process the ground truth data collected by the ground truth vehicle and the perception data of the ground truth vehicle by the system under test, and both output structured data; In step S26, the structured data output by the ground truth vehicle: Tstart, the acquisition start timestamp; T1 and T2, the timestamps for entering and leaving the main event trigger area; Tend, the acquisition end timestamp; T△, the event duration timestamp, T△ = T2 - T1; Typegt, the event type; Correspondingly, the structured data output by the system under test is Dstart, D1 and D2, Dend, D△, Typedut; In the step S3, based on the structured data output by the ground truth vehicle and the system under test, the following evaluations are carried out for each traffic event type test: (1) Condition 1: Compare whether the Type values are the same, that is, Typegt = Typedut; (2) Condition 2: Compare whether T1 and D1 are the same, allowing a time error within 2s, that is, |T1 - D1| ≤ 1.5s; (3) Condition 3: Compare whether T2 and D2 are the same, allowing a time error within 2s, that is, |T2 - D2| ≤ 1.5s; (4) Condition 4: Compare whether T△ and D△ are the same, allowing a time error within 2s, that is, |T△ - D△| ≤ 2s; Only when all the above 4 conditions are met simultaneously is it determined that the test of this event passes this time, otherwise it is determined to fail, and the number of failures is recorded; Manually record the number of times Si of each test and the total number of tests S, and record the number of times Ci of each test passed by the system under test and the total number of passed tests C, where i represents the type value, that is, the event type; According to statistics, calculate the detection rate of traffic event recognition: For each item, αi = Ci / Si, In total, α = C / S.

2. The method for perceiving and testing traffic events by the roadside perception system based on V2I according to claim 1, characterized in that, In the step S1, the set test area for the long straight road is a quadrilateral area, and 6 points are recorded and numbered, including 4 rectangle vertices and 2 cross-section points.

3. The method for perceiving and testing traffic events by the roadside perception system based on V2I according to claim 1, wherein In the step S1, the set test area for the holographic intersection is a cross-shaped area, and 32 points are recorded and numbered.

4. The method for perceiving and testing traffic events by the roadside perception system based on V2I according to claim 1, wherein, In the step S24, the truth vehicle enters along the lane from the starting point entrance direction and passes through the designated main event trigger area, and various traffic event operations are completed within this area; when the tester enters and leaves the main event trigger area, the marking timestamps are recorded through the on-vehicle data acquisition system of the vehicle itself.

5. The method for perceiving and testing traffic events by the roadside perception system based on V2I according to claim 1, characterized in that In the step S24, various traffic events include speeding, low speed, illegal lane change, illegal parking, reverse driving, etc.

6. The method for perceiving and testing traffic events by the roadside perception system based on V2I according to claim 1, characterized in that, In the step S24, the number of test times for each traffic event completed by each vehicle type is more than 10 times.

7. The method for perceiving and testing traffic events by the roadside perception system based on V2I according to claim 1, characterized in that In the step S25, after the truth vehicle leaves the test area, it needs to drive an additional distance of at least 100 meters, and then the tester stops the on-vehicle data acquisition of the truth vehicle to ensure the integrity of the truth data.

Images