Method for testing roadside fusion sensing end-to-end time delay

Abstract

The invention relates to the technical field of vehicle-road cooperation, and discloses a method for testing roadside fusion sensing end-to-end time delay, which comprises the following steps of: 1, acquiring the initial point coordinate of a true value vehicle, and determining the test driving track of the true value vehicle; 2, enabling the true-value vehicle to start from the initial point, run according to the running track and return to the initial point; 3, track data of the true value vehicle returning to the initial point are collected, and the track data comprise coordinate values and corresponding timestamps obtained in a true track and sensing coordinate values and corresponding timestamps obtained in a sensing equipment coordinate system; and step 4, calculating the end-to-end time delay according to the obtained real track point timestamp and the perception track point timestamp. According to the invention, the visual angle difference of the equipment is effectively calibrated, the actual position of the true-value vehicle can accurately correspond to the detection point output by the sensing equipment, the accuracy of time delay measurement is greatly improved, and the timeliness of a sensing system is optimized.

Description

Technical Field

[0001] The present invention relates to the field of vehicle-road collaboration technology, and in particular to a method for testing end-to-end delay of roadside fusion perception. Background Art

[0002] With the rapid progress of intelligent transportation and autonomous driving technologies, the vehicle-to-everything (V2X) system has shown significant potential in enhancing road safety, alleviating traffic congestion, and optimizing traffic management. As a core component of the system, roadside fusion perception technology integrates multiple sensors such as cameras, lidar, and millimeter-wave radar to achieve comprehensive perception of the road environment. These sensors can monitor dynamic information on the road in real time, including vehicle location, speed, and driving direction, and process them efficiently through edge computing to generate structured data, providing key support for autonomous vehicles and traffic management systems.

[0003] However, the current fusion perception system generally has significant end-to-end delay problems in the entire link of data collection, processing and transmission. The superposition of this delay may cause delays in vehicle perception results, especially in high-speed driving or complex traffic scenarios (such as intersections). This delay will directly affect the vehicle's real-time decision-making ability and the overall performance of the system. With the widespread application of vehicle-road cooperative technology, it is necessary to improve the precise quantification capability and reduce the end-to-end delay to meet the performance requirements of the roadside perception system. Summary of the invention

[0004] The present invention aims to provide a method for testing the end-to-end delay of roadside fusion perception, so as to optimize the end-to-end delay problem of the existing roadside perception system, which further affects the vehicle's response ability and overall performance in high-speed traffic environment or complex environment.

[0005] To achieve the above object, the present invention adopts the following technical solution, a method for testing the end-to-end delay of roadside fusion perception, comprising the following steps:

[0006] Step 1: Get the initial point coordinates C of the true value car 0 , and determine the test driving trajectory of the real value vehicle;

[0007] Step 2: The true value vehicle starts from the initial point, drives along the set driving trajectory and returns to the initial point;

[0008] Step 3: Collect the trajectory data of the real car returning to the initial point, including the coordinate value C' obtained in the real trajectory and the corresponding timestamp t 1 , as well as the coordinate value C" obtained in the perception device coordinate system and the corresponding timestamp t 2 ;

[0009] Step 4: According to the acquired real trajectory point timestamp t 1 and the timestamp of the perception trajectory point t 2 , calculate the end-to-end delay Δt, Δt = t 1 -t 2 .

[0010] The principles and advantages of this solution are:

[0011] Existing roadside fusion perception systems often have difficulty accurately assessing the end-to-end delay from when a vehicle is detected, when data processing is completed, to when output results are generated when sensing vehicle position and motion information. This delay is caused by multiple factors, including delays in sensor data acquisition and fusion processing, delays in the communication network between devices, and the time it takes to transmit data to the cloud or edge computing devices. These delays have little impact in low-speed or simple traffic scenarios, but in high-speed vehicles or complex traffic environments (such as multi-lane intersections or roads with high traffic volume), the accumulation of delays will significantly affect the vehicle's real-time perception and response capabilities, which may reduce the overall efficiency and safety of the system.

[0012] In addition, the existing technology lacks an effective mechanism to monitor and evaluate this delay in real time. Especially in a multi-sensor fusion perception environment, it is difficult for the system to distinguish the delay caused by different links (such as processing time or network delay), which further limits the optimization and improvement of the roadside perception system. The reasons are mainly as follows:

[0013] 1. Sensor data processing time: Roadside fusion perception equipment needs to fuse and process the raw data obtained by multiple sensors (such as cameras, lidar, etc.). This process usually includes data preprocessing, feature extraction, target recognition, tracking, and synchronization and fusion of multi-source data. Each step takes a certain amount of time to process, and as the amount and complexity of data increases, the delay will also increase accordingly. Especially in complex scenarios, the extension of data processing time is one of the main factors affecting latency.

[0014] 2. Communication network delay: In the vehicle-road cooperative system, roadside equipment usually transmits perception data to the cloud or other terminal devices through wireless networks. In this process, the speed of data transmission will be affected by factors such as network bandwidth, delay, packet loss rate and interference. Especially when the amount of data is large or the network is congested, the communication delay will increase significantly. In addition, the communication protocol and data transmission format between different nodes of the system will further affect the data transmission efficiency and lead to the accumulation of delay.

[0015] 3. Edge computing and cloud processing: Although edge computing can reduce the latency caused by data transmission, complex data analysis and decision-making may still require cloud processing. This cloud-edge collaborative model not only achieves efficient computing, but also increases the latency of round-trip data transmission, further affecting the end-to-end response time.

[0016] Based on this, this solution combines high-precision timestamp recording and spatial mapping technology to accurately evaluate the vehicle starting from the initial point within the detection range, and when it returns to the initial point, it records the real timestamp and perception timestamp obtained at the same time, thereby calculating the end-to-end delay of the system. In order to improve the accuracy of delay calculation, it is necessary to ensure the accurate correspondence of vehicle positions and maximize the overlap of the vehicle returning to the initial point. Therefore, this solution establishes a spatial mapping relationship between the actual position of the vehicle and the recognition result of the perception device, calibrates the perspective difference of the device, ensures that the actual position of the true value vehicle and the detection point output by the perception device can accurately correspond, and thus greatly improves the accuracy of delay measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the process framework of the present invention.

[0018] Figure 2 It is a schematic diagram of the test process in the present invention. DETAILED DESCRIPTION

[0019] The following is further described in detail through specific implementation methods:

[0020] Embodiment 1

[0021] In this embodiment, a method for testing the end-to-end delay of roadside fusion perception is used to measure the time difference between the real trajectory data and the perception trajectory data when the vehicle returns to the initial point by combining high-precision timestamp recording and spatial mapping technology, and calculate the end-to-end delay of the system. And by constructing a spatial mapping model between the real position of the vehicle and the recognition result of the perception device, the perspective deviation between the devices is corrected to ensure that the real vehicle position and the detection point output by the perception device can be accurately matched, thereby significantly improving the accuracy of the delay measurement. In this embodiment, as shown in the attached Figure 1 As shown, the following steps are included:

[0022] S1, obtain the initial point coordinates C of the true value car 0 , and determine the true value vehicle test driving trajectory.

[0023] In intelligent transportation systems, the selection of test road environments is critical. The selected test roads should cover a variety of traffic scenarios, such as complex intersections in cities or straight lanes on highways, to ensure that the perception equipment can capture a variety of real traffic conditions. Perception equipment (such as cameras, lidar, etc.) should be placed at multiple locations along the road to ensure full coverage of the test area. In the selected test road environment, select the initial point position of the real value vehicle, as shown in the attached figure. Figure 2 As shown, you can choose a point at the intersection as the initial point C 0 , and record the coordinate value of the current initial point as the basis for the accuracy of the subsequent benchmarking points, and use this test road to determine the driving trajectory of the true value vehicle, so that the true value vehicle can travel according to the driving trajectory, ensuring that effective and accurate data is obtained.

[0024] S2, makes the true value car start from the initial point, drive according to the set driving trajectory and return to the initial point.

[0025] After determining the driving trajectory, the true value car is moved from the initial point C 0 Start driving along the driving trajectory, so that the true value car returns to the initial point C as accurately as possible 0 Location.

[0026] S3, collect the trajectory data of the true value vehicle returning to the initial point again, including the coordinate value C' obtained in the real trajectory and the corresponding timestamp t 1 , as well as the coordinate value C" obtained in the perception device coordinate system and the corresponding timestamp t 2 .

[0027] In a real test environment, the real trajectory of the real value vehicle is recorded in real time, including its real coordinates and corresponding timestamps. In this embodiment, in order to obtain accurate real value vehicle position information, GNSS (Global Navigation Satellite System) or RTK (Real-time Kinematic Positioning Technology) can be used. The accuracy of these systems can reach the centimeter level. RTK uses differential positioning technology to correct satellite signals using ground reference stations, thereby significantly improving positioning accuracy. The acquisition coordinates (X, Y) and timestamps are recorded at a fixed frequency when the vehicle moves. 0 , to represent the actual trajectory of the vehicle, in order to obtain trajectory data. At the same time, the roadside fusion perception device also detects the real value vehicle in real time through multiple sensors such as cameras and lidars, and records the perceived trajectory, including perception coordinates and timestamps, to ensure that the perception device can accurately identify the vehicle position and time, and ensure the accuracy of data collection in a dynamic environment.

[0028] When the true value vehicle drives to the initial point again, the coordinate value C' and the corresponding timestamp t 1 , as well as the perceived coordinate value C" and the corresponding timestamp t measured by the perception device 2 .

[0029] In this embodiment, in order to ensure the accuracy of time recording of the true value vehicle and the perception device, it is necessary to ensure that all devices in the system are synchronized using the same clock source. Common time synchronization methods include using NTP (Network Time Protocol) or PTP (Precision Time Protocol). At the same time, in order to ensure the accuracy and effectiveness of the data, the collected trajectory data can be filtered. By pre-processing the data, the data volume can be reduced while reducing the impact of noise.

[0030] Among them, the acquired trajectory data is filtered, and the filtering method is:

[0031]

[0032] In the formula, K k is the Kalman gain; is the current state estimate; z k is the measured value; A and B are the system state matrices; C is the observation matrix, which represents the conversion of the internal state of the system into the value that the sensor can measure.

[0033] In this embodiment, in order to ensure that the trajectory data coordinate value C' and the perception coordinate value C" obtained when the vehicle is in motion can be as close as possible to the initial point C 0 High overlap means ensuring that the vehicle can more accurately return to the initial point, thereby obtaining more accurate data collection, and comprehensively evaluating the overall efficiency of sensor detection, data processing, and system response. It also includes building a driving trajectory correction model to shorten C', C" and C 0 The error between .

[0034] Building a driving trajectory correction model includes the following sub-steps:

[0035] S3.1, select several calibration points, and based on perspective transformation, establish a mapping relationship between the world coordinate system and the coordinate system in the perception device to obtain the initial perspective transformation matrix H.

[0036] In this embodiment, at least four known spatial points are calibrated as calibration points on the test road to ensure accurate coordinate information. The calibration points need to cover the entire test area and be evenly distributed to avoid global errors caused by local deviations. i ,Y i ) and the coordinates (x) in the sensing device i ,y i In this embodiment, the coordinates of these calibration points (X i ,Y i) can be measured by high-precision positioning equipment (RTK-GNSS system), thereby establishing a spatial mapping relationship between the vehicle's actual world coordinates and the perception device; and obtaining the initial perspective transformation matrix H.

[0037] In this embodiment, the obtained transformation matrix H is a 3x3 matrix used to map a point from the image plane coordinate system (x, y) to the coordinate system (X', Y') of the real world. According to the established initial perspective transformation matrix H, the image plane coordinate system (x, y) (two-dimensional coordinate system) of the perception device is mapped to the three-dimensional coordinate system (X, Y) of the real world through perspective transformation. The transformation process is expressed as:

[0038]

[0039] In actual operation, H is calculated by least square method or SVD decomposition from at least four known corresponding points, which may have certain errors. Therefore, the transformed coordinate system (X′, Y′) can be optimized by normalization calculation to obtain the three-dimensional coordinate system (X, Y), which is expressed as

[0040] Where Z′ represents the normalized calculation.

[0041] By applying perspective transformation (Homography), a mapping relationship matrix between the world coordinate system and the detection points of the perception device is established, and the coordinates recorded by the perception device are accurately converted into coordinates in the world coordinate system, thereby ensuring that the position of the perception target is aligned with the true value vehicle.

[0042] S3.2, the calibration points are transformed again by the constructed initial perspective transformation matrix H to obtain the corresponding actual trajectory point information, and the error between the actual point after transformation and the true point is obtained.

[0043] In actual operation, since the constructed initial perspective transformation matrix H will have certain errors, the actual trajectory point information is obtained by reversely transforming the selected calibration points by the initial perspective transformation matrix H, and then comparing it with the known real point information, so as to clearly know the existing errors and adjust the initial perspective transformation matrix H.

[0044] S3.3, adjusting the initial perspective transformation matrix H according to the point position error using a set optimization method to obtain a mapping matrix.

[0045] In order to further optimize the result of perspective transformation, the Levenberg-Marquardt algorithm can be used to combine the advantages of gradient descent and Newton's method for optimization. The optimization method is expressed as:

[0046]

[0047] In the formula, J k is the Jacobian matrix, λ is the adjustment factor, r k is the residual.

[0048] S4, according to the acquired real trajectory point timestamp t 1 and the timestamp of the perception trajectory point t 2 , calculate the end-to-end delay Δt, Δt = t 1 -t 2 .

[0049] In this embodiment, since the perception device and the true value vehicle are not in the same coordinate system, in order to ensure the accuracy of the measured time delay, the coordinates recorded by the perception device must be converted into the actual coordinates of the true value vehicle using the perspective transformation matrix H. However, the position error after the perspective transformation may affect the accuracy of the time delay, so it is necessary to further correct the obtained time delay and adjust the mapping matrix. In this embodiment, the least squares method is used to adjust the mapping matrix, and the correction method is:

[0050]

[0051] In the formula, E(H) is the minimized error; i represents each point; and N represents the total number of points. By minimizing the error E(H), the optimal mapping matrix H can be obtained, thereby improving the accuracy of delay calculation.

[0052] The end-to-end delay is calculated based on the acquired real trajectory point timestamp t1 and the perceived trajectory point timestamp t2, which reflects the time difference from the actual passing of the true vehicle through the initial point to the detection of the vehicle by the perception device. The overall efficiency of sensor detection, data processing and system response is comprehensively evaluated, effectively improving the accuracy and practicality of system evaluation, thereby optimizing the timeliness of the perception system and improving the overall coordination of vehicle-road collaboration.

[0053] Compared with the traditional end-to-end delay calculation method, this solution uniquely introduces perspective transformation technology in delay calculation to ensure the accurate correspondence between the spatial positions of the perception device and the true value vehicle. This innovative design effectively avoids measurement errors caused by differences in perspective and improves the consistency and reliability of data. In addition, this solution adopts a multi-point calibration and verification mechanism to further enhance the accuracy and stability of the measurement results by correcting multiple known positions. This comprehensive approach not only improves the accuracy of delay calculation, but also provides a more reliable solution for real-time monitoring in complex traffic environments.

[0054] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions and / or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical solution of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A method for testing end-to-end delay of roadside fusion perception, characterized in that: The following steps are included: Step 1, obtain the initial point coordinates C0 of the true value car and determine the test driving trajectory of the true value car; Step 2: The true value vehicle starts from the initial point, drives along the set driving trajectory and returns to the initial point; Step 3, collect the trajectory data of the true value vehicle returning to the initial point again, including the coordinate value C1 obtained in the real trajectory and the corresponding timestamp t1, and the perception coordinate value C2 obtained in the perception device coordinate system and the corresponding timestamp t2; Step 4: Calculate the end-to-end delay Δt based on the acquired real trajectory point timestamp t1 and the perceived trajectory point timestamp t2, Δt=t1-t2.

2. A method for testing end-to-end delay of roadside fusion perception according to claim 1, characterized in that: In step 3, it also includes constructing a driving trajectory correction model to shorten the error between C,, C,, and C0.

3. A method for testing roadside fusion perception end-to-end delay according to claim 2, characterized in that: In step 3, constructing the driving trajectory correction model includes the following sub-steps: Step 3.1, select several calibration points, establish a mapping relationship between the world coordinate system and the coordinate system in the perception device based on perspective transformation, and obtain the initial perspective transformation matrix H; Step 3.2, transform the calibration points through the constructed initial perspective transformation matrix H to obtain the corresponding actual trajectory point information, and obtain the error between the actual point after transformation and the true point; Step 3.3, adjust the initial perspective transformation matrix H according to the point error using the set optimization method to obtain the mapping matrix.

4. A method for testing end-to-end delay of roadside fusion perception according to claim 1, characterized in that: In step 3, the acquired trajectory data is filtered, and the filtering method is as follows: In the formula, K k is the Kalman gain; is the current state estimate; z k is the measured value; A and B are the system state matrices; C is the observation matrix, which represents the conversion of the internal state of the system into the value that the sensor can measure.

5. The method for testing the end-to-end delay of roadside fusion perception according to claim 3 is characterized by: In step 3.1, at least four known spatial points are calibrated on the test road as calibration points, and the world coordinates (X i ,Y i ) and the coordinates (x) in the sensing device i ,y i );Establish a spatial mapping relationship between the actual world coordinates of the vehicle and the perception device; Get the initial perspective transformation matrix H.

6. A method for testing end-to-end delay of roadside fusion perception according to claim 5, characterized in that: According to the established initial perspective transformation matrix H, the image plane coordinate system (x, y) of the perception device is mapped to the three-dimensional coordinate system (X, Y) of the real world through perspective transformation, which is expressed as: Where Z represents the normalized calculation.

7. A method for testing roadside fusion perception end-to-end delay according to claim 1, characterized in that: In step 4, the obtained time delay is also corrected in the following manner: Where E(H) is the minimized error; i represents each point; and N represents the total number of points.

8. The method for testing the end-to-end delay of roadside fusion perception according to claim 3 is characterized by: In step 3.3, the optimization method is: In the formula, J k is the Jacobian matrix, λ is the adjustment factor, r k is the residual.

9. The method for testing the end-to-end delay of roadside fusion perception according to claim 1, characterized in that: It also includes recording and collecting at a fixed frequency when acquiring trajectory data.

10. The method for testing the end-to-end delay of roadside fusion perception according to claim 5, characterized in that: It also includes that the selected test roads must cover a variety of traffic scenarios, and the sensing equipment must be set up at multiple locations along the road; the calibration points must cover the entire test area and be evenly distributed.

Images