Intelligent driving sensing site testing method

Abstract

The invention relates to an intelligent driving perception site test method, which comprises the following steps of: building a closed intelligent driving perception test site, and gridding the test site; the target vehicle runs from the starting point according to a preset track, and an intelligent driving system of the intelligent driving vehicle collects intelligent driving sensing test data of the target vehicle through a sensor and obtains true value test data of the target vehicle at the same time; and performing time synchronization and format synchronization on the intelligent driving sensing test data obtained by the intelligent driving vehicle and the true value test data of the target vehicle, and then performing comparison and judgment. The method is based on the intelligent driving system, only video recording equipment needs to be installed, a truth value system does not need to be installed for testing, and cost is reduced; the equipment involved in the invention is convenient to debug, the data processing is simple, and the test efficiency is obviously improved; the method is based on a closed site, and guarantees are provided for testing safety.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving technology, and in particular relates to an intelligent driving perception field testing method. Background Art

[0002] It is understood that intelligent driving vehicles consist of three parts: the perception system, the central decision-making system, and the underlying control system execution system. The perception system is the foundation and key link of intelligent driving, including several groups of sensors deployed on the vehicle. Intelligent driving vehicles mainly perceive the surrounding environment and vehicle-related status information through sensors such as cameras, lidar, and millimeter-wave radar.

[0003] For intelligent driving systems based on deep learning, the traffic roads and environments in real life are relatively complex, and the goals of traffic participants vary greatly, resulting in frequent safety problems in intelligent driving, and there are still great safety issues in perception system detection. For safety reasons, it is necessary to improve the accuracy of sensor recognition targets of intelligent driving vehicles, further enable planning and decision-making, and improve intelligent driving performance. At present, intelligent driving vehicle-based perception system testing is mainly divided into two methods: pure algorithm testing and real vehicle testing. Among them, pure algorithm testing is to test offline data sets and compare the test results with the labeled true values. However, this method does not take into account the real roads and vehicle environment complexity, sensor calibration errors, hardware equipment failures, and excessive system load during real-life vehicle testing. The real vehicle test is based on the real vehicle road environment, and the target equipment is equipped with a true value system, and the perception recognition output results are read for evaluation. However, this method is complex to debug and use, and the equipment cost is high and the targets are limited.

[0004] Therefore, for the perception testing of intelligent driving vehicles, a low-cost testing method is needed to obtain the true values ​​of multiple targets under actual vehicle testing conditions. Summary of the Invention

[0005] The purpose of the present invention is to propose an intelligent driving perception field testing method to address the problems existing in the prior art.

[0006] In order to achieve the above objectives, the present invention provides a method for intelligent driving perception field testing, comprising the following steps: Step 1: Build a closed intelligent driving perception test site and grid the test site; Step 2: Place the intelligent driving vehicle equipped with the intelligent driving system and sensors horizontally at the center of the grid. Establish a vehicle coordinate system with the intelligent driving vehicle as the origin. Then, preset the starting point and driving trajectory of the target vehicle and place the target vehicle at the preset starting point. Step 3: The target vehicle starts to travel along the preset trajectory from the starting point. During the driving process, the intelligent driving system of the intelligent driving vehicle collects the intelligent driving perception test data of the target vehicle through sensors and uses a video recording device to record the movement of the target vehicle to obtain the true value test data of the target vehicle; Step 4: Synchronize the time and format of the intelligent driving perception test data obtained by the intelligent driving vehicle and the true value test data of the target vehicle; Step 5: Compare and judge the synchronized intelligent driving perception test data and the target vehicle true value test data.

[0007] The present invention is based on real vehicle testing, and the test site is closed and gridded, which facilitates the acquisition of target information. At the same time, no true value system is installed to obtain target information, and only video recording equipment needs to be installed to record data, which makes equipment debugging simple and low-cost. The present invention further adopts the following technical solutions: In step 2, the sensors include a lidar, a camera, and a millimeter-wave radar, and the coordinate origin of the intelligent driving vehicle coincides with the center of the grid.

[0008] In step 3, the intelligent driving perception test data includes laser, vision, millimeter wave, and fused output information, and the fused output information is the fusion information of laser and vision.

[0009] Preferably, the intelligent driving perception test data specifically includes the perceived position, perceived heading, perceived speed of the target vehicle, and the perceived lateral relative distance and perceived longitudinal relative distance between the target vehicle and the intelligent driving vehicle.

[0010] In step 3, the video recording device records the images of the target vehicle from the start to the end of the movement, and then records the position of the target vehicle and its corresponding time by replaying the video, thereby obtaining the true value test data of the target vehicle.

[0011] Preferably, the target vehicle true value test data includes the actual heading and actual speed of the target vehicle, and the actual lateral relative distance and actual longitudinal relative distance between the target vehicle and the intelligent driving vehicle.

[0012] Preferably, the target vehicle true value test data also includes accuracy and recall rate.

[0013] Preferably, the number of the target vehicles is one or more.

[0014] Compared with traditional testing methods, the method of the present invention is highly scalable. In addition to vehicle perception testing, it can also be used for fixed traffic sign, pedestrian, and VRU testing scenarios. Based on an intelligent driving system, the method of the present invention only requires the installation of video recording equipment, eliminating the need for a true value system for testing, thus reducing costs. The equipment involved in the present invention is easy to debug and data processing is simple, significantly improving testing efficiency. The method of the present invention is based on a closed site, ensuring testing safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present invention will be further described below with reference to the accompanying drawings.

[0016] Figure 1 It is a logic block diagram of the present invention.

[0017] Figure 2 This is a principle block diagram of the intelligent driving system in the present invention.

[0018] Figure 3 This is a schematic diagram of the preset different trajectories for multiple target vehicles in the present invention. DETAILED DESCRIPTION

[0019] Example 1 like Figure 1 As shown, a method for intelligent driving perception field testing specifically includes the following steps: Step 1: Build a closed intelligent driving perception test site and grid the test site.

[0020] Step 2: Place the intelligent driving vehicle equipped with an intelligent driving system and sensors horizontally at the center of the grid. Establish a vehicle coordinate system with the intelligent driving vehicle as the origin. Then, preset the starting point and driving trajectory of the target vehicle and place the target vehicle at the preset starting point. The sensors installed on the intelligent driving vehicle include lidar, cameras, and millimeter-wave radar. The coordinate origin of the intelligent driving vehicle coincides with the center of the grid.

[0021] Step 3: The target vehicle starts to drive along the preset trajectory from the starting point. During the driving process, the intelligent driving system of the intelligent driving vehicle collects the intelligent driving perception test data of the target vehicle through sensors, and uses a video recording device to record the motion picture of the target vehicle to obtain the true value test data of the target vehicle (i.e. the actual data of the target vehicle). Among them, the intelligent driving perception test data is collected by the laser radar, visual camera, and millimeter wave radar installed on the intelligent driving vehicle, which are the output information of laser, vision, millimeter wave and fusion respectively. The fusion information is the information output after the fusion of laser and vision (see Figure 2Intelligent driving perception test data is the intelligent driving perception test results of a target vehicle acquired by the intelligent driving perception system during its driving process. Various sensors installed on the intelligent driving vehicle identify and detect the target vehicle, outputting printed results, and then using this tool for analysis. This intelligent driving perception test data specifically includes the target vehicle's perceived position, heading, speed, and the perceived lateral and longitudinal relative distances between the target vehicle and the intelligent driving vehicle.

[0022] The video recording device uses a camera and can be installed in the test site, on the intelligent driving vehicle, or on the target vehicle. The video recording device records the target vehicle from the start to the end of its motion. The target vehicle's position and corresponding time are then recorded by replaying the video, thereby obtaining the target vehicle's true test data. Specifically, a body coordinate system is established with the intelligent driving vehicle as the origin. The target vehicle's true test data is obtained by driving along a preset trajectory. Because the test site is a closed and gridded area, the target vehicle's real-time coordinates within the site are easily obtained, and the target vehicle's true test data is then obtained from the target vehicle's coordinates. The target vehicle's true test data specifically includes the target vehicle's actual heading, actual speed, accuracy, recall rate, and the actual relative lateral and longitudinal distances between the target vehicle and the intelligent driving vehicle. Furthermore, the target vehicle's position is easily determined from its heading (30 degrees, 60 degrees, etc.) as it drives along a preset trajectory. The target vehicle's actual heading is obtained from the preset trajectory, while the actual relative lateral and longitudinal distances between the target vehicle and the intelligent driving vehicle represent the target vehicle's horizontal and vertical coordinates within the site, respectively. The actual speed of the target vehicle is calculated using the spline interpolation method. The specific calculation method is as follows: the time when the target vehicle is at the coordinate point is known from the video recording device, and each grid has its corresponding distance to the origin. Then, the spline interpolation method (interpolation of the three points before and after) can be used to calculate the speed of the target vehicle at the coordinate point. The B-spline curve formula is: Accuracy refers to the proportion of all correct predictions (positive and negative) to the total, that is, the proportion of target results in the evaluation results; recall refers to the proportion of correctly predicted positives to all actual positives, that is, the proportion of target categories recalled from the focus area. Accuracy and recall are important evaluation indicators for selecting targets in a mixed environment, where accuracy = number of correct information items extracted / number of information items extracted; Recall rate = number of correct information items extracted / number of information items in the sample. Furthermore, when the target vehicle is known, the accuracy rate is compared to the output results. If the output of the intelligent driving system is within an acceptable error range compared to the true value, the accuracy of the intelligent driving system output data is high. When the number of targets is known, the recall rate can also be compared. The output of the intelligent driving system is first verified for accuracy. The subsequent step is to statistically analyze the speed, acceleration, heading, and other information calculated by different algorithms and compare them with the true value to determine the optimal algorithm or iteration direction.

[0023] Step 4: Synchronize the time and format of the intelligent driving perception test data obtained by the intelligent driving vehicle and the true value test data of the target vehicle.

[0024] The PTP (Precision Time Protocol) clock synchronization method is used to synchronize the time of the intelligent driving perception test data and the true value test data; data format synchronization is to convert the rosbag data format of the intelligent driving perception test data and the true value test data according to the requirements of the test system.

[0025] Step 5: Compare and judge the synchronized intelligent driving perception test data and the target vehicle true value test data.

[0026] Specifically, the observed values ​​of the target vehicle's heading, speed, acceleration, lateral and longitudinal distances, etc. are compared with the true values. The comparison methods include mean value, variance, standard deviation, etc. to judge the performance of different algorithms, and then provide a basis for algorithm optimization, and provide data and test reports for subsequent algorithm optimization work.

[0027] In addition, in actual testing, the target vehicle in the test site can be one or more. Figure 3 The specific steps are as follows: (1) Grid the closed test site and set the horizontal length of a grid to x and the vertical length to y; (2) Set the intelligent driving vehicle as S1 and the target vehicle as S2; (3) Place the intelligent driving vehicle S1 at the center of the grid with the front of the vehicle facing upwards, and establish a vehicle body coordinate system with the center of the grid as the origin; place the target vehicle S2 parallel to the right rear of S1, and preset the driving trajectory to be straight driving; (4) The intelligent driving system of the intelligent driving vehicle S1 is activated, and the video recording device is activated at the same time to record the driving data of the target vehicle S2; (5) Data processing is performed on the intelligent driving perception test data collected by the intelligent driving system and the target vehicle true value test data recorded by the video recording device, that is, the two are synchronized in time and format, and then compared and judged.

[0028] The intelligent driving perception field test method of the present invention can also be expanded to multiple target vehicles, different trajectories, other markers, etc. The multiple target vehicles can travel along different preset trajectories. For example, three target vehicles are set up, namely S2, S3, and S4, where target vehicle S2 travels parallel to intelligent driving vehicle S1, target vehicle S3 travels perpendicular to intelligent driving vehicle S1, and target vehicle S4 travels at an angle to intelligent driving vehicle S1 (see Figure 3 ).

[0029] The present invention is based on real vehicle testing, and the test site is closed and gridded to facilitate the acquisition of target information. At the same time, no true value system is installed to obtain target information. Only a video recording device needs to be installed to record data. The equipment debugging is simple and the cost is low. Compared with traditional testing methods, the method of the present invention is highly scalable. In addition to being used for vehicle perception testing, the method of the present invention can also be used for fixed traffic signs, pedestrians, and VRU testing, but is not suitable for situations where speed, heading, etc. change dramatically. The cost is low. The method of the present invention is based on an intelligent driving system and only requires the installation of a video recording device. There is no need to install a true value system for testing, which reduces costs. The efficiency is high. The equipment involved in the present invention is easy to debug and data processing is simple. Safety is improved. The method of the present invention is based on the test results of closed sites for different scenarios, and can also compare the performance of various sensors to improve perception performance detection and provide protection for test safety.

[0030] In addition to the above embodiments, the present invention may also have other implementations. Any technical solution formed by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A method for testing intelligent driving perception field, characterized in that: The method comprises the following steps: Step 1: Build a closed intelligent driving perception test site and grid the test site; Step 2: Place the intelligent driving vehicle equipped with the intelligent driving system and sensors horizontally at the center of the grid. Establish a vehicle coordinate system with the intelligent driving vehicle as the origin. Then, preset the starting point and driving trajectory of the target vehicle and place the target vehicle at the preset starting point. Step 3: The target vehicle starts from the starting point and drives along the preset trajectory. During the driving process, the intelligent driving system of the intelligent driving vehicle collects the intelligent driving perception test data of the target vehicle through sensors and uses a video recording device to record the motion picture of the target vehicle to obtain the true value test data of the target vehicle; Step 4: Synchronize the time and format of the intelligent driving perception test data obtained by the intelligent driving vehicle and the true value test data of the target vehicle; Step 5: Compare and judge the synchronized intelligent driving perception test data and the target vehicle true value test data.

2. The intelligent driving perception field testing method according to claim 1, characterized in that: In step 2, the sensors include a lidar, a camera, and a millimeter-wave radar, and the coordinate origin of the intelligent driving vehicle coincides with the center of the grid.

3. The intelligent driving perception field testing method according to claim 2, characterized in that: In step 3, the intelligent driving perception test data includes laser, vision, millimeter wave, and fused output information, and the fused output information is the fusion information of laser and vision.

4. The intelligent driving perception field testing method according to claim 3, characterized in that: The intelligent driving perception test data specifically includes the perceived position, perceived heading, perceived speed of the target vehicle, and the perceived lateral relative distance and perceived longitudinal relative distance between the target vehicle and the intelligent driving vehicle.

5. The intelligent driving perception field testing method according to claim 4, characterized in that: In step 3, the video recording device records the images of the target vehicle from the start to the end of the movement, and then records the position of the target vehicle and its corresponding time by replaying the video, thereby obtaining the true value test data of the target vehicle.

6. The intelligent driving perception field testing method according to claim 5, characterized in that: The target vehicle true value test data includes the actual heading and actual speed of the target vehicle, and the actual lateral relative distance and actual longitudinal relative distance between the target vehicle and the intelligent driving vehicle.

7. The intelligent driving perception field testing method according to claim 6, characterized in that: The target vehicle true value test data also includes accuracy and recall rate.

8. The intelligent driving perception field testing method according to claim 7, characterized in that: The number of the target vehicles is one or more.

Images