A method and system for evaluating and visualizing a multi-sensor autonomous driving algorithm

Abstract

The present invention discloses a method and system for evaluating and visualizing a multi-sensor autonomous driving algorithm, the method comprising: the autonomous driving vehicle saves the data collected by multiple sensors as raw sensor data; respectively uses the autonomous driving algorithms before and after optimization to calculate the raw sensor data to fuse the control data respectively; visualizes the raw sensor data and the fused control data of the autonomous driving algorithms before and after optimization; loads the fused control data of the autonomous driving algorithms before and after optimization, and selects a comparison period, and generates an algorithm evaluation report by comparing the changes in the braking time and braking acceleration of the vehicle under the same scene in the period. The present invention uses the same raw sensor data, so that while 100% restoring the historical scene, the data obtained by different autonomous driving algorithms can also be compared, so that the generated report can be closer to the actual scene and more convincing.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and system for evaluating and visualizing a multi-sensor autonomous driving algorithm. Background Art

[0002] Autonomous driving technology obtains information about surrounding vehicles and the cloud through the Internet of Vehicles, which can achieve more efficient vehicle dispatch and alleviate traffic congestion caused by too many vehicles. At the same time, autonomous driving technology can liberate human drivers, thus attracting many algorithm developers to join the wave of autonomous driving research.

[0003] For example, in the invention patent application with application number CN201910801897.0, a method, device and equipment for testing the positioning module of an autonomous driving vehicle are disclosed, the method comprising: obtaining a test case, the test case comprising a test section configured based on a high-precision map, vehicle model data and driving status data; running the test case, generating a simulation scene of the autonomous driving vehicle running on the test section according to the vehicle model data and driving status data; obtaining the reference position information of the autonomous driving vehicle in the simulation scene, and generating a test control instruction based on the reference position information; outputting a simulation signal for positioning according to the test control instruction, so that the positioning module receives the simulation signal, and generates the real-time position information of the autonomous driving vehicle according to the simulation signal; obtaining the real-time position information, and generating a test report according to the comparison result of the real-time position information and the reference position information. This scheme realizes the automated testing of the positioning module in the simulation scene, reduces the testing cost and improves the testing efficiency. However, the above solution also has the following problems: The above solution mentions comparing the simulated position and the reference position. Although it is helpful to generate a comparison report, we know that when autonomous driving vehicles pass the same place, the signals received by each sensor are also very different. If only simulated scenes and signals are used, this approach is limited to algorithm research and cannot be better adapted to actual application scenarios. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is: how to provide an evaluation and visualization method and system for a multi-sensor autonomous driving algorithm that can be closer to the actual application scenario, can display data more intuitively, and can automatically generate an evaluation report for the vehicle's autonomous driving algorithm.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A method for evaluating and visualizing a multi-sensor autonomous driving algorithm comprises the following steps:

[0007] Step 1) The autonomous driving vehicle saves the data collected by multiple sensors as raw sensor data;

[0008] Step 2) The original sensor data is calculated using the autonomous driving algorithm before optimization and the autonomous driving algorithm after optimization, so as to obtain the fusion control data of the autonomous driving algorithms before and after optimization under the same scenario;

[0009] Step 3) Visualizing the original sensor data, the fused control data of the autonomous driving algorithm before optimization, and the fused control data of the autonomous driving algorithm after optimization;

[0010] Step 4) Load the fusion control data of the autonomous driving algorithm before and after optimization, select a comparison period, and generate an algorithm evaluation report by comparing the changes in the vehicle's braking time and braking acceleration under the same scenario within the comparison period.

[0011] Preferably, in step 2), the historical data of the same period of the autonomous driving algorithm before and after optimization are replayed and saved respectively, and the historical data includes original sensor data and fusion control data.

[0012] Preferably, in step 3), the historical data of the same period of the autonomous driving algorithm before and after optimization saved in step 2) is displayed in real time and visualized, and the visualization method includes camera video display and overhead view display.

[0013] Preferably, step 3) also includes locating and displaying the braking time points in the historical data.

[0014] Preferably, in step 4), a time period set before and after the braking time point is selected as the braking time period, and the braking time period is used as the comparison time period, and an algorithm evaluation report is generated by comparing the changes in the braking moment and braking acceleration of the vehicle in the same scenario within the time period.

[0015] Preferably, in step 4), when the automatic algorithms before and after optimization brake at the same time within the braking time period, the braking duration and braking acceleration of the automatic driving algorithms before and after optimization are compared, and the rationality of the automatic driving algorithms before and after optimization is judged based on the comparison results and an algorithm evaluation report is generated.

[0016] Preferably, in step 4), when the automatic algorithms before and after optimization brake at different times within the braking time period, the braking time points of the automatic driving algorithms before and after optimization are recorded respectively, and the rationality of the automatic driving algorithms before and after optimization is judged according to the braking time points and an algorithm evaluation report is generated.

[0017] A system for implementing the above-mentioned multi-sensor autonomous driving algorithm evaluation and visualization method, comprising:

[0018] A sensor assembly, the sensor assembly comprising a plurality of sensors distributed on the autonomous driving vehicle, the sensor assembly being used to collect autonomous driving data of the autonomous driving vehicle as raw sensor data;

[0019] A data acquisition device, the data acquisition device is connected to the sensor assembly and is used to store raw sensor data collected by the sensor assembly;

[0020] An automatic driving algorithm module, the automatic driving algorithm module is connected to the data acquisition device and is used to perform calculations based on the original sensor data and obtain fused control data;

[0021] A data storage module, which is connected to the automatic driving algorithm module and is used to store the calculated fusion control data;

[0022] A data reading module, the data reading module is connected to the data acquisition device and the data storage module respectively, so as to read the original sensor data and the fusion control data from the data acquisition device and the data storage module respectively;

[0023] An offline visualization module, which is connected to the data reading module and is used to display in real time the raw sensor data of the same period and the fused control data obtained by the automatic driving algorithm before and after optimization, and the offline visualization module includes a camera display module and a bird's-eye view display module, the camera display module is used to display the data by camera video, and the bird's-eye view display module is used to display vehicle and lane line information;

[0024] An algorithm evaluation module, which is connected to the offline visualization module, is used to evaluate the braking data of the autonomous driving algorithm before and after optimization and generate an algorithm evaluation report.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] 1. This method uses offline data loading. Algorithm developers can select two sets of data of the autonomous driving algorithm before and after optimization for playback and data analysis. This method can compare the two types of data, display video and data in real time, and automatically generate analysis reports. This helps to improve the development efficiency of algorithm developers and makes it easier for developers to locate braking time points and problem points.

[0027] 2. The present invention uses the same original sensor data, so that while 100% restoring the historical scene, it can also compare the data obtained by different autonomous driving algorithms, so that the generated report can be closer to the actual scene and more convincing.

[0028] 3. The present invention saves the original sensor data and also saves the fusion control data of the autonomous driving algorithm before and after optimization. When the autonomous driving algorithm developer needs to optimize the algorithm, the offline visualization module provided by this solution can better display the current data. After optimizing the algorithm, the obtained data can also be compared with the original data to generate an algorithm evaluation report. In addition, the present invention has a visualization interface that can display data very intuitively and automatically generate a vehicle braking report (accurate to milliseconds), which can accurately evaluate whether the braking is more reasonable before and after the algorithm is optimized.

[0029] 4. As autonomous driving algorithm developers iteratively develop algorithms, due to changes in the fusion control algorithm, the vehicle may choose different time points and different accelerations to brake when dealing with the same scenario. By comparing the performance of different autonomous driving algorithms in the same scenario, the autonomous driving algorithms before and after optimization can be used for offline data playback, thus eliminating interference from the autonomous driving environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flowchart of a method for evaluating and visualizing a multi-sensor autonomous driving algorithm of the present invention;

[0031] Figure 2 A flow chart for visual display of the evaluation and visualization method of the multi-sensor autonomous driving algorithm of the present invention;

[0032] FIG3 is a flow chart of generating an algorithm evaluation report in the evaluation and visualization method of a multi-sensor autonomous driving algorithm of the present invention;

[0033] Figure 4 The system block diagram of the evaluation and visualization system of the multi-sensor autonomous driving algorithm of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0035] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0036] It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. In the description of the present invention, it should be noted that the orientation or position relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invention product is usually placed when used, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance. In addition, the terms "horizontal", "vertical", etc. do not mean that the components are absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0037] As attached Figure 1 To the attached Figure 3 As shown, a method for evaluating and visualizing a multi-sensor autonomous driving algorithm includes the following steps:

[0038] Step 1) The autonomous driving vehicle saves the data collected by multiple sensors as raw sensor data. Specifically, the raw sensor data is saved as a .dat format file;

[0039] Step 2) The original sensor data is calculated using the autonomous driving algorithm before optimization and the autonomous driving algorithm after optimization, so as to obtain the fusion control data of the autonomous driving algorithms before and after optimization under the same scenario;

[0040] Step 3) Visualizing the original sensor data, the fused control data of the autonomous driving algorithm before optimization (algorithm 1), and the fused control data of the autonomous driving algorithm after optimization (algorithm 2);

[0041] Step 4) Load the fusion control data of the autonomous driving algorithm before and after optimization, select a comparison period, and generate an algorithm evaluation report by comparing the changes in the vehicle's braking time and braking acceleration under the same scenario within the comparison period.

[0042] In this embodiment, in step 2), the historical data of the same period of the autonomous driving algorithm before and after optimization are played back and saved respectively. Specifically, the historical data of the autonomous driving algorithm before and after optimization are saved as dat1 file one and dat2 file two respectively, and the historical data includes original sensor data and fusion control data.

[0043] In this embodiment, in step 3), the historical data of the same period of the automatic driving algorithm before and after optimization saved in step 2) is displayed in real time and visualized, and the visualization method includes camera video display and overhead view display.

[0044] In this embodiment, step 3) also includes locating and displaying the braking time point in the historical data.

[0045] In this embodiment, in step 4), a time period set before and after the braking time point is selected as the braking time period, and the braking time period is used as the comparison time period, and an algorithm evaluation report is generated by comparing the changes in the braking moment and braking acceleration of the vehicle in the same scenario within the time period.

[0046] In this embodiment, in step 4), when the automatic algorithms before and after optimization brake at the same time within the braking time period, the braking duration and braking acceleration of the automatic driving algorithms before and after optimization are compared, and the rationality of the automatic driving algorithms before and after optimization is judged based on the comparison results and an algorithm evaluation report is generated.

[0047] In this embodiment, in step 4), when the automatic algorithms before and after optimization brake at different times within the braking time period, the braking time points of the automatic driving algorithms before and after optimization are recorded respectively, and the rationality of the automatic driving algorithms before and after optimization is judged according to the braking time points and an algorithm evaluation report is generated.

[0048] As attached Figure 4 As shown, a system for implementing the evaluation and visualization method of the multi-sensor autonomous driving algorithm includes:

[0049] A sensor component, the sensor component includes a plurality of sensors distributed on the autonomous driving vehicle, and the sensor component is used to collect autonomous driving data of the autonomous driving vehicle as raw sensor data;

[0050] A data acquisition device, the data acquisition device is connected to the sensor component and is used to store the original sensor data collected by the sensor component;

[0051] An autonomous driving algorithm module, which is connected to a data acquisition device and is used to perform calculations based on raw sensor data and obtain fused control data;

[0052] A data storage module, which is connected to the automatic driving algorithm module and is used to store the calculated fusion control data;

[0053] A data reading module, the data reading module is connected to the data acquisition device and the data storage module respectively, so as to read the original sensor data and the fusion control data from the data acquisition device and the data storage module respectively;

[0054] An offline visualization module, which is connected to the data reading module and is used to display in real time the original sensor data of the same period and the fusion control data obtained by the automatic driving algorithm before and after optimization, and the offline visualization module includes a camera display module and a bird's-eye view display module. The camera display module is used to display the data by camera video, and the bird's-eye view display module is used to display vehicle and lane line information;

[0055] The algorithm evaluation module is connected to the offline visualization module and is used to evaluate the braking data of the autonomous driving algorithm before and after optimization and generate an algorithm evaluation report.

[0056] Compared with the prior art, this method uses an offline data loading method. Algorithm developers can select two sets of data of the automatic driving algorithm before and after optimization for playback and data analysis. This method can compare the two data, display the video and data in real time, and automatically generate an analysis report, which helps to improve the development efficiency of algorithm developers and is more convenient for developers to locate the braking time point and problem points. The present invention uses the same original sensor data, so that while 100% restoring the historical scene, it can also compare the data obtained by different automatic driving algorithms, so that the generated report can be closer to the actual scene and more convincing. In the present invention, by saving the original sensor data, the fusion control data of the automatic driving algorithm before and after optimization is also saved. When the automatic driving algorithm developer needs to optimize the algorithm, the offline visualization module provided by this solution can better display the current data. At the same time, after optimizing the algorithm, the obtained data can also be compared with the original data to generate an algorithm evaluation report; in addition, the present invention has a visualization interface, which can display the data very intuitively, and can automatically generate a vehicle braking report (accurate to milliseconds), and very accurately evaluate whether the braking is more reasonable before and after the algorithm is optimized. As autonomous driving algorithm developers iteratively develop algorithms, due to changes in the fusion control algorithm, the vehicle may choose different time points and different accelerations to brake when dealing with the same scenario. By comparing the performance of different autonomous driving algorithms in the same scenario, the autonomous driving algorithms before and after optimization can be used for offline data playback, thus eliminating interference from the autonomous driving environment.

[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. A method for evaluating and visualizing a multi-sensor autonomous driving algorithm, characterized in that: The following steps are involved: Step 1) The autonomous driving vehicle saves the data collected by multiple sensors as raw sensor data; Step 2) The original sensor data is calculated using the autonomous driving algorithm before optimization and the autonomous driving algorithm after optimization, so as to obtain the fusion control data of the autonomous driving algorithms before and after optimization under the same scenario; Step 3) Visualizing the original sensor data, the fused control data of the autonomous driving algorithm before optimization, and the fused control data of the autonomous driving algorithm after optimization; Step 4) Load the fusion control data of the autonomous driving algorithm before and after optimization, select a comparison period, and generate an algorithm evaluation report by comparing the changes in the vehicle's braking time and braking acceleration under the same scenario within the comparison period.

2. The method for evaluating and visualizing a multi-sensor autonomous driving algorithm according to claim 1, characterized in that: In step 2), the historical data of the same period of the autonomous driving algorithm before and after optimization are replayed and saved respectively, and the historical data includes the original sensor data and the fusion control data.

3. The method for evaluating and visualizing a multi-sensor autonomous driving algorithm according to claim 2, characterized in that: In step 3), the historical data of the same period of the autonomous driving algorithm before and after optimization saved in step 2) is displayed in real time and visualized, and the visualization method includes camera video display and overhead view display.

4. The method for evaluating and visualizing a multi-sensor autonomous driving algorithm according to claim 3, characterized in that: Step 3) also includes locating and displaying the braking time points in the historical data.

5. The method for evaluating and visualizing a multi-sensor autonomous driving algorithm according to claim 4, characterized in that: In step 4), a time period set before and after the braking time point is selected as the braking time period, and the braking time period is used as the comparison time period. An algorithm evaluation report is generated by comparing the changes in the braking moment and braking acceleration of the vehicle under the same scenario within the time period.

6. The method for evaluating and visualizing a multi-sensor autonomous driving algorithm according to claim 5, characterized in that: In step 4), when the automatic algorithms before and after optimization brake at the same time within the braking time period, the braking duration and braking acceleration of the automatic driving algorithms before and after optimization are compared, and the rationality of the automatic driving algorithms before and after optimization is judged based on the comparison results and an algorithm evaluation report is generated.

7. The method for evaluating and visualizing a multi-sensor autonomous driving algorithm according to claim 6, characterized in that: In step 4), when the automatic algorithms before and after optimization brake at different times within the braking time period, the braking time points of the automatic driving algorithms before and after optimization are recorded respectively, and the rationality of the automatic driving algorithms before and after optimization is judged based on the braking time points and an algorithm evaluation report is generated.

8. A system for implementing the method for evaluating and visualizing a multi-sensor autonomous driving algorithm as claimed in any one of claims 1 to 7, characterized in that: include: A sensor assembly, the sensor assembly comprising a plurality of sensors distributed on the autonomous driving vehicle, the sensor assembly being used to collect autonomous driving data of the autonomous driving vehicle as raw sensor data; A data acquisition device, the data acquisition device is connected to the sensor assembly and is used to store raw sensor data collected by the sensor assembly; An automatic driving algorithm module, the automatic driving algorithm module is connected to the data acquisition device and is used to perform calculations based on the original sensor data and obtain fused control data; A data storage module, which is connected to the automatic driving algorithm module and is used to store the calculated fusion control data; A data reading module, the data reading module is connected to the data acquisition device and the data storage module respectively, so as to read the original sensor data and the fusion control data from the data acquisition device and the data storage module respectively; An offline visualization module, which is connected to the data reading module and is used to display in real time the raw sensor data of the same period and the fused control data obtained by the automatic driving algorithm before and after optimization, and the offline visualization module includes a camera display module and a bird's-eye view display module, the camera display module is used to display the data by camera video, and the bird's-eye view display module is used to display vehicle and lane line information; An algorithm evaluation module, which is connected to the offline visualization module, is used to evaluate the braking data of the autonomous driving algorithm before and after optimization and generate an algorithm evaluation report.

Images