Evaluation method and device of automatic driving system, equipment and storage medium

Abstract

The invention relates to the technical field of automatic driving, in particular to an evaluation method and device of an automatic driving system, equipment and a storage medium. The method comprises the following steps: firstly, collecting a real driving environment and a real driving behavior applied to a real own vehicle by a driver in order to cope with the real driving environment; a simulation environment is constructed according to a real driving environment, namely the simulation environment is used for simulating the real driving environment, then an automatic driving system is used for replacing a driver, and simulation driving behaviors made by the automatic driving system for controlling normal driving of a simulation vehicle in the simulation environment are collected. And finally, by taking the real driving behavior as a reference, evaluating whether the simulated driving behavior of the automatic driving system is matched with the real driving behavior so as to obtain an evaluation result of the automatic driving system. According to the method, the real driving behavior of the driver is taken as a reference, so that the evaluation result can reflect whether the simulated driving behavior given by the automatic driving system can be matched with the actual driving behavior, and the accuracy of the evaluation result is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an evaluation method, device, equipment and storage medium for an autonomous driving system. Background Art

[0002] The autonomous driving system has the function of assisting the driving of vehicles. Before the autonomous driving system goes online, its safety performance needs to be evaluated, that is, whether it can safely assist the driving of vehicles. The existing technology uses simulation tools to simulate driving scenarios, and controls the simulated vehicle to drive in the simulated driving scenarios through the autonomous driving system. The performance of the autonomous driving system is evaluated based on the driving information of the simulated vehicle. Since the actual driving behavior of the driver on the vehicle in the above driving scenarios is not taken into account, that is, the driving behavior of the driver is not used as a benchmark for evaluating the driving of the simulated vehicle by the autonomous driving system, the evaluation results of the autonomous driving system in the existing technology deviate from reality, which in turn leads to the low accuracy of the evaluation results of the autonomous driving system in the existing technology.

[0003] In summary, the evaluation results of the existing technology for autonomous driving systems have low accuracy.

[0004] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides an evaluation method, device, equipment and storage medium for an autonomous driving system, which solves the problem of low accuracy of evaluation results of autonomous driving systems in the prior art.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides an evaluation method for an autonomous driving system, comprising:

[0008] Acquire raw data generated by a driver driving a real vehicle, and filter the raw data to obtain a real driving scene, where the real driving scene includes a real driving environment and a real driving behavior generated by the driver in response to the real driving environment;

[0009] Constructing a simulation environment according to the real driving environment, and obtaining a simulated driving behavior generated by the automatic driving system for a simulated vehicle in the simulation environment;

[0010] An evaluation result of the automatic driving system is obtained based on the actual driving behavior and the simulated driving behavior.

[0011] In one implementation, the raw data is screened to obtain a real driving scene, including:

[0012] The raw data is screened to obtain each single driving scene;

[0013] Driving scenes with missing data are removed from each of the single driving scenes to obtain a real driving scene.

[0014] In one implementation, obtaining an evaluation result of the automatic driving system according to the real driving behavior and the simulated driving behavior includes:

[0015] Acquiring real vehicle motion data generated by the real self-vehicle under the action of the real driving behavior;

[0016] Acquiring vehicle simulation motion data generated by the simulated vehicle under the action of the simulated driving behavior;

[0017] An evaluation result of the automatic driving system is obtained based on the real motion data of the vehicle and the simulated motion data of the vehicle.

[0018] In one implementation, the real driving scenario is a single-lane following vehicle start scenario, and the evaluation result of the automatic driving system is obtained based on the real vehicle motion data and the simulated vehicle motion data, including:

[0019] Determine the actual mileage of the actual vehicle in the actual vehicle motion data within the start-up time period;

[0020] Determining the simulated mileage of the simulated vehicle in the vehicle simulated motion data within a start-up time period;

[0021] Determining a mileage difference between the actual mileage and the simulated mileage;

[0022] An evaluation result of the automatic driving system is obtained according to the mileage difference.

[0023] In one implementation, obtaining an evaluation result of the autonomous driving system according to the mileage difference includes:

[0024] Obtaining a reference threshold value of the actual driving mileage relative to the simulated driving mileage;

[0025] An evaluation result of the automatic driving system is obtained according to the mileage difference and the benchmark threshold.

[0026] In one implementation, the real driving scenario is a braking scenario of a preceding vehicle, and obtaining an evaluation result of the automatic driving system based on the real motion data of the vehicle and the simulated motion data of the vehicle includes:

[0027] Determine the real relative motion data of the real self-vehicle relative to the real preceding vehicle in the real vehicle motion data;

[0028] Determining simulated relative motion data of the simulated vehicle relative to a simulated preceding vehicle in the simulated vehicle motion data;

[0029] An evaluation result of the automatic driving system is obtained based on the real relative motion data and the simulated relative motion data.

[0030] In one implementation, obtaining an evaluation result of the automatic driving system according to the real relative motion data and the simulated relative motion data includes:

[0031] Applying a time-to-collision algorithm to the real relative motion data to predict the real time required for the real self-vehicle to collide with the real front vehicle;

[0032] Applying a time-to-collision algorithm to the simulated relative motion data to predict a simulation time required for the simulated vehicle to collide with the simulated preceding vehicle;

[0033] An evaluation result of the automatic driving system is obtained based on the real duration and the simulation duration.

[0034] In a second aspect, an embodiment of the present invention further provides an evaluation device for an autonomous driving system, wherein the device includes the following components:

[0035] A screening module is used to obtain raw data generated by a driver driving a real vehicle, and screen the raw data to obtain a real driving scene, wherein the real driving scene includes a real driving environment and a real driving behavior generated by the driver in response to the real driving environment;

[0036] A simulation module, used to construct a simulation environment based on the real driving environment, and obtain a simulated driving behavior generated by the automatic driving system for a simulated vehicle in the simulation environment;

[0037] An evaluation module is used to obtain an evaluation result of the automatic driving system based on the real driving behavior and the simulated driving behavior.

[0038] In a third aspect, an embodiment of the present invention further provides a terminal device, wherein the terminal device includes a memory, a processor, and an evaluation program for an autonomous driving system stored in the memory and executable on the processor, and when the processor executes the evaluation program for the autonomous driving system, the steps of the evaluation method for the autonomous driving system described above are implemented.

[0039] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which an evaluation program for an autonomous driving system is stored. When the evaluation program for the autonomous driving system is executed by a processor, the steps of the evaluation method for the autonomous driving system described above are implemented.

[0040] Beneficial effects: The present invention first collects the real driving environment and the real driving behavior imposed on the real vehicle by the driver in order to cope with the real driving environment. And constructs a simulation environment based on the real driving environment, that is, uses the simulation environment to simulate the real driving environment, and then replaces the driver with the automatic driving system, and collects the simulated driving behavior made by the automatic driving system to control the normal driving of the simulated vehicle in the simulated environment. Finally, the real driving behavior is used as a benchmark to evaluate whether the simulated driving behavior of the automatic driving system is consistent with the real driving behavior, so as to obtain the evaluation result of the automatic driving system. From the above analysis, it can be seen that the present invention takes the real driving behavior of the driver as a benchmark, so that the evaluation result can reflect whether the simulated driving behavior given by the automatic driving system can match the actual driving behavior, thereby improving the accuracy of the evaluation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is the overall flow chart of the present invention.

[0042] Figure 2 This is a flowchart for scoring an autonomous driving system in an embodiment of the present invention;

[0043] Figure 3 is a two-dimensional schematic diagram of scoring in an embodiment of the present invention;

[0044] Figure 4 A three-dimensional schematic diagram of scoring in an embodiment of the present invention;

[0045] Figure 5 A structural diagram of an evaluation device for an automatic driving system provided by the present invention;

[0046] Figure 6 This is a block diagram of the internal structure principle of the terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following is a clear and complete description of the technical solution of the present invention in combination with the embodiments and the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Research has found that the autonomous driving system has the function of assisting the driving of vehicles. Before the autonomous driving system goes online, its safety performance needs to be evaluated, that is, whether it can safely assist the driving of vehicles. The existing technology uses simulation tools to simulate driving scenarios, and controls the simulated vehicle to drive in the simulated driving scenario through the autonomous driving system. The performance of the autonomous driving system is evaluated based on the driving information of the simulated vehicle. Since the actual driving behavior of the driver on the vehicle in the above driving scenario is not taken into account, that is, the driving behavior of the driver is not used as a benchmark for evaluating the driving of the simulated vehicle by the autonomous driving system, the evaluation results of the autonomous driving system in the existing technology deviate from reality, which in turn leads to the low accuracy of the evaluation results of the autonomous driving system in the existing technology.

[0049] In order to solve the above technical problems, the present invention provides an evaluation method, device, equipment and storage medium for an autonomous driving system, which solves the problem of low accuracy of evaluation results of autonomous driving systems in the prior art. In specific implementation, the original data generated by the driver driving a real vehicle is first obtained, and the original data is screened to obtain a real driving scene, which includes a real driving environment and the real driving behavior generated by the driver for the real driving environment; then, based on the real driving environment, a simulation environment is constructed, and the simulated driving behavior generated by the autonomous driving system for the simulated vehicle in the simulation environment is obtained; finally, based on the real driving behavior and the simulated driving behavior, the evaluation result of the autonomous driving system is obtained.

[0050] For example, the real driving scenario is that a real vehicle overtakes a slow-moving vehicle ahead of it at a speed of 120 km / h on a smooth highway. The smooth highway, the real vehicle being 8 meters away from the vehicle ahead, and the slow-moving vehicle ahead are the real driving environment; overtaking the vehicle ahead at a speed of 120 km / h is the real driving behavior of the driver.

[0051] A simulation environment is constructed according to the real driving environment, that is, the real driving environment is simulated by the simulation environment. The automatic driving system controls the simulated vehicle to overtake the simulated front vehicle in the simulation environment, and collects the simulated speed when the simulated vehicle overtakes the simulated front vehicle. This speed is the simulated driving behavior.

[0052] If the above simulation speed is very close to 120km / h, it means that the control of the vehicle by the automatic driving system is close to the control of the vehicle by the driver, that is, the automatic driving system has good control performance over the vehicle. On the contrary, if the above simulation speed is far different from 120km / h, it means that the automatic driving system cannot replace the driver to control the vehicle, that is, the performance of the automatic driving system is not good.

[0053] The evaluation method of the automatic driving system of this embodiment can be applied to a terminal device, which can be a terminal product with data processing function, such as a computer. Figure 1 As shown in , the evaluation method of the autonomous driving system specifically includes the following steps:

[0054] S100, obtaining raw data generated by a driver driving a real vehicle, and filtering the raw data to obtain a real driving scene, where the real driving scene includes a real driving environment and a real driving behavior generated by the driver in response to the real driving environment;

[0055] S200, constructing a simulation environment according to the real driving environment, and obtaining a simulated driving behavior generated by the automatic driving system for a simulated vehicle in the simulation environment;

[0056] S300: Obtaining an evaluation result of the automatic driving system based on the actual driving behavior and the simulated driving behavior.

[0057] Embodiment 1: In this embodiment, step S100 includes the following specific steps: screening the original data to obtain each single driving scene; removing the driving scenes with missing data from each of the single driving scenes to obtain a real driving scene.

[0058] like Figure 2 As shown, the original data is collected, and the original data is used to characterize the original driving scene of the real self-vehicle. After the original data is collected, the original data is cleaned to remove redundant data. After cleaning, human driving trip, human driving bag, and human driving slice are screened out from the original data. Among them, human driving trip refers to human driving mileage, that is, a human driving trip is the complete data collected by the data collection tool on the vehicle after the driver drives the vehicle to travel a route. The duration corresponding to a human driving trip is 1h~4h; a human driving bag is a data packet formed by the data collected by the vehicle within 30s of the driving distance, and a human driving bag constitutes a human driving trip; a human driving slice is a data segment or slice collected by the vehicle during driving. A slice is a period of data of a trip, which may span multiple bags. The slice in this embodiment is a slice that has been labeled one by one by the pre-mining tool, such as the original data with labels such as this section of road is a highway, or this section of road is a four-lane road, etc., which can be used to run the automatic scene screening G1 process to mine the required single category scene.

[0059] like Figure 2As shown in the figure, the human driving trip, human driving bag, and human driving slice are quality inspected respectively to screen out the human driving trip, human driving bag, and human driving slice with better quality, and then the automatic scene screening G1 is performed. The automatic scene screening is to screen out each single topic scene, and then the bag-level quality inspection is performed on each single category scene to exclude the single category scene with missing pinner data fragments, or exclude the single category scene with missing traj data fragments, and then determine the topic corresponding to each single category scene. After manual review, data fragments are screened out from each data fragment of a single scene, and the screened data fragments are simulated on the simulation platform to obtain a simulation environment, and the simulation environment is scored to determine whether the simulation environment is close to the real scene.

[0060] A single-category scenario is a scenario that corresponds to only one topic. For example, the intersections include intersection A, intersection B, intersection C, and intersection D. A vehicle turning left from intersection A to intersection B is a single-category scenario, recorded as single-category scenario A; a vehicle turning left from intersection C to intersection D is also a single-category scenario, recorded as single-category scenario B. The topics corresponding to single-category scenarios A and B are both left turns. For example, the self-vehicle not changing lanes is a topic, the front vehicle speed is zero is a topic, and the self-vehicle keeping up with the current vehicle in time after it starts is also a topic.

[0061] RawTagSlice is used to represent the attribute information of a single category of scenes, and the attribute information includes TripID (TripID is the trip ID), StartTimestamp (StartTimestamp is the start timestamp of the trip), and EndTimestamp (EndTimestamp is the end timestamp of the trip).

[0062] The autonomous driving system is evaluated using several single-category scenarios belonging to the same topic, and the autonomous driving system is evaluated or scored using a single dimension. The evaluation result or scoring result represents whether the autonomous driving system can control the vehicle to drive safely in the scenario belonging to the topic.

[0063] The real driving scene includes the real driving environment and real driving behavior. The real driving environment includes the real traffic environment and the real self-vehicle's real-time motion data and road information. The real traffic environment includes the real-time motion data of the vehicles around the real self-vehicle and the information of traffic lights. The real self-vehicle is the self-vehicle, and the surrounding vehicles are the vehicles around the self-vehicle. The real driving behavior refers to the driver's control behavior on the real self-vehicle, such as braking, accelerating, overtaking, etc.

[0064] Embodiment 2 is based on embodiment 1. In this embodiment, the actual driving scenario is a single-lane following vehicle start scenario. The single-lane following vehicle start scenario is a scenario in which the vehicle follows the start of the preceding vehicle on a single lane.

[0065] In this embodiment, step S300 includes the following specific steps S301a to S305a:

[0066] S301a, obtaining the real vehicle motion data generated by the real vehicle under the real driving behavior, the real vehicle motion data being the real driving mileage within the start-up time period.

[0067] The start-up time period ranges from 10 seconds to 20 seconds, and the actual mileage is the actual distance traveled by the vehicle within the start-up time period under the control of the driver.

[0068] S302a, obtaining vehicle simulation motion data generated by the simulated vehicle under the simulated driving behavior, wherein the vehicle simulation motion data is the simulated driving mileage of the simulated vehicle within a startup time period.

[0069] The simulated mileage is the distance that the simulated vehicle travels during the startup period under the control of the automatic driving system.

[0070] S303a, determining a mileage difference x between the actual mileage and the simulated mileage.

[0071] The mileage difference x is the distance that the simulated vehicle lags behind the real vehicle during the startup period.

[0072] S304a: Obtain a reference threshold value of the actual driving mileage relative to the simulated driving mileage.

[0073] The reference threshold in this embodiment is 0.5.

[0074] S305a, obtaining an evaluation result y of the automatic driving system according to the mileage difference x and the reference threshold 0.5:

[0075]

[0076] The larger the mileage difference x, the smaller y is, that is, the farther the simulated vehicle lags behind the real vehicle, which means the performance of the autonomous driving system is worse; anyway, the smaller the mileage difference x, the larger y is, that is, the smaller the simulated vehicle lags behind the real vehicle, which means the performance of the autonomous driving system is better.

[0077] Embodiment 3, based on embodiment 1, in this embodiment, the real driving scene is the front vehicle braking scene, which is the scene where the vehicle brakes or decelerates following the front vehicle braking. In this embodiment, step S300 includes the following specific steps S301b to S305b:

[0078] S301b, obtaining real vehicle motion data generated by the real self-vehicle under the real driving behavior, the real vehicle motion data being real relative motion data of the real self-vehicle relative to the real preceding vehicle.

[0079] The real relative motion data includes the relative position, relative speed, and relative acceleration between the real self-vehicle and the real front vehicle.

[0080] S302b, obtaining vehicle simulation motion data generated by the simulated vehicle under the simulated driving behavior, wherein the vehicle simulation motion data is simulated relative motion data of the simulated vehicle relative to the simulated preceding vehicle.

[0081] The automatic driving system controls the movement of the simulated vehicle so that the simulated vehicle and the simulated vehicle in front of it move relative to each other. For example, the automatic driving system controls the simulated vehicle to move at a speed of -3m / s. 2 Braking, when the simulated vehicle is moving at -3m / s 2 When braking, there will be relative motion with the simulated front vehicle, where -3m / s 2 Braking is the simulated driving behavior. The simulated relative motion data includes the relative position, relative acceleration and relative speed between the simulated vehicle and the simulated front vehicle.

[0082] S303b, applying a time-to-collision algorithm to the real relative motion data to predict a real time length y′ required for the real self-vehicle to collide with the real front vehicle.

[0083] The time-to-collision algorithm is also known as the time to collision algorithm. The time to collision algorithm is also known as the TTC algorithm. The TTC algorithm is applied to the real relative motion data between the real self-vehicle and the real front vehicle. The TTC algorithm can predict how long it will take for the real self-vehicle to collide with the real front vehicle.

[0084] S304b, applying a time-to-collision algorithm to the simulated relative motion data to predict a simulation time length x′ required for the simulated vehicle to collide with the simulated front vehicle.

[0085] Similarly, the TTC algorithm is also used to predict how long it will take for the simulated vehicle to hit the simulated front vehicle.

[0086] S305b, obtaining an evaluation result z of the automatic driving system according to the real duration y′ and the simulation duration x′:

[0087]

[0088] The larger the z is, the better the performance of the autonomous driving system is; the smaller the z is, the worse the performance of the autonomous driving system is.

[0089] For the vehicle's own deceleration in the braking scenario, Figure 3 As shown, Figure 3 The horizontal axis is the simulation time x′, Figure 3 The ordinate is the real time y′, from Figure 3 It can be seen that when the driver's deceleration remains unchanged, that is, when the deceleration of the real vehicle remains unchanged, the greater the deceleration of the simulated vehicle, the smaller z. If the deceleration of the real vehicle remains unchanged but the deceleration of the simulated vehicle increases, it means that the automatic driving system cannot achieve the same driving effect as the driver, so the evaluation result is that z is smaller.

[0090] Figure 4 The relationship between the simulation duration x′ and the real duration y′ and z is intuitively shown. Figure 4 The x axis is x′, and the y axis is y′.

[0091] This embodiment also provides an evaluation device for an automatic driving system, such as Figure 5 As shown, the device comprises the following components:

[0092] The screening module 01 is used to obtain raw data generated by the driver driving the real vehicle, and screen the raw data to obtain a real driving scene, wherein the real driving scene includes a real driving environment and a real driving behavior generated by the driver in response to the real driving environment;

[0093] The simulation module 02 is used to construct a simulation environment according to the real driving environment, and obtain the simulated driving behavior generated by the automatic driving system for the simulated vehicle in the simulation environment;

[0094] The evaluation module 03 is used to obtain an evaluation result of the automatic driving system according to the real driving behavior and the simulated driving behavior.

[0095] Based on the above embodiments, the present invention further provides a terminal device, whose principle block diagram can be shown as follows: Figure 6As shown. The terminal device includes a processor, a memory, a network interface, and a display screen connected via a system bus. Among them, the processor of the terminal device is used to provide computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for evaluating an autonomous driving system is implemented. The display screen of the terminal device can be a liquid crystal display screen or an electronic ink display screen.

[0096] Those skilled in the art will understand that Figure 6 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal device to which the scheme of the present invention is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0097] In one embodiment, a terminal device is provided. The terminal device includes a memory, a processor, and an evaluation program of an autonomous driving system stored in the memory and executable on the processor. When the processor executes the evaluation program of the autonomous driving system, the following operation instructions are implemented:

[0098] Acquire raw data generated by a driver driving a real vehicle, and filter the raw data to obtain a real driving scene, where the real driving scene includes a real driving environment and a real driving behavior generated by the driver in response to the real driving environment;

[0099] Constructing a simulation environment according to the real driving environment, and obtaining a simulated driving behavior generated by the automatic driving system for a simulated vehicle in the simulation environment;

[0100] An evaluation result of the automatic driving system is obtained based on the actual driving behavior and the simulated driving behavior.

[0101] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for evaluating an automatic driving system, characterized in that: include: Acquire raw data generated by a driver driving a real vehicle, and filter the raw data to obtain a real driving scene, where the real driving scene includes a real driving environment and a real driving behavior generated by the driver in response to the real driving environment; Constructing a simulation environment according to the real driving environment, and obtaining a simulated driving behavior generated by the automatic driving system for a simulated vehicle in the simulation environment; An evaluation result of the automatic driving system is obtained based on the actual driving behavior and the simulated driving behavior.

2. The method for evaluating an automatic driving system according to claim 1, wherein: The raw data is screened to obtain a real driving scenario, including: The raw data is screened to obtain each single driving scene; Driving scenes with missing data are removed from each of the single driving scenes to obtain a real driving scene.

3. The method for evaluating an automatic driving system according to claim 1, wherein: Obtaining an evaluation result of the automatic driving system according to the real driving behavior and the simulated driving behavior, including: Acquiring real vehicle motion data generated by the real self-vehicle under the action of the real driving behavior; Acquiring vehicle simulation motion data generated by the simulated vehicle under the action of the simulated driving behavior; An evaluation result of the automatic driving system is obtained based on the real motion data of the vehicle and the simulated motion data of the vehicle.

4. The method for evaluating an automatic driving system according to claim 3, wherein: The real driving scenario is a single-lane following vehicle start scenario. The evaluation result of the automatic driving system is obtained based on the real vehicle motion data and the simulated vehicle motion data, including: Determine the actual mileage of the actual vehicle in the actual vehicle motion data within the start-up time period; Determining the simulated mileage of the simulated vehicle in the vehicle simulated motion data within a start-up time period; Determining a mileage difference between the actual mileage and the simulated mileage; An evaluation result of the automatic driving system is obtained according to the mileage difference.

5. The method for evaluating an automatic driving system according to claim 4, wherein: According to the mileage difference, an evaluation result of the autonomous driving system is obtained, including: Obtaining a reference threshold value of the actual driving mileage relative to the simulated driving mileage; An evaluation result of the automatic driving system is obtained according to the mileage difference and the benchmark threshold.

6. The method for evaluating an automatic driving system according to claim 3, wherein: The real driving scenario is a braking scenario of a preceding vehicle. The evaluation result of the automatic driving system is obtained based on the real motion data of the vehicle and the simulated motion data of the vehicle, including: Determine the real relative motion data of the real self-vehicle relative to the real preceding vehicle in the real vehicle motion data; Determining simulated relative motion data of the simulated vehicle relative to a simulated preceding vehicle in the simulated vehicle motion data; An evaluation result of the automatic driving system is obtained based on the real relative motion data and the simulated relative motion data.

7. The method for evaluating an automatic driving system according to claim 6, wherein: Obtaining an evaluation result of the automatic driving system according to the real relative motion data and the simulated relative motion data, including: Applying a time-to-collision algorithm to the real relative motion data to predict the real time required for the real self-vehicle to collide with the real front vehicle; Applying a time-to-collision algorithm to the simulated relative motion data to predict a simulation time required for the simulated vehicle to collide with the simulated preceding vehicle; An evaluation result of the automatic driving system is obtained based on the real duration and the simulation duration.

8. An evaluation device for an automatic driving system, characterized in that: The device comprises the following components: A screening module is used to obtain raw data generated by a driver driving a real vehicle, and screen the raw data to obtain a real driving scene, wherein the real driving scene includes a real driving environment and a real driving behavior generated by the driver in response to the real driving environment; A simulation module, used to construct a simulation environment based on the real driving environment, and obtain a simulated driving behavior generated by the automatic driving system for a simulated vehicle in the simulation environment; An evaluation module is used to obtain an evaluation result of the automatic driving system based on the real driving behavior and the simulated driving behavior.

9. A terminal device, characterized in that: The terminal device includes a memory, a processor, and an evaluation program for the autonomous driving system stored in the memory and executable on the processor. When the processor executes the evaluation program for the autonomous driving system, the steps of the evaluation method for the autonomous driving system as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an evaluation program for the autonomous driving system. When the evaluation program for the autonomous driving system is executed by the processor, the steps of the evaluation method for the autonomous driving system as described in any one of claims 1 to 7 are implemented.

Images