Automatic driving vehicle driving scene reconstruction and decision deduction evaluation method and system

Abstract

The present application relates to a kind of automatic driving vehicle driving scene reconstruction and decision deduction evaluation method and system, comprising the following steps: reconstructing the driving scene in virtual environment using the driving data of a certain moment in real driving scene, and using the moment as the starting time of virtual environment, using scene reactive model to update the state information of all elements in driving scene in next moment virtual environment, after iteration, a series of continuous time driving scene of virtual environment is obtained;Wherein, the scene reactive model is based on the driving data of a plurality of continuous time of automatic driving vehicle in real world driving scene, and is trained in advance;Based on the series of continuous time driving scene of virtual environment obtained, the automatic driving decision effect of automatic driving vehicle is deduced and evaluated.The present application can be widely applied in the field of intelligent vehicle.

Description

Technical Field

[0001] This invention belongs to the field of intelligent vehicles, specifically relating to a method and system for reconstructing driving scenarios and making decisions based on a reactive model for autonomous vehicles. Background Technology

[0002] In recent years, artificial intelligence technology has gradually begun to be commercially applied in the fields of intelligent transportation and vehicles, and intelligent connected vehicles are gradually coming into people's view. Generally speaking, the autonomous driving system of an autonomous vehicle can be divided into four modules: perception, localization, decision-making, and control. Among them, the decision-making algorithm configured in the decision-making module is equivalent to the brain of the autonomous vehicle, playing a crucial role in achieving high-level autonomous driving and even driverless driving.

[0003] Because testing and training decision-making algorithms in real-world environments is costly, reconstructing driving scenarios using real data in virtual environments is a promising approach. Currently, the mainstream method for reconstructing driving scenarios involves directly replaying the collected real-world data in a virtual environment. However, in this case, people and vehicles in the virtual environment do not react accordingly to different decisions made by autonomous vehicles, thus failing to adequately evaluate and simulate potential real-world situations. Summary of the Invention

[0004] To address the aforementioned problems, the purpose of this invention is to provide a method and system for reconstructing driving scenarios and making decisions for autonomous vehicles. By using a reactive model to simulate the reactions of other people, vehicles, and other objects in the real environment to the driving decisions of autonomous vehicles, the invention can effectively improve the decision-making performance of autonomous vehicles.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a method for reconstructing driving scenarios and making decisions for evaluating autonomous vehicles, comprising the following steps:

[0007] The driving scenario is reconstructed in a virtual environment using driving data at a certain moment in a real driving scenario. This moment is used as the starting moment of the virtual environment. The state information of all elements in the driving scenario in the virtual environment is updated at the next moment using a scenario-reactive model. After iteration, a series of driving scenarios in the virtual environment at consecutive moments are obtained.

[0008] The scenario-responsive model is pre-trained based on driving data collected from autonomous vehicles at multiple consecutive moments in real-world driving scenarios.

[0009] The effectiveness of autonomous driving decisions of autonomous vehicles is evaluated based on a series of consecutive moments in the virtual driving environment.

[0010] Furthermore, the driving data at each moment in the real-world driving scenario includes the current state of each element in the driving scenario. The state of each element is represented by a vector of a preset dimension. The elements in the driving scenario include dynamic elements and static elements. The dynamic elements include pedestrians and vehicles, and the static elements include lane topology and traffic rules.

[0011] Furthermore, the scenario-responsive model includes vehicle dynamics models for autonomous vehicles as well as response models for other vehicles and pedestrians;

[0012] The input to the vehicle dynamics model of the autonomous vehicle is the state information and decision-making action of the autonomous vehicle at the current moment; the output is the state information of the autonomous vehicle at the next moment; the state information of the autonomous vehicle at the current moment includes the vehicle's position, angle and speed;

[0013] The input to the reaction model for other vehicles and pedestrians is the environmental state information at the current moment, including the state information of dynamic and static elements contained in the driving environment at the current moment; the output is the state information of dynamic and static elements contained in the driving environment at the next moment.

[0014] Furthermore, the vehicle dynamics model of the autonomous vehicle is based on a neural network; the reaction models of other vehicles and pedestrians are based on a graph neural network.

[0015] Furthermore, the method of reconstructing the driving scenario in a virtual environment using driving data at a certain moment in a real driving scenario, and using that moment as the starting moment of the virtual environment, updating the state information of all elements in the driving scenario in the virtual environment at the next moment using a scene reactive model, and iterating to obtain a series of consecutive moments of the virtual environment's driving scenario, includes:

[0016] Based on driving data at a certain moment in a real-world driving scenario, the driving scenario is reconstructed in a virtual environment, and this data is used as the starting point for the reconstruction of the virtual driving scenario.

[0017] By reconstructing the state information of all elements in the virtual driving scene at the start time and inputting the decision-making actions of the autonomous vehicle into the trained scene reaction model, the state information of all elements in the virtual driving scene at the next moment can be obtained.

[0018] Repeat the previous step to obtain a series of virtual driving scenarios at consecutive moments, each moment containing the state information of all elements in the virtual driving scenario.

[0019] Furthermore, the method for extrapolating and evaluating the autonomous driving decision-making performance of autonomous vehicles based on a series of consecutive virtual environment driving scenarios includes:

[0020] The cost of the virtual driving scenario at a series of consecutive time points is calculated based on the evaluation function.

[0021] Based on the cost value of a series of consecutive virtual scenes, a comprehensive evaluation result of the autonomous driving decision-making effect is obtained.

[0022] Furthermore, the comprehensive evaluation result of the autonomous driving decision-making effect based on the cost value of the virtual scene at a series of consecutive time moments includes: summing the cost values ​​of the virtual scene at a series of consecutive time moments, and evaluating the autonomous driving decision-making effect based on the total cost value.

[0023] Secondly, the present invention provides an autonomous vehicle driving scenario reconstruction and decision-making inference and evaluation system, comprising:

[0024] The scene reconstruction module is used to reconstruct the driving scene in a virtual environment using driving data at a certain moment in a real driving scene. The moment is used as the starting moment of the virtual environment. The scene reaction model is used to update the state information of all elements in the driving scene in the virtual environment at the next moment. After iteration, a series of consecutive moments of the virtual environment driving scene are obtained. The scene reaction model is pre-trained based on the driving data of the autonomous vehicle at multiple consecutive moments in the real world driving scene.

[0025] The effect evaluation module is used to extrapolate and evaluate the autonomous driving decision-making effect of autonomous vehicles based on a series of consecutive virtual driving scenarios.

[0026] Thirdly, the present invention provides a processing device, the processing device including at least a processor and a memory, the memory storing a computer program, and the processor executing the steps of the autonomous vehicle driving scenario reconstruction and decision inference evaluation method when running the computer program.

[0027] Fourthly, the present invention provides a computer storage medium storing computer-readable instructions thereon, which can be executed by a processor to implement the steps of the autonomous vehicle driving scenario reconstruction and decision inference evaluation method.

[0028] The present invention, by adopting the above technical solution, has the following advantages: The driving scenario reconstruction method provided by the present invention simulates the reactions of other people, vehicles, and other objects in the real environment to the driving decision results of autonomous vehicles through a reactive model, thereby evaluating the decision-making effect of autonomous vehicles. This avoids evaluating and testing the decision-making effect of autonomous vehicles in the real world. Attached Figure Description

[0029] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. In the drawings:

[0030] Figure 1 This is a flowchart of the autonomous vehicle driving scenario reconstruction method provided in an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0032] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0033] In some embodiments of the present invention, a method for reconstructing and evaluating driving scenarios for autonomous vehicles is provided. After collecting real-world driving data, including information such as the position and speed of obstacles in the surrounding environment, a reactive model is used to construct the reactions of other people, vehicles, and other objects in the surrounding environment to the driving decisions made by the autonomous vehicle, thereby reconstructing the real-world driving scenario. By simulating the reactions of other people, vehicles, and other objects in the real environment to the driving decisions made by the autonomous vehicle using a reactive model, the decision-making effectiveness of the autonomous vehicle can be evaluated. This avoids evaluating and testing the decision-making effectiveness of the autonomous vehicle in the real world.

[0034] Correspondingly, other embodiments of the present invention provide an autonomous vehicle driving scenario reconstruction and decision-making inference and evaluation system, device and medium.

[0035] Example 1

[0036] like Figure 1 As shown, this embodiment provides a method for reconstructing driving scenarios and making decision-making inferences and evaluations for autonomous vehicles, including the following steps:

[0037] S1: Based on the driving data collected from multiple consecutive moments in real-world driving scenarios of autonomous vehicles, a pre-built scenario-responsive model is trained.

[0038] S2: Reconstruct the driving scenario in the virtual environment using driving data at a certain moment, and take that moment as the starting moment of the virtual environment. Use the scenario reactive model in step S2 to update the state information of all elements in the driving scenario in the virtual environment at the next moment. After iteration, a series of driving scenarios in the virtual environment at consecutive moments are obtained as the deduction result.

[0039] S3: Based on the obtained driving scenarios of a series of consecutive virtual environments, evaluate the autonomous driving decision-making effect of autonomous vehicles.

[0040] Preferably, in step S1 above, the driving data at each moment includes the current state of each element in the real-world driving scenario, and the state dimension of each element is fixed. The elements in the real-world driving scenario include dynamic elements and static elements. Dynamic elements include pedestrians, vehicles, etc., while static elements include lane topology and traffic rules, such as the color status of traffic lights.

[0041] Preferably, in step S1 above, the pre-built scenario reaction model includes the vehicle dynamics model of the autonomous vehicle and the reaction models of other vehicles and pedestrians in the driving environment.

[0042] Preferably, the input to the vehicle dynamics model of the aforementioned autonomous vehicle is the state information of the autonomous vehicle at the current moment and the decision action 'a', and the output is the state information of the autonomous vehicle at the next moment. The state information of the autonomous vehicle at the current moment includes the vehicle's position, angle, and velocity (x, y, vx, vy, yaw), where x and y are the coordinates of the autonomous vehicle, vx and vy are the velocities of the autonomous vehicle in the x and y directions, respectively, and yaw is the current orientation angle of the autonomous vehicle.

[0043] Preferably, the vehicle dynamics model of the autonomous vehicle can be constructed using neural network methods or other learning-based methods, as long as the input and output are consistent with those described above.

[0044] Preferably, the reaction models for other vehicles and pedestrians mentioned above are constructed based on graph neural networks. The graph neural network consists of multiple nodes and connections between them, with each node representing an element in the driving environment at the current moment. The input to the graph neural network is the environmental state information at the current moment, including the states of dynamic elements such as people and vehicles, as well as static elements such as lanes, lane boundaries, traffic lights, and pedestrian crossings; the output is the environmental state information at the next moment.

[0045] Preferably, step S2 can be implemented through the following steps:

[0046] Step S21: Based on the driving data at a certain moment in the real-world driving scenario, reconstruct the driving scenario in the virtual environment and use it as the virtual driving scenario at the starting moment of the reconstruction.

[0047] Reconstructing the driving scene in a virtual environment refers to building a driving scene in a virtual environment that is completely identical to the real-world driving scene, using this as the starting point for reconstruction. This means that the starting point for reconstruction in the virtual environment includes all element state information from a specific moment in the real-world driving scene. Any frame from the real-world driving scene can be used as the starting point for reconstruction in the virtual environment.

[0048] Step S22: Input the state information of all elements in the virtual driving scene at the initial moment of reconstruction and the decision action a of the autonomous vehicle into the trained scene reaction model to obtain the state information of all elements in the virtual driving scene at the next moment.

[0049] Step S23: Repeat step S22 to obtain a series of virtual driving scenarios at consecutive moments, each moment containing the state information of all elements in the driving scenario.

[0050] Preferably, step S3 can be implemented through the following steps:

[0051] Step S31: Calculate the cost of the virtual driving scenarios at a series of consecutive time points obtained in step S2 based on the evaluation function.

[0052] When calculating the cost of the virtual driving scenario at each moment based on the evaluation function, the function can be selected according to the design requirements. For example, the evaluation function can be defined as rewarding faster driving speeds.

[0053] Step S32: Based on the cost value of the virtual scene at a series of consecutive moments, obtain a comprehensive evaluation result of the autonomous driving decision-making effect.

[0054] In the comprehensive evaluation of the effectiveness of autonomous driving decision-making, the cost value of a series of virtual scenarios at consecutive moments is summed up, and the lower the total cost value, the better the effectiveness of autonomous driving decision-making.

[0055] Example 2

[0056] The above-described embodiment 1 provides a method for reconstructing driving scenarios and making decisions for evaluating autonomous vehicles. Correspondingly, this embodiment provides a system for reconstructing driving scenarios and making decisions for evaluating autonomous vehicles. The system provided in this embodiment can implement the method for reconstructing driving scenarios and making decisions for evaluating autonomous vehicles in embodiment 1. This system can be implemented through software, hardware, or a combination of both. For example, the system may include integrated or separate functional modules or units to execute the corresponding steps in the methods of embodiment 1. Since the system in this embodiment is basically similar to the method embodiment, the description process in this embodiment is relatively simple. For relevant details, please refer to the description of embodiment 1. The system embodiment provided in this embodiment is merely illustrative.

[0057] This embodiment provides an autonomous vehicle driving scenario reconstruction and decision-making inference and evaluation system, including:

[0058] The scene reconstruction module is used to reconstruct the driving scene in a virtual environment using driving data at a certain moment in a real driving scene. The moment is used as the starting moment of the virtual environment. The scene reaction model is used to update the state information of all elements in the driving scene in the virtual environment at the next moment. After iteration, a series of consecutive moments of the virtual environment driving scene are obtained. The scene reaction model is pre-trained based on the driving data of the autonomous vehicle at multiple consecutive moments in the real world driving scene.

[0059] The effect evaluation module is used to extrapolate and evaluate the autonomous driving decision-making effect of autonomous vehicles based on a series of consecutive virtual driving scenarios.

[0060] Example 3

[0061] This embodiment provides a processing device corresponding to the autonomous vehicle driving scenario reconstruction and decision inference evaluation method provided in Embodiment 1. The processing device can be a client-side processing device, such as a mobile phone, laptop, tablet, desktop computer, etc., to execute the method of Embodiment 1.

[0062] The processing device includes a processor, a memory, a communication interface, and a bus. The processor, memory, and communication interface are connected via the bus to enable communication between them. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the autonomous vehicle driving scenario reconstruction and decision-making evaluation method provided in Embodiment 1.

[0063] In some embodiments, the memory may be high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device.

[0064] In other embodiments, the processor can be a general-purpose processor of various types, such as a central processing unit (CPU) or a digital signal processor (DSP), and is not limited thereto.

[0065] Example 4

[0066] The autonomous vehicle driving scenario reconstruction method of this embodiment 1 can be specifically implemented as a computer program product. The computer program product may include a computer-readable storage medium on which computer-readable program instructions are loaded for executing the autonomous vehicle driving scenario reconstruction and decision inference evaluation method described in this embodiment 1.

[0067] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for reconstructing driving scenarios and performing decision-making simulations and evaluations for autonomous vehicles, characterized in that, Includes the following steps: The driving scenario is reconstructed in a virtual environment using driving data at a certain moment in a real driving scenario. This moment is used as the starting moment of the virtual environment. The state information of all elements in the driving scenario in the virtual environment is updated at the next moment using a scenario-reactive model. After iteration, a series of driving scenarios in the virtual environment at consecutive moments are obtained. The scenario-responsive model is pre-trained based on driving data collected from autonomous vehicles at multiple consecutive moments in real-world driving scenarios. Based on the obtained driving scenarios in a series of consecutive virtual environments, the autonomous driving decision-making effect of autonomous vehicles is simulated and evaluated. The method of evaluating the autonomous driving decision-making performance of autonomous vehicles based on a series of consecutive virtual environment driving scenarios includes: The evaluation function is used to calculate the cost of the virtual driving scenario at a series of consecutive time points. Based on the cost value of a series of consecutive virtual scenes, a comprehensive evaluation result of the autonomous driving decision-making effect is obtained.

2. The method for reconstructing driving scenarios and making decisions for evaluating autonomous vehicles as described in claim 1, characterized in that, The driving data at each moment in the real-world driving scenario includes the current state of each element in the driving scenario. The state of each element is represented by a vector of a preset dimension. The elements in the driving scenario include dynamic elements and static elements. The dynamic elements include pedestrians and vehicles, and the static elements include lane topology and traffic rules.

3. The method for reconstructing driving scenarios and making decisions for evaluating autonomous vehicles as described in claim 2, characterized in that, The scenario-responsive model includes vehicle dynamics models for autonomous vehicles and response models for other vehicles and pedestrians; The input to the vehicle dynamics model of the autonomous vehicle is the state information and decision-making action of the autonomous vehicle at the current moment; the output is the state information of the autonomous vehicle at the next moment; the state information of the autonomous vehicle at the current moment includes the vehicle's position, angle and speed; The input to the reaction model for other vehicles and pedestrians is the environmental state information at the current moment, including the state information of dynamic and static elements contained in the driving environment at the current moment; the output is the state information of dynamic and static elements contained in the driving environment at the next moment.

4. The method for reconstructing driving scenarios and making decisions for evaluating autonomous vehicles as described in claim 3, characterized in that, The vehicle dynamics model of the autonomous vehicle is based on a neural network; the reaction models of other vehicles and pedestrians are based on a graph neural network.

5. The method for reconstructing driving scenarios and making decisions for evaluating autonomous vehicles as described in claim 1, characterized in that, The method describes reconstructing a driving scenario in a virtual environment using driving data from a real driving scenario at a specific moment, using that moment as the starting point of the virtual environment, and updating the state information of all elements in the driving scenario in the virtual environment at the next moment using a scenario-reactive model. This iterative process yields a series of consecutive moments representing the driving scenario in the virtual environment. This iterative process is a decision-making process, including: Based on driving data at a certain moment in a real-world driving scenario, the driving scenario is reconstructed in a virtual environment, and this data is used as the starting point for the reconstruction of the virtual driving scenario. By reconstructing the state information of all elements in the virtual driving scene at the start time and inputting the decision-making actions of the autonomous vehicle into the trained scene reaction model, the state information of all elements in the virtual driving scene at the next moment can be obtained. Repeat the previous step to obtain a series of virtual driving scenarios at consecutive moments, each moment containing the state information of all elements in the virtual driving scenario.

6. The method for reconstructing driving scenarios and making decisions for evaluating autonomous vehicles as described in claim 1, characterized in that, The method of obtaining a comprehensive evaluation result of the autonomous driving decision-making effect based on the cost value of the virtual scene at a series of consecutive time moments includes: summing the cost values ​​of the virtual scene at a series of consecutive time moments, and evaluating the autonomous driving decision-making effect based on the total cost value.

7. A system for reconstructing driving scenarios and performing decision-making and evaluation for autonomous vehicles, characterized in that, include: The scene reconstruction module is used to reconstruct the driving scene in a virtual environment using driving data at a certain moment in a real driving scene. The moment is used as the starting moment of the virtual environment. The scene reaction model is used to update the state information of all elements in the driving scene in the virtual environment at the next moment. After iteration, a series of consecutive moments of the virtual environment driving scene are obtained. The scene reaction model is pre-trained based on the driving data of the autonomous vehicle at multiple consecutive moments in the real world driving scene. The effect evaluation module is used to evaluate the autonomous driving decision-making effect of autonomous vehicles based on the driving scenario of a series of consecutive virtual time moments. The evaluation of the autonomous driving decision-making performance of autonomous vehicles is based on a series of consecutive virtual environment driving scenarios, including: The evaluation function is used to calculate the cost of the virtual driving scenario at a series of consecutive time points. Based on the cost value of a series of consecutive virtual scenes, a comprehensive evaluation result of the autonomous driving decision-making effect is obtained.

8. A processing apparatus, the processing apparatus comprising at least a processor and a memory, the memory storing a computer program, characterized in that, When the processor runs the computer program, it performs steps to implement the autonomous vehicle driving scenario reconstruction and decision inference evaluation method according to any one of claims 1 to 6.

9. A computer storage medium, characterized in that, It stores computer-readable instructions that can be executed by a processor to implement the steps of the autonomous vehicle driving scenario reconstruction and decision evaluation method according to any one of claims 1 to 6.

Images