An autonomous driving deduction system and deduction method based on digital twins

Abstract

The present invention belongs to the field of autonomous driving technology, specifically an autonomous driving deduction system and method based on digital twins. The system of the present invention includes a digital twin scene building module, an environmental perception module, a simulation deduction module and a decision optimization module. The digital twin scene building module is used to model physical entities and build digital twin scenes based on a game engine; the environmental perception module is used to perceive the vehicle's surroundings in real time and estimate the vehicle's own state; the simulation deduction module is used to deduce multiple decision plans at the same time in parallel in a digital twin scene; the decision optimization module is used to evaluate the results of the simulation deduction, select the optimal decision plan at the current moment, and act in reverse on the physical entity. The present invention applies digital twin technology to the simulation deduction of autonomous driving, which can optimize autonomous driving decisions and can deal with emergencies in a targeted manner, thereby improving the safety and reliability of the autonomous driving system.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving technology, and in particular relates to an autonomous driving deduction system and deduction method. Background Art

[0002] In recent years, the advancement of electronic information, artificial intelligence, and digital technologies has driven the development of autonomous driving. In real-world autonomous driving scenarios, multiple decision-making options are faced at every moment, especially in the face of emergencies, where even the slightest misjudgment can lead to catastrophic consequences. Therefore, for autonomous driving to be truly implemented, it is crucial to ensure the accuracy of every decision, especially in the face of emergencies, which require faster and more accurate decisions.

[0003] Traditional autonomous driving simulation environments can simulate and simulate autonomous driving scenarios, but these environments require manual configuration of detailed settings. The interpretation of various behaviors in different environments requires manual input of numerous parameters, which requires rigorous professional knowledge. Given limited human resources, the ever-changing real world cannot be fully simulated through simple, repetitive parameter adjustments. Furthermore, traditional autonomous driving simulation environments can only simulate and simulate scenarios in a virtual environment, without direct interaction with the physical world. Summary of the Invention

[0004] The purpose of this invention is to propose an autonomous driving simulation system and method based on digital twins to solve the problems existing in the current autonomous driving simulation environment.

[0005] The digital twin-based autonomous driving deduction system proposed in the present invention includes: a digital twin scene construction module, an environment perception module, a simulation deduction module, and a decision optimization module, wherein:

[0006] The digital twin scene building module is used to model physical entities and build digital twin scenes based on the game engine;

[0007] The environment perception module is used to perceive the vehicle's surrounding environment and estimate the vehicle's own state in real time;

[0008] The simulation deduction module is used to perform parallel deduction of multiple decision-making plans at the same time in the digital twin scenario;

[0009] The decision optimization module is used to evaluate the results of simulation deduction, select the optimal decision plan at the current moment, and reversely act on the physical entity.

[0010] in:

[0011] The digital twin scene building module includes a twin model library, a weather system, and a real-time rendering visualization part, wherein:

[0012] The twin model library constructs digital twins of real physical entities. Each twin is composed of a series of real sub-object modules, each responsible for independent attribute definition, corresponding to the complex physical structure of objects in the real world.

[0013] The weather system can dynamically simulate the real effects of clouds, rain, snow, fog and lighting as well as the influence of related physical quantity coefficients;

[0014] The real-time rendering visualization part can instantiate and integrate the twin model based on the data of the object entity, and build a dynamic digital twin scene based on the game engine.

[0015] The environmental perception module can receive sensor data from the physical world in real time, perceive the surrounding environment, obtain real-time road condition information and pedestrian information feedback, accurately and effectively estimate its own status, and update it to the digital twin scene at the same time.

[0016] In the digital twin scenario, the simulation module combines the information feedback obtained by the environmental perception module and its own state estimation, and performs parallel simulation based on multiple possible decision-making schemes. The simulation is carried out in different processes without affecting each other and can ensure real-time performance. After the simulation is completed, the behavior and state under different decision-making schemes are predicted.

[0017] The simulation deduction module introduces an emergency sample library; the emergency sample library is built by simulating all conceivable emergency dangerous situations in the emergency sample library in advance; when encountering an emergency in a real situation, the emergency sample library is first searched. If a similar emergency can be retrieved, the corresponding decision-making plan is directly adopted to reduce the processing time of the emergency; otherwise, simulation deduction is performed again, and the emergency and corresponding decision are recorded in the emergency sample library.

[0018] The decision optimization module evaluates the simulation results from aspects such as the vehicle's own status and surrounding environment, and then selects the decision plan corresponding to the optimal result and performs vehicle control in the digital twin scene, thereby reversely controlling the autonomous driving vehicle in the physical world by controlling the vehicle twin in the digital twin scene.

[0019] The present invention also proposes a digital twin-based autonomous driving deduction method based on the above-mentioned autonomous driving deduction system, and the specific steps are as follows.

[0020] Step S1: Build an autonomous driving digital twin scenario

[0021] S1-1: Build a static twin model for objects in the autonomous driving scene (including vehicles, sensors, buildings, pedestrians, etc.);

[0022] S1-2: Classify the constructed static twin models and define the specific attributes and data objects of different categories of twins, so as to establish a static mapping relationship between twins and physical entities;

[0023] S1-3: Based on the game physics engine, the interaction process of physical entities in the physical world is simulated in the digital twin scene, thereby establishing a dynamic mapping relationship between the twin and the physical entity;

[0024] S1-4: Build a weather system to simulate the real effects of clouds, rain, snow, fog and lighting;

[0025] S1-5: Realize real-time rendering and visualization of autonomous driving digital twin scenes based on the game rendering engine.

[0026] Step S2: In the physical world, perceive and locate the vehicle’s surroundings and feed the results back to the digital twin scene.

[0027] S2-1: In the physical world, use sensors (including cameras, lidar, millimeter-wave radar, etc.) to perceive the environment around the vehicle and obtain information about surrounding vehicles, pedestrians, and other possible obstacles;

[0028] S2-2: In the physical world, use sensors (including cameras, temperature sensors, humidity sensors, etc.) to perceive current weather conditions;

[0029] S2-3: Based on road condition information and prior maps, the vehicle is positioned using SLAM technology.

[0030] S2-4: Feedback the environmental perception and positioning results into the digital twin scene.

[0031] Step S3: Simulate the digital twin scenario

[0032] S3-1: Based on the feedback from step S2-4 and the data from physical sensors, the traffic and weather conditions in the physical world are restored in the digital twin scene;

[0033] S3-2: Quickly make several reasonable decisions based on the current autonomous driving scenario;

[0034] S3-3: Parallel deduction of different autonomous driving decisions in step S3-2 on an event-by-event basis;

[0035] S3-4: Feedback the deduction results to the decision optimization module.

[0036] Step S4: Evaluate the simulation results and make corresponding decisions

[0037] S4-1: Analyze the results of step S3-4 and evaluate the quality of the autonomous driving decision based on the vehicle driving conditions in the simulation and whether there are potential accidents.

[0038] S4-2: Select the best decision-making plan;

[0039] S4-3: Control the autonomous driving vehicle in the physical world to make optimal decisions through reverse mapping of the vehicle twin and the physical entity.

[0040] In the autonomous driving simulation method of the present invention, the simulation simulation module can effectively deal with emergencies in autonomous driving scenarios, including:

[0041] Conduct simulations and deductions of emergencies in advance and build an emergency sample library;

[0042] In the emergency sample database, each emergency has a corresponding optimized decision-making plan, which is updated iteratively;

[0043] During the autonomous driving process, when encountering an emergency, a search and match is performed from the emergency sample library. When the match is successful, the corresponding decision-making plan is directly used. When the match fails, simulation deduction is performed to execute the optimized optimal decision.

[0044] The digital twin-based autonomous driving deduction system and method provided by the present invention applies digital twin technology to the simulation deduction of autonomous driving, which can not only optimize autonomous driving decisions, but also deal with emergencies in a targeted manner, thereby improving the response sensitivity of the autonomous driving system in the face of emergencies, thereby effectively improving the safety and reliability of the autonomous driving system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a structural block diagram of the digital twin-based autonomous driving deduction system of the present invention.

[0046] Figure 2 It is a structural block diagram of the digital twin scene building module in the present invention.

[0047] Figure 3 It is a structural block diagram of the simulation deduction module in the present invention.

[0048] Figure 4 It is a flowchart illustrating the emergency response in the simulation and deduction module of the present invention.

[0049] Figure 5 This is an example illustration of the digital twin-based autonomous driving simulation system of the present invention in an urban road scenario. DETAILED DESCRIPTION

[0050] The following is a detailed description of the digital twin-based autonomous driving deduction system and method provided by the present invention in conjunction with the accompanying drawings and specific embodiments.

[0051] Figure 1 This is a block diagram of the digital twin-based autonomous driving simulation system described in this invention. The system includes four modules: a digital twin scenario building module, an environmental perception module, a simulation simulation module, and a decision optimization module. Details are the same as those described in the Summary of the Invention.

[0052] Figure 2 This is a block diagram of the digital twin scenario building module in the digital twin-based autonomous driving simulation system described in the present invention. The specific implementation method includes the following steps:

[0053] Step S1: Take real weather information parameters as input

[0054] The real weather information parameters include visibility, temperature, wind speed, wind direction, rainfall, etc., which are used to build digital twin scenes by collecting real physical data;

[0055] Step S2: Take road condition information and pedestrian information as input

[0056] The road condition information parameters include real-time display information of traffic lights, traffic flow, road conditions, crosswalks, etc., and the real-time information is collected for use in building digital twin scenarios; the pedestrian information includes pedestrian flow, pedestrian categories, etc., and the real-time pedestrian data is collected for subsequent autonomous driving deduction of digital twins; in the actual implementation process, the road condition information and pedestrian information can be used for decision-making and deduction of autonomous driving; in actual applications, the road condition information can also include image sensor data information, position sensor, camera, etc.

[0057] Step S3: Building a digital twin simulation system

[0058] The digital twin deduction system is built based on the input of the above steps. The different weather information collected will give different inputs to the shader model. After calculations based on physical atmospheric scattering, volumetric clouds, volumetric light, wind speed and temperature changes with altitude, etc., a real-time digital twin weather scene is obtained; the collected road condition information and pedestrian information will be combined with the weather system to form the final digital twin deduction system.

[0059] The simulation deduction module in the digital twin-based autonomous driving deduction system of the present invention is described in detail in the following sections: Figure 3As shown. Among them, data input refers to the input of the digital twin scene construction, and deduction is performed in the digital twin scene. The twin that performs autonomous driving deduction will obtain the current visibility based on the weather information parameters, from which the maximum safe distance of the visual range can be deduced. The friction coefficient is obtained based on the current road condition information, from which the maximum braking distance under the current road conditions can be deduced. The pedestrian flow information in the scene is obtained based on the current pedestrian information, from which the next direction of travel of the pedestrian and the safe activity range of the current vehicle can be deduced. The digital twin autonomous driving deduction system described in the present invention can make decisions on the twin behavior based on the above information, and the specific steps are:

[0060] Step S1: Calculate the real-time driving parameters of the current twin based on real physical inputs. The driving parameters include maximum braking distance, visibility, visual range, maximum safe distance, probability of emergencies, etc.

[0061] Step S2: Based on the aforementioned driving parameters as input to the deduction system, a decision is made based on a model derived from a large amount of data, and the execution strategy and feasibility of the next moment are analyzed. The model derived from the large amount of data is derived based on the deduction system of the digital twin.

[0062] Step S3: Feedback the deduction results to the physical world to obtain the output results of the digital twin deduction system. The output results are the optimal operation results.

[0063] In the current field of autonomous driving, most simulations of autonomous driving behavior rely on physical simulation or non-physics-based simulations, resulting in wasted resources and precision errors. The autonomous driving simulation system combined with digital twins, provided by this invention, performs physics-based simulation and decision-making within digital twin scenarios, achieving more realistic results while ensuring accuracy and real-time performance, while also reducing verification costs.

[0064] In the digital twin-based autonomous driving simulation system of the present invention, the process of dealing with emergencies in the simulation simulation module is as follows: Figure 4 The specific steps are as follows:

[0065] Step S1: simulate and simulate emergencies in advance and build an emergency sample library;

[0066] Step S2: When an emergency occurs during the autonomous driving process, the corresponding emergency is searched and matched from the emergency sample library;

[0067] Step S3: If the corresponding emergency event is matched in the emergency event sample library, the corresponding decision plan is executed;

[0068] Step S4: If no corresponding emergency event is matched in the emergency event sample library, simulation deduction is performed in the simulation deduction module to obtain an optimized decision-making plan;

[0069] Step S5: Iteratively update the emergency event sample library.

[0070] Traffic emergencies are common in both normal vehicle driving and autonomous driving scenarios. When faced with emergencies, quick and accurate decision-making is essential. This invention, by introducing an emergency sample library, effectively reduces emergency processing time, ensuring efficient and accurate autonomous driving decision-making, thereby improving the safety and reliability of the autonomous driving system.

[0071] Figure 5 This demonstrates an example of the digital twin-based autonomous driving simulation system described in this invention operating in an urban road scenario. The details are as follows:

[0072] After the system is activated, the autonomous vehicle drives normally on real city roads, while its corresponding autonomous twin vehicle drives normally in the pre-built digital twin scene. The physical world entity and the twin in the digital twin scene correspond one-to-one. Vehicle data and sensor perception data in the physical world are updated in real time in the digital twin scene, ensuring synchronization between the digital twin scene and the physical world.

[0073] At some point, the autonomous vehicle encounters an emergency. The perception module detects a pedestrian in front of the vehicle in the physical world, prompting the system to enter the emergency handling phase. First, the system searches the emergency sample library. If a similar emergency is found in the sample library, the decision corresponding to that event—brake—is directly adopted. If no similar emergency is found in the sample library, different decision plans are planned based on the current situation, such as:

[0074] (1) Braking;

[0075] (2) Turn the steering wheel to the left and slow down;

[0076] (3) Turn the steering wheel to the left and accelerate;

[0077] Based on the above decision-making scenarios, a parallel simulation was conducted within the digital twin scenario. The simulation comprehensively considered vehicle speed, perceived pedestrian speed, road traffic conditions, and weather conditions. All possible accident-causing decision-making scenarios were eliminated, and the remaining scenarios were evaluated one by one, ultimately selecting the optimal decision: braking.

[0078] After obtaining the decision plan, the vehicle twin in the digital twin scene is controlled to perform braking operations, thereby performing reverse control on the real-world vehicle, so that the real-world vehicle also performs braking operations.

[0079] Finally, after the emergency is handled, the autonomous vehicle drives normally again, and the digital twin scenario runs synchronously until the end of the journey.

Claims

1. An autonomous driving deduction system based on digital twins, characterized by: include: Digital twin scenario building module, environmental perception module, simulation deduction module and decision optimization module, including: The digital twin scene building module is used to model physical entities and build digital twin scenes based on the game engine; The environment perception module is used to perceive the vehicle's surrounding environment and estimate the vehicle's own state in real time; The simulation deduction module is used to perform parallel deduction of multiple decision-making plans at the same time in the digital twin scenario; The decision optimization module is used to evaluate the results of simulation deduction, select the optimal decision solution at the current moment, and reversely act on the physical entity; The digital twin scene building module includes a twin model library, a weather system, and a real-time rendering visualization part, among which: The twin model library constructs digital twins of real physical entities. Each twin is composed of a series of real sub-object modules, each responsible for independent attribute definition, corresponding to the complex physical structure of objects in the real world. The weather system is used to dynamically simulate the real effects of clouds, rain, snow, fog and lighting, as well as the influence of related physical quantity coefficients; The real-time rendering visualization part can instantiate and integrate the twin model based on the data of the object entity, and build a dynamic digital twin scene based on the game engine; The environmental perception module is used to receive sensor data from the physical world in real time, perceive the surrounding environment, obtain real-time road condition information and pedestrian information feedback, and accurately and effectively estimate its own status and update it into the digital twin scene; The simulation deduction module combines the information feedback obtained by the environmental perception module and its own state estimation in the digital twin scene, and performs parallel simulation deduction based on multiple possible decision-making schemes; the simulation deduction is carried out in different processes without affecting each other and ensuring real-time performance. The simulation deduction module performs simulation deduction, including the twin performing autonomous driving deduction obtaining the current visibility based on weather information parameters, thereby deducing the maximum safe distance within the visual range; obtaining the friction coefficient based on the current road condition information, thereby deducing the maximum braking distance under the current road condition; obtaining the pedestrian flow information in the scene based on the current pedestrian information, thereby deducing the next direction of travel of the pedestrians and the safe activity range of the current vehicle; after the simulation deduction is completed, the behavior and state under different decision-making schemes are predicted; The decision optimization module evaluates the simulation results based on the vehicle's own status and surrounding environment, and then selects the decision plan corresponding to the optimal result and performs vehicle control in the digital twin scene, thereby reversely controlling the autonomous driving vehicle in the physical world by controlling the vehicle twin in the digital twin scene.

2. The digital twin-based autonomous driving deduction system according to claim 1, characterized in that: The simulation module introduces an emergency sample library; the emergency sample library is built by simulating all possible emergency dangerous situations in advance in the emergency sample library; when encountering an emergency in real life, the emergency sample library is first searched. If a similar emergency can be retrieved, the corresponding decision-making plan is directly adopted to reduce the processing time of the emergency; Otherwise, simulation deduction is carried out again, and the emergency and corresponding decision are recorded in the emergency sample library.

3. The autonomous driving deduction method based on the autonomous driving deduction system according to claim 1 or 2, characterized in that: The specific steps are: Step S1: Build an autonomous driving digital twin scenario S1-1: Build static twin models for objects in the autonomous driving scene, including vehicles, sensors, buildings, and pedestrians. S1-2: Classify the constructed static twin models and define the specific attributes and data objects of different categories of twins, so as to establish a static mapping relationship between twins and physical entities; S1-3: Based on the game physics engine, the interaction process of physical entities in the physical world is simulated in the digital twin scene, thereby establishing a dynamic mapping relationship between the twin and the physical entity; S1-4: Build a weather system to simulate the real effects of clouds, rain, snow, fog and lighting; S1-5: Realize real-time rendering and visualization of autonomous driving digital twin scenes based on game rendering engines; Step S2: In the physical world, the vehicle's surroundings are perceived and positioned, and the results are fed back into the digital twin scene. S2-1: In the physical world, use sensors to perceive the environment around the vehicle and obtain information about surrounding vehicles, pedestrians, and other possible obstacles; S2-2: In the physical world, use sensors to perceive current weather conditions; S2-3: Based on road condition information and prior maps, the vehicle is positioned using SLAM technology. S2-4: Feedback the environmental perception and positioning results into the digital twin scene; Step S3: In the digital twin scenario, simulation deduction is performed through the simulation deduction module; S3-1: Based on the feedback from step S2-4 and the data from physical sensors, the traffic and weather conditions in the physical world are restored in the digital twin scene; S3-2: Quickly make several reasonable decisions based on the current autonomous driving scenario; S3-3: Parallel deduction of different autonomous driving decisions in step S3-2 on an event-by-event basis; S3-4: Feedback the deduction results to the decision optimization module; Step S4: Evaluate the simulation results and make corresponding decisions; S4-1: Analyze the results of step S3-4 and evaluate the quality of the autonomous driving decision based on the vehicle driving conditions in the simulation and whether there are potential accidents; S4-2: Select the best decision-making plan; S4-3: Control the autonomous driving vehicle in the physical world to make optimal decisions through reverse mapping of the vehicle twin and the physical entity.

4. The automatic driving deduction method according to claim 3, characterized in that: In step S3, in the digital twin scenario, the digital twin's autonomous driving deduction system makes decisions on the twin's behavior. The specific steps are: Step A1: Calculate the real-time driving parameters of the current twin based on real physical input; the driving parameters include maximum braking distance, visibility, visual range, maximum safe distance, and probability of emergency events; Step A2: Based on the aforementioned driving parameters as input to the deduction system, a decision is made based on the model derived from a large amount of data, and the execution strategy and feasibility at the next moment are analyzed; Step A3: Feedback the deduction results to the physical world to obtain the output results under the digital twin deduction system; the output results are the optimal operation results.

5. The automatic driving deduction method according to claim 3, characterized in that: In step S3, in the digital twin scenario, the specific steps for responding to emergencies in the simulation module are as follows: Step B1: Conduct simulations of emergencies in advance and build an emergency sample database; Step B2: When an emergency occurs during autonomous driving, the corresponding emergency is searched and matched from the emergency sample library; Step B3: If the corresponding emergency event is matched in the emergency event sample library, the corresponding decision plan is executed; Step B4: If the corresponding emergency event is not matched in the emergency event sample library, a simulation is performed in the simulation deduction module to obtain an optimized decision-making plan; Step B5: Iteratively update the emergency event sample library.

Images