Automatic driving safety early warning dynamic twin closed loop parallel simulation deduction method and system

Abstract

This invention discloses a method and system for dynamic twin closed-loop parallel simulation and extrapolation of safety warnings for autonomous driving. The method includes: acquiring multi-source data from real traffic scenarios and constructing a dynamic twin simulation environment that evolves synchronously with the real scenario; generating multiple future scenario evolution branches based on current scenario state and trajectory prediction information, and performing ultra-real-time parallel extrapolation to obtain scenario extrapolation results at multiple future time points; determining the consistency between the extrapolation results and real observation results, and dynamically pruning branches that do not meet the consistency conditions; automatically pruning branches with low threat levels based on threat level indicators; interacting with the onboard autonomous driving control system and the dynamic twin simulation environment to form a virtual-real closed-loop feedback; and outputting safety warnings, control suggestions, and safety decisions based on the pruned extrapolation results. This invention achieves ultra-real-time extrapolation, dynamic correction, and proactive warning of potential risks in complex traffic scenarios, improving the real-time performance, accuracy, and reliability of autonomous driving safety warnings.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving safety early warning technology, and in particular to a dynamic twin closed-loop parallel simulation deduction method and system for autonomous driving safety early warning. Background Technology

[0002] With the increasing application of unmanned logistics vehicles and low-speed unmanned vehicles in open road scenarios, the operating environment is shifting towards open environments characterized by frequent mixing of pedestrians and vehicles, complex dynamic interference, and severe obstruction. In these scenarios, vehicles not only need to be able to perceive the current environment but also need to be able to anticipate and proactively warn of potential future risks. Existing systems still face significant challenges in areas such as insufficient environmental perception, delayed safety warnings, and unstable decision-making and control strategies, and the system's security and reliability need to be improved.

[0003] Existing autonomous driving methods typically employ a sequential processing chain of "perception-prediction-planning-control," focusing on understanding the environment at the current moment and short-term trajectory prediction. However, in complex and dynamic traffic environments, relying solely on current perception results and short-term predictions is insufficient to fully analyze the evolution trend of the scenario over a future period, especially to proactively predict and provide early warnings of potential collision risks, dangerous approach behaviors, and abnormal behaviors under complex interactions among multiple stakeholders.

[0004] On the other hand, while existing research on digital twins and virtual-real fusion attempts to construct virtual mapping environments of real traffic scenarios, it still suffers from problems such as insufficient real-time performance, unstable synchronization between virtual and real states, and weak closed-loop feedback capabilities in highly dynamic autonomous driving scenarios. In particular, it is difficult to maintain consistency between the real and twin scenarios in high real-time scenarios. Simulation results are prone to deviating from the real scenario, virtual-real interaction feedback is insufficient, and it is difficult to support long-term dynamic extrapolation, thereby weakening the credibility and executability of safety warning results.

[0005] Furthermore, existing methods for extrapolating future scenarios often suffer from problems such as rapid expansion of branch size, excessive invalid branches, decreased extrapolation efficiency, and severe redundancy when conducting multi-path, multi-branch evolution analysis. Real-time parallel simulation safety early warning for vehicle dynamics faces challenges such as the difficulty in designing a multi-dimensional closed-loop evaluation system, real-time feedback issues between virtual and real-world closed loops, discrepancies between simulation and reality, and a lack of efficient safety early warning extrapolation mechanisms.

[0006] Therefore, there is an urgent need for a dynamic twin closed-loop parallel simulation method that can establish a real-time mapping and closed-loop feedback relationship between real traffic scenarios and twin scenarios, generate future multi-branch scenario evolution paths based on multi-target motion data, vehicle status, trajectory prediction results and behavioral intent information, and achieve dynamic correction of the simulation path and proactive early warning output through scenario consistency judgment, deviation threshold pruning and threat degree pruning, so as to realize proactive correction of future risks and safety early warning output. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for dynamic twin closed-loop parallel simulation and extrapolation of autonomous driving safety early warning, which realizes ultra-real-time extrapolation, dynamic correction and proactive early warning of potential risks in complex traffic scenarios, improves the real-time performance, accuracy and reliability of autonomous driving safety early warning, and overcomes the problems of insufficient future risk extrapolation capability of autonomous driving systems, weak closed-loop interaction capability between real scenarios and twin scenarios, redundancy of future extrapolation branches and lack of dynamic pruning mechanism in the existing technology.

[0008] To achieve the above objectives, the present invention provides the following solution: A method for dynamic twin closed-loop parallel simulation and derivation of autonomous driving safety warning includes the following steps: S1: Acquire multi-source data from real traffic scenarios, perform spatiotemporal alignment and fusion processing on the multi-source data, and construct a dynamic twin simulation environment that evolves synchronously with real traffic scenarios. S2 generates multiple future scenario evolution branches based on the current scene state, vehicle operation state, surrounding dynamic target state, trajectory prediction results, behavioral intention information, and traffic rules and road constraints. S3, In the dynamic twin simulation environment, perform ultra-real-time parallel simulation and deduction of the future scene evolution branch to obtain the scene deduction results at multiple future moments; S4, compare the consistency between the scenario deduction results and the actual observation results, and dynamically trim, regenerate or locally correct the deduction branches in the scenario deduction results that do not meet the preset consistency conditions. S5 performs risk assessments on each simulation branch based on threat level indicators and automatically prunes simulation branches with a comprehensive threat level below a preset threshold. S6 sends the driving commands, state-action decisions and execution results from the vehicle autonomous driving control system to the dynamic twin simulation environment, supervises and corrects the scenario simulation results, and feeds the correction results back to the vehicle autonomous driving control system, forming a virtual-real closed-loop feedback. S7 outputs security warnings, control recommendations, and security decisions based on scenario simulation results after consistency correction and threat level trimming.

[0009] Furthermore, in S1, the multi-source data includes: dynamic target state information, vehicle state information, predicted trajectory information, and road environment element information; The dynamic target state information includes the position, speed, acceleration, orientation, attitude, and historical trajectory of vehicles, pedestrians, and other traffic participants; the vehicle state information includes the vehicle's real-time position, heading angle, speed, longitudinal acceleration, lateral acceleration, steering status, and current control commands; and the road environment element information includes road boundaries, lane structure, traffic signs, static obstacles, traffic area constraints, and map prior information.

[0010] Furthermore, in step S1, the multi-source data undergoes spatiotemporal alignment and fusion processing to construct a dynamic twin simulation environment that evolves synchronously with the real traffic scenario, specifically including: Data from different sensors, different sampling frequencies, and different coordinate systems are mapped to a unified time base and a unified spatial reference system; By combining target association and environment registration, the dynamic target state, road structure information and prediction information in the same scene are uniformly expressed to obtain the fusion result; The fusion results are injected into the simulation space built on CARLA to construct a dynamic twin simulation environment that includes road topology, static scene elements, dynamic target states and their evolution information.

[0011] Furthermore, S2, based on the current scene state, the vehicle's operating state, the state of surrounding dynamic targets, trajectory prediction results, behavioral intent information, and traffic rules and road constraints, generates multiple future scene evolution branches, specifically including: Based on the current position, speed, orientation, and historical behavior characteristics of the vehicle and surrounding traffic participants, and combined with trajectory prediction results, candidate motion trends for several future moments are obtained. Based on the behavioral intent recognition results, models are built for vehicle steering, lane changing, acceleration, deceleration, and avoidance, as well as pedestrian crossing, stopping, and traversing behaviors; Traffic rules, road passability area constraints, and safety distance constraints are introduced to screen out unreasonable or rule-violating candidate evolution paths; Generate multiple legal future scenario evolution branches that satisfy the scenario constraints.

[0012] Furthermore, in step S3, the future scene evolution branch is simulated and extrapolated in real-time within a dynamic twin simulation environment to obtain scene extrapolation results at multiple future moments, specifically including: Each future scenario evolution branch is loaded into a dynamic twin simulation environment; Based on the target state evolution rules corresponding to each branch, the vehicle control state, and environmental constraints, the scene changes within one or more future time windows are updated hour by hour. The relative positional relationships, speed relationships, traffic conflict relationships, and spatial proximity between the vehicle and surrounding traffic participants at each future moment are obtained, forming a continuous scenario projection result for multiple future moments.

[0013] Further, in step S4, the consistency between the scenario deduction results and the actual observation results is compared and determined. For scenario deduction results that do not meet the consistency conditions, dynamic pruning, regeneration, or local correction is performed on the deduction branches. Specifically, this includes: The system receives the latest real observation information in real time and compares this real observation information with the predicted state of each inference branch at the corresponding time. The consistency comparison includes target position deviation, velocity deviation, direction deviation, interaction relationship deviation, and scene structure consistency. When the deviation of a certain inference branch exceeds a preset threshold, it is determined that the inference branch is not consistent with the real scene. Trimming or deleting inference branches that lack consistency, or regenerating or locally correcting the inference branches by incorporating the latest real observation information.

[0014] Furthermore, S5 involves performing a risk assessment on each simulation branch based on a threat level index, and automatically pruning simulation branches with a comprehensive threat level below a preset threshold, specifically including: The spatiotemporal relationship between the vehicle and surrounding traffic participants in each simulation branch is analyzed, and at least one risk measure information among the minimum safe distance, time to collision, collision probability, and comprehensive threat level is calculated respectively. Based on the calculated risk measurement information, a comprehensive threat index is formed for each simulation branch; The inference branches with an overall threat level index higher than a preset threshold are retained, while the inference branches with an overall threat level index lower than a preset threshold are automatically pruned.

[0015] Furthermore, in S6, the virtual-real closed-loop feedback is used to correct perception errors, prediction errors, decision-making biases, and inconsistencies between virtual and real states. When the on-board autonomous driving control system executes a control action, it sends the control action information and the corresponding execution result back to the dynamic twin simulation environment, so that the vehicle state in the virtual environment is synchronized with the real execution result. After completing future scenario simulation and risk assessment, the dynamic twin simulation environment feeds back the corrected risk results, control suggestions, and state correction information to the vehicle autonomous driving control system to assist the system in adjusting subsequent perception, prediction, or control strategies.

[0016] Furthermore, in S7, the safety warning includes at least one of the following: collision risk warning, abnormal approach warning, potential crossing warning, and sudden target intrusion warning; Control recommendations include at least one of the following: reduce speed, maintain current lane, continue to observe, apply active braking, and avoid collision. Safety decisions are generated based on the type of risk, the level of risk, and the time when the risk occurs.

[0017] This invention also provides a dynamic twin closed-loop parallel simulation and deduction system for autonomous driving safety warning, applied to the above-described method for dynamic twin closed-loop parallel simulation and deduction of autonomous driving safety warning, comprising: The data acquisition module is used to acquire multi-source data from real traffic scenarios; The twin building module is used to perform spatiotemporal alignment and fusion processing on multi-source data to build a dynamic twin simulation environment that evolves synchronously with real traffic scenarios; The branch generation module is used to generate multiple future scenario evolution branches based on the current scene state, the vehicle's operating state, the state of surrounding dynamic targets, trajectory prediction results, behavioral intent information, and traffic rules and road constraints. The parallel simulation module is used to perform ultra-real-time parallel simulation and extrapolation of the future scene evolution branches in a dynamic twin simulation environment to obtain the scene extrapolation results at multiple future moments. The consistency determination module is used to compare and determine the consistency between the scenario deduction results and the actual observation results, and to dynamically trim, regenerate or locally correct the deduction branches in the scenario deduction results that do not meet the preset consistency conditions. The threat assessment module is used to assess the risk of each simulation branch based on the threat level index, and to automatically prune simulation branches whose overall threat level is lower than a preset threshold. The closed-loop feedback module is used to send driving commands, state-action decisions and execution results from the vehicle autonomous driving control system to the dynamic twin simulation environment, supervise and correct the scenario simulation results, and feed back the correction results to the vehicle autonomous driving control system to form a virtual-real closed-loop feedback. The early warning output module is used to output security warnings, control suggestions, and security decisions based on the scenario simulation results after consistency correction and threat level trimming.

[0018] According to specific embodiments of the present invention, the technical effects of the dynamic twin closed-loop parallel simulation deduction method and system for autonomous driving safety early warning provided by the present invention are mainly reflected in: (1) Ultra-real-time proactive early warning: By constructing a dynamic twin simulation environment and generating multiple future scenario evolution branches, ultra-real-time parallel simulation and deduction can be carried out in the dynamic twin simulation environment. Risk evolution information can be obtained before the real risk occurs, realizing the transformation from passive response to proactive early warning and solving the problem of lagging early warning in existing technologies.

[0019] (2) Virtual-Real Closed-Loop Correction: By feeding back the driving commands, decisions and execution results of the vehicle autonomous driving control system to the twin environment, and feeding back the risk information corrected by simulation to the vehicle autonomous driving control system, a two-way closed-loop feedback mechanism is formed to continuously correct perception errors, prediction errors, decision deviations and virtual-real inconsistencies, and avoid simulation results deviating from the real scene.

[0020] (3) Dual dynamic pruning: Through consistency judgment, the inference branches that deviate from the real scenario are pruned or regenerated. Through threat assessment, low-risk or redundant branches are automatically pruned. While suppressing invalid branches, high-risk scenarios are focused on, effectively controlling the branch size and improving inference efficiency.

[0021] (4) Continuous long-term time series simulation: Based on the multi-branch generation and time-by-time update mechanism, the relative positional relationship, speed relationship, traffic conflict relationship and spatial proximity of the vehicle and surrounding traffic participants at each future time are obtained, forming continuous scene simulation results, which makes up for the shortcomings of the existing technology in long-term time series simulation capability.

[0022] In summary, this invention achieves real-time simulation, dynamic correction, and proactive early warning of future risks in complex traffic scenarios through dynamic twin construction, multi-branch generation, ultra-real-time parallel extrapolation, consistency pruning, threat degree pruning, and virtual-real closed-loop feedback, thereby improving the real-time performance, accuracy, and reliability of safety warnings in autonomous driving systems. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the dynamic twin closed-loop parallel simulation deduction method for autonomous driving safety early warning in an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall framework modules provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the construction of a real-time sensing multi-target dynamic twin simulation environment provided in an embodiment of the present invention; Figure 4 The flowchart illustrates the multi-target motion data-driven ultra-real-time parallel simulation method provided in this embodiment of the invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] This invention provides a dynamic twin closed-loop parallel simulation method and system for safety early warning in autonomous driving. This method is primarily designed for complex traffic scenarios involving mixed human and vehicle traffic and frequent dynamic interference, such as in industrial parks, factory roads, and open roads. It is particularly suitable for proactive safety early warning in unmanned logistics vehicles, low-speed autonomous vehicles, and other autonomous driving systems. The method revolves around a technical chain of "real-world scene perception—dynamic twin construction—future multi-branch simulation—dynamic pruning—closed-loop feedback—early warning output." By constructing a dynamic twin simulation environment in virtual space that evolves synchronously with the real-world scene, it performs ultra-real-time simulation, dynamic correction, and proactive early warning for future risks. Its technical route is as follows: Figure 1 As shown.

[0028] like Figure 1 As shown, the autonomous driving safety warning dynamic twin closed-loop parallel simulation deduction method provided by the present invention includes the following steps: S1: Acquire multi-source data from real traffic scenarios, perform spatiotemporal alignment and fusion processing on the multi-source data, and construct a dynamic twin simulation environment that evolves synchronously with real traffic scenarios. S2 generates multiple future scenario evolution branches based on the current scene state, vehicle operation state, surrounding dynamic target state, trajectory prediction results, behavioral intention information, and traffic rules and road constraints. S3, Perform ultra-real-time parallel simulation and deduction of the future scene evolution branch in a dynamic twin simulation environment to obtain the scene deduction results at multiple future moments; S4, compare the consistency between the scenario deduction results and the actual observation results, and dynamically trim, regenerate or locally correct the deduction branches in the scenario deduction results that do not meet the preset consistency conditions. S5 performs risk assessments on each simulation branch based on threat level indicators and automatically prunes simulation branches whose overall threat level is below a set threshold. S6 sends the driving commands, state-action decisions and execution results from the vehicle autonomous driving control system to the dynamic twin simulation environment, supervises and corrects the scenario simulation results, and feeds the correction results back to the vehicle autonomous driving control system, forming a virtual-real closed-loop feedback. S7 outputs security warnings, control suggestions, and security decisions based on scenario simulation results after consistency correction and threat level trimming.

[0029] The vehicle-mounted autonomous driving control system described in this invention refers to a real operating system installed in an autonomous vehicle, including a perception module, a positioning module, a prediction module, a decision planning module, a control execution module, and the vehicle itself. It is used to generate driving commands, state-action decisions, and actual vehicle execution results, and to interact with the dynamic twin simulation environment to form a virtual-real closed-loop feedback.

[0030] like Figure 2 As shown, the overall framework of this invention includes multi-view dynamic twin modeling and BEV view construction, dynamic trajectory prediction, vehicle decision-making, parallel twin simulation, multi-view image generation, and safety warning output. First, a dynamic twin simulation environment corresponding to the real traffic scenario is constructed based on multi-source data such as vehicles, pedestrians, road structure, and vehicle status, forming a BEV view representation. Then, combining the dynamic trajectory prediction results and vehicle decision information, future scenario evolution branches are generated, and ultra-real-time parallel simulation is performed in the dynamic twin simulation environment. Subsequently, multi-view scene images are generated based on the simulation results, and trajectory, preset text, and scene semantic information are input into the visual language model. Finally, future trajectory points, safety warning results, and control commands are output. This framework embodies a closed-loop processing process from real-scene perception, dynamic twin construction, future scenario deduction to safety warning output.

[0031] In step S1, firstly, multi-source data from real traffic scenarios is acquired, such as... Figure 3 As shown. The multi-source data includes dynamic target state information, vehicle state information, predicted trajectory information, and road environment element information. Specifically, the dynamic target state information includes the position, speed, acceleration, orientation, attitude, and historical trajectory of vehicles, pedestrians, and other traffic participants; the vehicle state information includes the vehicle's real-time position, heading angle, speed, longitudinal acceleration, lateral acceleration, steering status, and current control commands; the predicted trajectory information includes the target's position sequence, speed change trend, and interactive evolution trend at several future moments; and the road environment element information includes road boundaries, lane structure, traffic signs, static obstacles, traffic area constraints, and prior map information.

[0032] After acquiring the aforementioned multi-source data, unified spatiotemporal alignment and fusion are performed on the multi-source data. Specifically, data from different sensors, different sampling frequencies, and different coordinate systems can be mapped to a unified time reference and a unified spatial reference system. Through target association and environment registration, the dynamic target state, road structure information, and prediction information in the same scene are uniformly expressed. This process eliminates temporal deviations, spatial deviations, and representational differences between multi-source data, improving the accuracy and consistency of subsequent twin construction.

[0033] After completing spatiotemporal alignment and fusion, the processing results are injected into the simulation space built based on CARLA to construct a dynamic twin simulation environment that evolves synchronously with the real traffic scene. Specifically, based on road structure information or map prior information in the real traffic scene, a virtual road scene corresponding to the real road is constructed in CARLA, and vehicle models, surrounding vehicle models, pedestrian models, static obstacle models, and traffic sign models are set in this virtual road scene. Subsequently, the dynamic target state information, vehicle state information, predicted trajectory information, and road environment element information in the real traffic scene are transformed into the simulation coordinate system, and the position, speed, orientation, and motion state of the corresponding objects in the CARLA simulation environment are updated in real time, so that the CARLA simulation environment evolves synchronously with the real traffic scene. The processing procedure used in this invention to construct the dynamic twin simulation environment based on CARLA includes: Multi-source data spatiotemporal alignment processing: used to achieve unified time reference synchronization, multi-coordinate system spatial mapping and transformation, and sensor data timestamp alignment, transforming multi-source data in real traffic scenarios to a unified spatiotemporal reference.

[0034] Simulation environment coordinate mapping processing: This is used to transform data from the real traffic scene coordinate system, vehicle coordinate system, and sensor coordinate system to the CARLA simulation coordinate system, while maintaining the spatial correspondence between the vehicle, surrounding traffic participants, and road environment elements.

[0035] Multi-source data fusion processing: used to achieve target association, environment registration, and unified expression of multi-source heterogeneous data, forming target state and scene state that can be written into the CARLA simulation environment.

[0036] Dynamic target state estimation and tracking processing: used to estimate and update the position, velocity, acceleration, orientation, attitude, and historical trajectory of vehicles, pedestrians, and other traffic participants.

[0037] CARLA Scene Digital Modeling Processing: Used to build road topology, lane structure, traffic signs, static obstacles, vehicle models, surrounding vehicle models, and pedestrian models in CARLA.

[0038] Virtual-real synchronization and dynamic update processing: used to refresh the vehicle state, surrounding target state and road environment state in the CARLA environment according to real-time observation results, so that the dynamic twin simulation environment based on CARLA keeps the evolution synchronized with the real traffic scene.

[0039] Through the above processing, it is possible to achieve unified spatiotemporal representation of multi-sensor data, target association, environmental registration, CARLA coordinate mapping, and real-time digital mapping, forming a dynamic twin simulation environment that evolves synchronously with real traffic scenarios.

[0040] The dynamic twin simulation environment is not a static map copy, but an updatable dynamic simulation environment that includes road topology, static scene elements, dynamic target states, and their evolution information. This environment can be continuously updated based on real-time observations, thus achieving real-time mapping and synchronous representation of real traffic scenarios. This dynamic twin simulation environment provides a unified state base and simulation platform for subsequent multi-branch deductions.

[0041] In step S2, after obtaining the dynamic twin simulation environment at the current moment, multiple future scenario evolution branches are generated based on the current scene state, vehicle state, surrounding dynamic target state, trajectory prediction results, behavioral intent information, and rule constraints. These future scenario evolution branches are candidate paths generated for different evolutionary modes of the future scenario, with different branches corresponding to different interaction patterns, movement trends, and potential risk events of traffic participants.

[0042] Specifically, firstly, candidate motion trends for several future moments can be obtained by combining the current position, speed, orientation, and historical behavior characteristics of the vehicle and surrounding traffic participants with trajectory prediction results; secondly, based on the behavioral intent recognition results, possible vehicle behaviors such as turning, changing lanes, accelerating, decelerating, and avoiding, as well as possible pedestrian behaviors such as crossing, stopping, and traversing, are modeled; then, traffic rules, road passable area constraints, and safe distance constraints are introduced to filter out unreasonable or rule-violating candidate evolution paths; finally, multiple future scenario evolution branches that meet the scenario constraints are formed.

[0043] By employing the above methods, we can avoid the insufficient uncertainty caused by relying solely on a single predicted trajectory, and simultaneously consider multiple possible evolutionary outcomes during risk simulation. For example... Figure 4As shown, the ultra-real-time parallel simulation process of this invention is based on the current state of the real scene, and combines trajectory prediction models, target movement intentions, prior knowledge bases, and scene indexes to generate multiple future scene evolution branches. The system iterates through each scene deduction branch and forms a multi-branch simulation scene in a dynamic twin simulation environment through traffic simulation entity cloning and replication, performing ultra-real-time deduction of the most likely movement routes of each target. As the physical time of the real scene progresses, the system judges the consistency between the deduction results and the actual observation results; scene deduction branches that do not meet the consistency conditions are dynamically pruned, while those that meet the consistency conditions are retained. Finally, based on the retained deduction results, dynamic prediction of safety elements is performed, providing a basis for subsequent safety warnings, control suggestions, and safety decision outputs.

[0044] In this embodiment, multiple future scenario evolution branches generated by S2 are subjected to ultra-real-time parallel simulation in a dynamic twin simulation environment to obtain scenario projection results for multiple future time points. The ultra-real-time simulation refers to the parallel simulation of short- to medium-short-term scenario changes within or faster than the actual time scale of the vehicle autonomous driving control system, thereby enabling the system to obtain risk evolution information in advance before the actual risk occurs.

[0045] Specifically, each future scenario evolution branch is loaded into a dynamic twin simulation environment, and the scenario changes within one or more future time windows are updated hourly based on the target state evolution rules, vehicle control state, and environmental constraints corresponding to each branch. During the update process, the relative positional relationships, speed relationships, traffic conflict relationships, and spatial proximity between the vehicle and surrounding traffic participants at each moment can be obtained simultaneously. This forms the continuous scenario projection results for each branch at different future moments.

[0046] Through the above deduction process, we can not only obtain "where the target might go," but also "how the future scenario will evolve," "which interactions will create risks," and "when different risks will appear." Therefore, the output of step S3 is not a single prediction point, but a set of scenario deduction results containing multiple future moments, multiple candidate paths, and corresponding interactions. This result will be further used for consistency determination and risk assessment.

[0047] In this embodiment, the scenario simulation results obtained in S3 are compared with the actual observation results for consistency judgment. Based on the judgment results, simulation branches that do not meet the preset consistency conditions are dynamically pruned or regenerated. The purpose is to prevent the simulation process from gradually deviating from the real scene over time, thereby improving the credibility and usability of parallel simulation.

[0048] Specifically, during the continuous operation of the onboard autonomous driving control system, new real-world observation information is constantly received, and this information is compared with the predicted states of each simulation branch at the corresponding time. The comparison can include target position deviation, speed deviation, direction deviation, interaction relationship deviation, and scene structure consistency. When the deviation of a simulation branch exceeds a preset threshold, it can be determined that the simulation branch lacks consistency with the real scene. Branches with insufficient consistency can be directly pruned or deleted; branches that still have reference value but have large local state deviations can be regenerated or locally corrected based on the latest real-world observation results.

[0049] This consistency determination mechanism continuously suppresses invalid branches that deviate significantly from the real scenario, preventing the inference tree from expanding indefinitely, while ensuring that the retained inference branches always maintain a high degree of synchronization with the real traffic scenario. This step is a key link in achieving "virtual-real closed-loop feedback" and "dynamic correction".

[0050] In this embodiment, after completing the consistency determination, a risk assessment is further performed on each simulation branch based on the threat level index, and simulation branches with lower overall threat levels are automatically pruned. The threat level index can be used to characterize the potential impact of different simulation branches on the vehicle's operational safety.

[0051] Specifically, the spatiotemporal relationship between the vehicle and surrounding traffic participants in each simulation branch can be analyzed to calculate risk metrics such as minimum safe distance, time-to-collision (TTC), collision probability, and overall threat level. The minimum safe distance characterizes the closest distance between the vehicle and the target traffic participant within the simulation time window; the time-to-collision (TTC) characterizes the estimated time required for a collision to occur under the current relative motion trend; the collision probability characterizes the likelihood of a potential collision event occurring in the simulation branch; and the overall threat level characterizes the overall impact of the simulation branch on the vehicle's operational safety. The above risk metrics can be calculated as follows, where equation (1) represents the minimum safe distance, equation (2) represents the time-to-collision (TTC) value, equation (3) represents the collision probability, and equation (4) represents the overall threat level. (1) (2) (3) (4) in, Indicates the target traffic participant number. Indicates the branch number of the deduction. Indicates the time window for simulation. Indicates the vehicle's position. Indicates the location of the target traffic participant. This indicates the minimum safe distance between the vehicle and the target traffic participant. This indicates the relative approach speed between the vehicle and the target traffic participant. This indicates a pre-defined minimum positive number. This represents the parameter indicating the uncertainty in predicting the target state. Indicates the probability of collision. Indicates the first The overall threat level of each branch of the simulation. Indicates the first The maximum collision probability in each deduction branch Indicates the distance to the risk item. Indicates time risk item, Indicates target traffic participants The corresponding time-to-collision event occurs when there is no approach trend between the vehicle and the target traffic participant. Set to the preset maximum value. Indicates avoidable risks. to This indicates the preset weight.

[0052] The distance risk item can be determined based on the minimum safe distance and the preset safe distance threshold. The time risk item can be determined based on the time to collision and the preset safe time threshold. The avoidable risk item can be determined based on the time required for the vehicle to complete deceleration, braking or avoidance.

[0053] in, Can be derived from the first Each target traffic participant in each of the simulation branches corresponds to Determined by taking the maximum value or a weighted value. It can be assigned to each target traffic participant. Determined in accordance with the preset safe distance threshold, It can be assigned to each target traffic participant. Determined in accordance with the preset safety time threshold, This can be determined by the remaining time before the risk occurs and the time required for the vehicle to complete the avoidance maneuver. The larger the value, the higher the security risk of the future scenario corresponding to that branch of the deduction.

[0054] In this embodiment, the threat level index corresponding to each simulation branch specifically includes the comprehensive threat level value. Risk level and main risk types. Among them, the overall threat level... Used to quantify the overall risk level of this simulation branch; the risk level can be determined according to... The comparison results with preset thresholds are categorized into high risk, medium risk, and low risk. The main risk type can be determined based on the largest contributing factor among collision probability risk, distance risk, time risk, and avoidable risk. For example, a first preset threshold is set to be greater than a second preset threshold; the specific value can be designed according to actual needs. When the value is greater than or equal to the first preset threshold, the inference branch is marked as a high-risk branch; when When the risk level is less than the first preset threshold and greater than or equal to the second preset threshold, the inference branch is marked as a medium-risk branch; when... If the risk level is less than the second preset threshold, the simulation branch is marked as a low-risk branch. Therefore, the system can use the comprehensive threat level, risk grade, and main risk type as threat indicators for this simulation branch, which can be used for subsequent branch retention, automatic pruning, and security warning output.

[0055] For simulation branches with a high overall threat level, they will be retained and included in the subsequent early warning and decision-making process; for simulation branches with a low overall threat level, limited reference value, or high overlap with other high-risk branches, they will be automatically pruned according to preset pruning rules to reduce simulation calculation redundancy and system resource consumption.

[0056] By combining consistency pruning and threat pruning, the size and quality of the branch set can be constrained simultaneously from two dimensions: "realism" and "risk." On the one hand, invalid branches that deviate too much from the real scenario are eliminated; on the other hand, key branches most likely to cause security risks are retained. This improves the overall simulation efficiency while ensuring the effectiveness of early warning.

[0057] In this embodiment, driving commands, state-action decisions, and execution results from the onboard autonomous driving control system are sent to a virtual twin environment. The simulation results are monitored and corrected, and the correction results are fed back to the onboard autonomous driving control system, forming a virtual-real closed-loop feedback. This closed-loop feedback mechanism is used to dynamically correct problems such as perception errors, prediction errors, decision biases, and inconsistencies between virtual and real states.

[0058] Specifically, after the onboard autonomous driving control system executes a certain control action, it can transmit the action information and execution result back to the dynamic twin simulation environment, so that the vehicle's state in the virtual environment can remain synchronized with the actual execution result. At the same time, after completing future scenario simulation and risk assessment, the dynamic twin simulation environment can also feed back the corrected risk results, control suggestions, and state correction information to the onboard autonomous driving control system to assist the system in adjusting subsequent perception, prediction, or control strategies.

[0059] Through this mechanism, a two-way update relationship is established between the onboard autonomous driving control system and the dynamic twin simulation environment: the onboard autonomous driving control system provides continuous observation and execution results to the dynamic twin simulation environment, while the dynamic twin simulation environment provides the onboard autonomous driving control system with future-oriented risk projection and correction information. This step realizes the transformation from "one-way simulation" to "virtual-real collaborative closed loop," thereby enhancing the credibility and executability of safety warning results.

[0060] In this embodiment, based on the inference results after consistency comparison judgment, threat level assessment, and closed-loop feedback correction, safety warning results, control suggestions, and safety decision results are output. The safety warning results may include collision risk warning, abnormal approach warning, potential crossing warning, and sudden target intrusion warning; the control suggestions may include one or more of the following: deceleration, maintaining the current lane, continuous observation, active braking, and avoidance.

[0061] Specifically, when the threat level of a certain retained branch reaches a preset risk threshold, the system can determine that there is a high safety risk in the corresponding future scenario, and output corresponding warning results based on the risk type, risk level, and risk occurrence time. Simultaneously, based on the relative motion relationship between the risk object and the vehicle, it provides corresponding control suggestions or safety decisions. For example, if the simulation results indicate that the target vehicle ahead may suddenly decelerate and potentially cause a rear-end collision, a deceleration warning and a suggestion to maintain a safe distance can be output; if the simulation results indicate that a pedestrian is crossing the road and may enter the vehicle's passage area, a braking warning or a suggestion to continue observation can be output.

[0062] Through the above methods, the embodiments of the present invention can realize the early prediction and proactive warning of potential risks in complex and dynamic traffic scenarios, avoiding the warning lag problem caused by relying only on the current state in the traditional serial "perception-prediction-planning-control" link, thereby improving the safety, real-time performance and reliability of the autonomous driving system in open environments.

[0063] This invention also provides a dynamic twin closed-loop parallel simulation and deduction system for autonomous driving safety warning, applied to the above-described method for dynamic twin closed-loop parallel simulation and deduction of autonomous driving safety warning, comprising: The data acquisition module is used to acquire multi-source data from real traffic scenarios; The twin building module is used to perform spatiotemporal alignment and fusion processing on multi-source data to build a dynamic twin simulation environment that evolves synchronously with real traffic scenarios; The branch generation module is used to generate multiple future scenario evolution branches based on the current scene state, the vehicle's operating state, the state of surrounding dynamic targets, trajectory prediction results, behavioral intent information, and traffic rules and road constraints. The parallel simulation module is used to perform ultra-real-time parallel simulation and extrapolation of the future scene evolution branches in a dynamic twin simulation environment to obtain the scene extrapolation results at multiple future moments. The consistency determination module is used to compare and determine the consistency between the scenario deduction results and the actual observation results, and to dynamically trim, regenerate or locally correct the deduction branches in the scenario deduction results that do not meet the preset consistency conditions. The threat assessment module is used to assess the risk of each simulation branch based on the threat level index, and to automatically prune simulation branches whose overall threat level is lower than a set threshold. The closed-loop feedback module is used to send driving commands, state-action decisions and execution results from the vehicle autonomous driving control system to the dynamic twin simulation environment, supervise and correct the scenario simulation results, and feed back the correction results to the vehicle autonomous driving control system to form a virtual-real closed-loop feedback. The early warning output module is used to output security warnings, control suggestions, and security decisions based on the scenario simulation results after consistency correction and threat level trimming.

[0064] The present invention also discloses an electronic device comprising one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more application programs are configured to perform the above-described dynamic twin closed-loop parallel simulation deduction method for autonomous driving safety warning.

[0065] The present invention also discloses a non-transitory computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the above-described method for dynamic twin closed-loop parallel simulation deduction of autonomous driving safety warning.

[0066] Those skilled in the art should understand that the technical solutions described in this invention are not limited to the specific embodiments described above.

[0067] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An automatic driving safety early warning dynamic twin closed loop parallel simulation deduction method, characterized in that, Includes the following steps: S1: Acquire multi-source data from real traffic scenarios, perform spatiotemporal alignment and fusion processing on the multi-source data, and construct a dynamic twin simulation environment that evolves synchronously with real traffic scenarios. S2 generates multiple future scenario evolution branches based on the current scene state, vehicle operation state, surrounding dynamic target state, trajectory prediction results, behavioral intention information, and traffic rules and road constraints. S3, In the dynamic twin simulation environment, perform ultra-real-time parallel simulation and deduction of the future scene evolution branch to obtain the scene deduction results at multiple future moments; S4, compare the consistency between the scenario deduction results and the actual observation results, and dynamically trim, regenerate or locally correct the deduction branches in the scenario deduction results that do not meet the preset consistency conditions. S5 performs risk assessments on each simulation branch based on threat level indicators and automatically prunes simulation branches with a comprehensive threat level below a preset threshold. S6 sends the driving commands, state-action decisions and execution results from the vehicle autonomous driving control system to the dynamic twin simulation environment, supervises and corrects the scenario simulation results, and feeds the correction results back to the vehicle autonomous driving control system, forming a virtual-real closed-loop feedback. S7 outputs security warnings, control recommendations, and security decisions based on scenario simulation results after consistency correction and threat level trimming. 2.The automatic driving safety pre-warning dynamic twin closed-loop parallel simulation deduction method according to claim 1, characterized in that, In S1, the multi-source data includes: dynamic target status information, vehicle status information, predicted trajectory information, and road environment element information; The dynamic target state information includes the position, speed, acceleration, orientation, attitude, and historical trajectory of vehicles, pedestrians, and other traffic participants; the vehicle state information includes the vehicle's real-time position, heading angle, speed, longitudinal acceleration, lateral acceleration, steering status, and current control commands; and the road environment element information includes road boundaries, lane structure, traffic signs, static obstacles, traffic area constraints, and map prior information.

3. The method for dynamic twin closed-loop parallel simulation and deduction of autonomous driving safety early warning according to claim 1, characterized in that, In step S1, the multi-source data undergoes spatiotemporal alignment and fusion processing to construct a dynamic twin simulation environment that evolves synchronously with the real traffic scenario, specifically including: Data from different sensors, different sampling frequencies, and different coordinate systems are mapped to a unified time base and a unified spatial reference system; By combining target association and environment registration, the dynamic target state, road structure information and prediction information in the same scene are uniformly expressed to obtain the fusion result; The fusion results are injected into the simulation space built on CARLA to construct a dynamic twin simulation environment that includes road topology, static scene elements, dynamic target states and their evolution information.

4. The method for dynamic twin closed-loop parallel simulation and deduction of autonomous driving safety early warning according to claim 1, characterized in that, S2, based on the current scene state, vehicle operating state, surrounding dynamic target state, trajectory prediction results, behavioral intent information, and traffic rules and road constraints, generates multiple future scene evolution branches, specifically including: Based on the current position, speed, orientation, and historical behavior characteristics of the vehicle and surrounding traffic participants, and combined with trajectory prediction results, candidate motion trends for several future moments are obtained. Based on the behavioral intent recognition results, models are built for vehicle steering, lane changing, acceleration, deceleration, and avoidance, as well as pedestrian crossing, stopping, and traversing behaviors; Traffic rules, road passability area constraints, and safety distance constraints are introduced to screen out unreasonable or rule-violating candidate evolution paths; Generate multiple legal future scenario evolution branches that satisfy the scenario constraints.

5. The method for dynamic twin closed-loop parallel simulation and deduction of autonomous driving safety early warning according to claim 1, characterized in that, S3 involves performing ultra-real-time parallel simulation and deduction of the future scene evolution branch in a dynamic twin simulation environment to obtain scene deduction results at multiple future moments, specifically including: Each future scenario evolution branch is loaded into a dynamic twin simulation environment; Based on the target state evolution rules corresponding to each branch, the vehicle control state, and environmental constraints, the scene changes within one or more future time windows are updated hour by hour. The relative positional relationships, speed relationships, traffic conflict relationships, and spatial proximity between the vehicle and surrounding traffic participants at each future moment are obtained, forming a continuous scenario projection result for multiple future moments.

6. The method for dynamic twin closed-loop parallel simulation and deduction of autonomous driving safety early warning according to claim 1, characterized in that, Step S4 involves comparing the scenario deduction results with the actual observation results for consistency determination, and dynamically pruning, regenerating, or locally correcting the deduction branches in the scenario deduction results that do not meet the consistency conditions. Specifically, this includes: The system receives the latest real observation information in real time and compares the real observation information with the predicted state of each inference branch at the corresponding time. The consistency comparison includes target position deviation, velocity deviation, direction deviation, interaction relationship deviation, and scene structure consistency. When the deviation of a certain inference branch exceeds a preset threshold, it is determined that the inference branch is not consistent with the real scene. Trimming or deleting inference branches that lack consistency, or regenerating or locally correcting the inference branches by incorporating the latest real observation information.

7. The method for dynamic twin closed-loop parallel simulation and deduction of autonomous driving safety early warning according to claim 1, characterized in that, S5 involves assessing the risk of each simulation branch based on a threat level index, and automatically pruning simulation branches with a comprehensive threat level below a preset threshold. Specifically, this includes: The spatiotemporal relationship between the vehicle and surrounding traffic participants in each simulation branch is analyzed, and at least one risk measure information among the minimum safe distance, time to collision, collision probability, and comprehensive threat level is calculated respectively. Based on the calculated risk measurement information, a comprehensive threat index is formed for each simulation branch; The inference branches with an overall threat level index higher than a preset threshold are retained, while the inference branches with an overall threat level index lower than a preset threshold are automatically pruned.

8. The method for dynamic twin closed-loop parallel simulation and deduction of autonomous driving safety early warning according to claim 1, characterized in that, In S6, the virtual-real closed-loop feedback is used to correct perception errors, prediction errors, decision-making biases, and inconsistencies between virtual and real states. When the on-board autonomous driving control system executes a control action, it sends the control action information and the corresponding execution result back to the dynamic twin simulation environment, so that the vehicle state in the virtual environment is synchronized with the real execution result. After completing future scenario simulation and risk assessment, the dynamic twin simulation environment feeds back the corrected risk results, control suggestions, and state correction information to the vehicle autonomous driving control system to assist the system in adjusting subsequent perception, prediction, or control strategies.

9. The method for dynamic twin closed-loop parallel simulation and deduction of autonomous driving safety early warning according to claim 1, characterized in that, In S7, the safety warning includes at least one of the following: collision risk warning, abnormal approach warning, potential crossing warning, and sudden target intrusion warning; Control recommendations include at least one of the following: reduce speed, maintain current lane, continue to observe, apply active braking, and avoid collision. Safety decisions are generated based on the type of risk, the level of risk, and the time when the risk occurs.

10. A dynamic twin closed-loop parallel simulation and deduction system for autonomous driving safety early warning, applied to the autonomous driving safety early warning dynamic twin closed-loop parallel simulation and deduction method according to any one of claims 1-9, characterized in that, include: The data acquisition module is used to acquire multi-source data from real traffic scenarios; The twin building module is used to perform spatiotemporal alignment and fusion processing on multi-source data to build a dynamic twin simulation environment that evolves synchronously with real traffic scenarios; The branch generation module is used to generate multiple future scenario evolution branches based on the current scene state, the vehicle's operating state, the state of surrounding dynamic targets, trajectory prediction results, behavioral intent information, and traffic rules and road constraints. The parallel simulation module is used to perform ultra-real-time parallel simulation and extrapolation of the future scene evolution branches in a dynamic twin simulation environment to obtain the scene extrapolation results at multiple future moments. The consistency determination module is used to compare and determine the consistency between the scenario deduction results and the actual observation results, and to dynamically trim, regenerate or locally correct the deduction branches in the scenario deduction results that do not meet the preset consistency conditions. The threat assessment module is used to assess the risk of each simulation branch based on the threat level index, and to automatically prune simulation branches whose overall threat level is lower than a preset threshold. The closed-loop feedback module is used to send driving commands, state-action decisions and execution results from the vehicle autonomous driving control system to the dynamic twin simulation environment, supervise and correct the scenario simulation results, and feed back the correction results to the vehicle autonomous driving control system to form a virtual-real closed-loop feedback. The early warning output module is used to output security warnings, control suggestions, and security decisions based on the scenario simulation results after consistency correction and threat level trimming.

Images