Automatic driving situation awareness and decision-making method and system

Abstract

The invention provides an automatic driving situation awareness and decision-making method and an automatic driving situation awareness and decision-making system. According to the method, standardized scene data is generated through multi-source data fusion, a self-adaptive digital twinning environment is constructed, semantic reasoning is performed by using a knowledge graph to generate a situation tag, a decision evaluation matrix is generated based on multi-hypothesis parallel simulation, an optimal action is selected in combination with user preference, and finally system self-evolution is realized through closed-loop learning optimization. Therefore, the security and robustness of the system in a complex scene are improved. The method can solve the technical problems that prediction is not stable due to lack of semantic situation understanding in the existing automatic driving technology, and behaviors are conservative and low in efficiency due to rigid decoupling of prediction and planning.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically to an autonomous driving situation perception and decision-making method and system. Background Technology

[0002] Interactive perception-based trajectory prediction and planning decision-making technology is one of the core decision-making frameworks of current mainstream high-level autonomous driving systems. It follows a modular "perception-prediction-planning" pipeline. First, a multi-sensor fusion module constructs a dynamic bird's-eye view of the vehicle's surrounding environment and identifies other traffic participants. Next, the prediction module uses a deep learning model to predict the possible trajectories of these traffic participants within the next 3-8 seconds. To handle uncertainty, advanced prediction models can output multimodal trajectories, predicting several future paths with different probabilities for a pedestrian or vehicle. Finally, the planning module receives these predicted trajectories as "dynamic obstacles" and inputs them into a cost function. This cost function comprehensively considers safety, comfort, efficiency, and rule compliance. The planning module uses an optimization algorithm to search for a self-driving path with the lowest cost that avoids all predicted trajectories. However, this planning decision-making method suffers from technical problems such as a lack of semantic contextual understanding, unrobust prediction, and rigid decoupling between prediction and planning, leading to conservative and inefficient behavior.

[0003] Therefore, how to enable autonomous driving decision-making systems to go beyond simple kinematic prediction and achieve deep semantic understanding and forward-looking risk prediction of complex unstructured scenarios is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes an autonomous driving context perception and decision-making method and system to solve the technical problems of lack of semantic context understanding, unstable prediction, and rigid decoupling between prediction and planning, which leads to conservative and inefficient behavior.

[0005] The technical solution adopted in this invention is an autonomous driving situational perception and decision-making method, comprising: Time synchronization, spatial registration, and semantic fusion are performed on multi-source heterogeneous data from vehicle sensors to generate standardized scene data. Based on the standardized scene data and prior environmental constraints, an adaptive digital twin environment is generated; Graph reasoning is performed on the standardized scene data to generate contextual semantic tags; Based on the adaptive digital twin environment and contextual semantic tags, a set of probabilistic behavioral hypotheses and a set of candidate actions corresponding to key dynamic targets are generated; Parallel simulations are performed by combining a set of probabilistic behavioral assumptions and a set of candidate actions to generate a decision evaluation matrix; Based on the decision evaluation matrix, combined with user driving preferences and preset decision criteria, the optimal driving action is selected and issued.

[0006] Furthermore, standardized scene data is generated, including: A unified timestamp is assigned to the received multi-source heterogeneous data; The timestamp of the data frame is read, and the vehicle attitude is obtained by interpolation based on the inertial measurement unit data; By combining preset sensor external parameters, multi-source heterogeneous data are unified into the vehicle coordinate system through rigid body transformation; The transformed multi-source heterogeneous data is fused to generate a fused object; The fusion object is subjected to validity detection and anomaly removal, and the cleaned fusion object is encapsulated into a data structure with unified fields.

[0007] Furthermore, generating an adaptive digital twin environment includes: Subscribe to and receive standardized scenario data published to different data themes, including: vehicle status theme provides vehicle status vectors, fusion object theme provides a dynamic target list, and map update theme provides road and environmental information; The vehicle status, spatial location and movement status of traffic participants are updated based on the standardized scenario data, and an environmental factor vector is constructed. Based on the environmental factor vector, the set of performance degradation coefficients of the environmental factor vector at the current moment is calculated using an adapted performance degradation mathematical model. A digital twin scene snapshot is generated based on the vehicle state, spatial location, and motion state, and an adaptive digital twin environment is generated by combining the performance degradation coefficient set.

[0008] Furthermore, the performance degradation mathematical model includes: LiDAR attenuation model adapted to rainfall: in, This represents the intensity of the LiDAR signal after a propagation distance d. Indicates the initial signal strength. Indicates the extinction coefficient; Camera attenuation model adapted to lighting conditions and sensor operating status: Where K represents the camera performance degradation coefficient. This represents the illumination mapping function that describes the relationship between illumination intensity and image signal-to-noise ratio. These represent the weighting coefficients of the illumination mapping function. This represents the lens dirt confidence score output by the visual algorithm used to detect dirt on camera lenses. The weighting coefficient represents the confidence level of lens dirtiness.

[0009] Furthermore, graph inference is performed on the standardized scene data to generate contextual semantic labels, including: The standardized scenario data is mapped to pre-built vehicle knowledge graph entities; By performing graph traversal and rule matching on the vehicle knowledge graph, the risk path from the matching entity to the risk entity is determined; Based on the risk path, confidence scores of results pointing to the same risk entity are fused to generate a probability correction factor corresponding to the risk entity. The risk path and probability correction factor corresponding to the risk entity are packaged into a contextual semantic tag.

[0010] Furthermore, a decision evaluation matrix is ​​generated, including: Based on the current vehicle state and task, sampling is performed in the two-dimensional space of acceleration and curvature to generate a set of candidate action sequences; Based on the preset dynamic target basic behavior model library and predefined behavior assumptions, a set of basic behavior assumptions are generated for the key dynamic targets in the digital twin scene snapshot; The prior probability of the basic behavioral hypothesis is adjusted based on the probability correction factor in the contextual semantic label. The prior probability is combined with the basic behavioral hypothesis to generate a probabilistic behavioral hypothesis set; Load the latest digital twin scene and decay parameters, and construct a simulation task tree by combining the candidate action sequence with all elements in the probabilistic behavior hypothesis set, and distribute it to each computing core of HPC; The computing core performs parallel simulations and extrapolates future scenarios according to a preset time step. The computing core records key simulation data, including vehicle spacing, relative speed, and virtual sensor perception results, during the simulation process. Calculate the cost of a single task in the simulation task tree based on the key simulation data; The expected total cost of each candidate action is calculated by weighting the single-task cost with the corresponding probability of the probabilistic behavior hypothesis set. The candidate action sequences and their corresponding expected total costs are combined to form a decision evaluation matrix.

[0011] Furthermore, the cost of a single task in the simulation task tree is calculated based on the key simulation data, including: Calculate the safety cost, efficiency cost, comfort cost, compliance cost, and social acceptance cost based on the key simulation data. The single-task cost is obtained by weighting and summing the safety cost, efficiency cost, comfort cost, compliance cost, and social acceptance cost according to a cost function. Where J represents the cost per task. The coefficients representing the weights of the cost function. Indicates cost components.

[0012] Furthermore, the optimal driving action is selected and issued, including: Based on the highest risk value in the decision evaluation matrix and its corresponding contextual semantic label, and combined with user preference configuration parameters, the decision criteria for this decision cycle are selected. The optimal candidate action is selected from the decision evaluation matrix according to the decision criteria.

[0013] Furthermore, it also includes: performing closed-loop optimization and adaptive adjustment of the cost function weights and decision criteria based on the execution results of the control instructions.

[0014] Furthermore, closed-loop optimization and adaptive adjustment are performed on the cost function weights and decision criteria, including: The standardized scene data is used as real-world observation data, and the decision evaluation matrix, probabilistic behavior hypothesis set, and control commands are used as prediction data. After aligning the real-world observation data and the predicted data in time and space, the inconsistency score between the real-world observation data and the predicted data is calculated according to a preset metric function library. The inconsistency score is compared with a dynamic threshold corresponding to the current scenario; In response to the inconsistency score exceeding a dynamic threshold, backtrack and collect case files containing data related to the current cognitive bias event; Fine-tune the lightweight model based on the metadata of the case file.

[0015] As can be seen from the above technical solution, the beneficial technical effects of the present invention are as follows: 1. By combining semantic information in complex traffic scenarios with dynamic target behavior prediction through vehicle-mounted knowledge graph reasoning and probability correction mechanisms, the limitations of existing technologies that rely solely on kinematic history and lack robustness are overcome, thereby improving the perception and prediction capabilities of autonomous driving in unstructured and sudden scenarios.

[0016] 2. By utilizing an adaptive digital twin environment, multi-hypothesis simulation and closed-loop learning optimization, prediction and planning are tightly coupled, and the cost function weights and model parameters are dynamically adjusted. This breaks through the bottleneck of rigid decoupling of prediction and planning and simulation being limited to offline verification in existing technologies, thereby achieving a balance of flexibility, security and efficiency in online decision-making. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0018] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention; Figure 2 This is a connection diagram of the device according to Embodiment 2 of the present invention; Figure 3 This is a structural diagram of the multi-source data fusion module in Embodiment 2 of the present invention; Figure 4 This is a flowchart of the operation of the multi-source data fusion module in Embodiment 2 of the present invention; Figure 5 This is a structural diagram of the dynamic digital twin construction module of Embodiment 2 of the present invention; Figure 6 This is a flowchart illustrating the operation of the dynamic digital twin construction module in Embodiment 2 of the present invention. Figure 7 This is a structural diagram of the semantic context reasoning module in Embodiment 2 of the present invention; Figure 8 This is a flowchart illustrating the operation of the semantic context reasoning module in Embodiment 2 of the present invention. Figure 9 This is a structural diagram of the multi-hypothesis simulation prediction module in Embodiment 2 of the present invention; Figure 10 This is a flowchart of the multi-hypothesis simulation prediction module operation in Embodiment 2 of the present invention; Figure 11 This is a structural diagram of the optimal behavior decision module in Embodiment 2 of the present invention; Figure 12 This is a flowchart illustrating the operation of the optimal behavior decision module in Embodiment 2 of the present invention. Figure 13 This is a structural diagram of the closed-loop learning optimization module in Embodiment 2 of the present invention; Figure 14 This is a flowchart of the closed-loop learning optimization module operation in Embodiment 2 of the present invention. Detailed Implementation

[0019] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0020] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0021] Example 1 This embodiment provides a method for autonomous driving situational perception and decision-making. The working principle of Embodiment 1 is explained in detail below: The method flowchart of this embodiment is as follows: Figure 1 As shown, this includes: performing time synchronization, spatial registration, and semantic fusion on multi-source heterogeneous data from vehicle sensors to generate standardized scene data.

[0022] By combining the standardized scenario data with prior environmental constraints, an adaptive digital twin environment is generated.

[0023] Graph reasoning is performed on the standardized scene data to generate contextual semantic tags.

[0024] Based on the adaptive digital twin environment and contextual semantic tags, a set of probabilistic behavioral hypotheses is generated for key dynamic targets in the scene, and parallel simulation and deduction are performed in conjunction with candidate actions to generate a decision evaluation matrix.

[0025] Based on the decision evaluation matrix, combined with user driving preferences and preset decision criteria, the optimal driving action is selected and issued, and the driving action is converted into control commands that the vehicle can execute.

[0026] Based on the execution results of the control commands, the cost function weights and decision parameters are optimized and adaptively adjusted in a closed loop.

[0027] In this embodiment, further, time synchronization, spatial registration, and semantic fusion are performed on the multi-source heterogeneous data from the vehicle-mounted sensors to generate standardized scene data, including: The received multi-source heterogeneous data is assigned a precise timestamp based on the system's unified time base.

[0028] Input data frames containing spatial location information from multi-source heterogeneous data into the coordinate transformation engine.

[0029] The timestamp of the data frame is read, and the vehicle attitude is obtained based on the inertial measurement unit data interpolation.

[0030] By combining preset sensor external parameters describing the sensor installation position and attitude, the multi-source heterogeneous data in each sensor coordinate system is transformed into a unified vehicle coordinate system through rigid body transformation.

[0031] The transformed multi-source heterogeneous data is fused to generate a fused object, including: The system identifies which measurements from different sensors actually point to the same physical object. For example, the system determines that the Radar target A, the LiDAR cluster B, and the vehicle C identified by the camera are highly consistent in spatial position and speed, and therefore associates them together.

[0032] For each successfully associated object, the system uses a Kalman filter algorithm to combine measurements from multiple sensors with different noise characteristics into a single, more robust, and more accurate state description.

[0033] The transformed multi-source heterogeneous data undergoes validity testing and anomaly removal. The cleaned multi-source heterogeneous data is then encapsulated into a data structure with unified fields, including identification information, timestamp, location, velocity vector, size, category, source sensor, and confidence level.

[0034] The data structure with the unified field is stored in a preset cache queue.

[0035] Standardized scenario data is extracted from the cache queue at a fixed frequency and published to the corresponding predefined data topic so that subsequent modules can subscribe to and use it.

[0036] The transformed multi-source heterogeneous data undergoes validity testing and anomaly removal to ensure that the data processed by subsequent modules possesses basic authenticity, completeness, and timeliness. The entire process includes: Syntax and format validation is performed on multi-source heterogeneous data to ensure that the data packets themselves are structurally complete and parsable, including: The received data packets are parsed at the protocol level. Any data packets that cannot be parsed by the syntax will be discarded immediately, and a parsing error log will be logged to prevent malformed data from causing subsequent processing modules to crash. After successful parsing, check if the key fields exist and if their values ​​are within the expected physical or logical range. Data that lacks key fields or whose field values ​​are significantly outside the reasonable range will be marked as "low quality" or discarded directly, depending on the importance of the data.

[0037] For each received V2X message (such as BSM, SPaT), the corresponding authentication algorithm is executed for verification, including: Extract the certificate from the message and verify whether the certificate was issued by a trusted root certificate authority (CA) and whether it is valid (usually achieved through a preloaded certificate trust list CTL). Using the public key in the certificate, the message signature and content are decrypted and hashed. Any message that fails to verify the signature or has an invalid certificate will be immediately discarded and a security alert will be triggered, as it may be a spoofing or forgery attack. For cloud data transmitted via protocols such as TCP / IP, verify the correctness of its transport layer and application layer checksums (such as TCPChecksum, SHA-256 Hash) to ensure that the data has not been accidentally or maliciously tampered with during transmission. Data packets that fail the checksum will be discarded and may be requested to be retransmitted.

[0038] Phase 3: Timeliness Verification Ensure the system uses "fresh" data to prevent replay attacks and erroneous decisions based on outdated information, including: The timestamp on the data packet is compared with the current high-precision system timestamp overlaid by SDFM, and the time difference between the two is calculated. If the time difference exceeds a validity window preset for this data type (e.g., 100ms for BSM messages and 5 minutes for weather data), the data is considered outdated and discarded. Maintain a cache of recently received messages, which stores the unique identifiers of the messages. When a new message is received, first check the cache. If the identifier of the message already exists, it means that it is a duplicate message and should be discarded immediately. This can effectively defend against simple replay attacks.

[0039] Using physical laws and prior knowledge to eliminate obviously unreasonable data points, including: Define a precise 3D vehicle mask that includes components such as the vehicle body, rearview mirrors, and roof rack; iterate through each point in the point cloud, and if its coordinates are inside the vehicle mask, mark and remove it. By applying a combination of filters, points with reflection intensity below a preset threshold are removed; these are typically airborne particles (rain, snow, dust). Points outside the effective working range of LiDAR are also removed. The distance distribution between each point and its neighborhood is calculated, and isolated points that significantly deviate from the distribution are removed. Through this combined approach, a cleaner point cloud that is more focused on real-world obstacles can be obtained.

[0040] Continuously track the Doppler velocity of each target. If a target's velocity remains close to zero for several consecutive frames and its position matches a static element (such as a guardrail or lamppost) in the high-precision map, it is classified as static clutter, its threat level is reduced, or it is removed from the list of dynamic targets.

[0041] Preliminary cross-source plausibility checks are performed. For example, if the cloud API reports that the current weather is "sunny," but the camera image detects that the windshield wipers are working at the highest speed, the system will lower the credibility score of the weather API data. If a V2X message claims that a car is at a certain location, but LiDAR does not detect any object at that location, the credibility of the V2X message will also be lowered. The results of this cross-validation are not discarded immediately, but are added to the data in the form of a "credibility score" for reference by downstream fusion and decision-making modules.

[0042] The above four-stage cleaning process can effectively filter out noise, invalid, outdated and false information, providing a high-quality and highly credible information foundation for the entire decision-making system.

[0043] In this embodiment, the standardized scene data is further combined with prior environmental constraints to generate an adaptive digital twin environment, including: Subscribe to and receive standardized scenario data published to different data themes, including: vehicle status theme provides vehicle status vectors, fusion object theme provides a dynamic list of targets containing dynamic traffic participant data, and map update theme provides road and environmental information.

[0044] The vehicle state is updated based on the vehicle state vector, including the vehicle's pose and dynamic state.

[0045] The spatial location and movement status of traffic participants in the digital twin environment are updated based on the dynamic target list.

[0046] Rainfall, illumination, and sensor operating status are extracted from the map update topic to construct an environmental factor vector.

[0047] The environmental factor vector is input into the attenuation model library to calculate the set of performance attenuation coefficients of the environmental factor vector at the current moment.

[0048] Generate a snapshot of the digital twin scene containing the vehicle status and the digital twin environment, and publish it to the data topic of the digital twin scene.

[0049] The set of performance degradation coefficients is published to the data topic of degradation parameters for subsequent simulation prediction.

[0050] After the above steps are completed, the process is repeated: wait for the next data update, return to the subscription and receiving steps, and enter a new twin construction loop.

[0051] In this embodiment, the attenuation model library further stores performance attenuation mathematical models for different environmental factor vectors, used to map environmental factors to performance attenuation coefficients, including: mapping the performance attenuation coefficient of rainfall according to the LiDAR attenuation model: in, This represents the intensity of the LiDAR signal after a propagation distance d. Indicates the initial signal strength. This represents the extinction coefficient.

[0052] Based on the performance degradation coefficient mapped from the camera attenuation model to the illumination and sensor operating conditions: Where K represents the camera performance degradation coefficient. This represents a nonlinear illumination mapping function that describes the relationship between illumination intensity and image signal-to-noise ratio. These represent the weighting coefficients of the illumination mapping function. This represents the lens dirt confidence score output by the visual algorithm used to detect dirt on camera lenses. The weighting coefficient represents the confidence level of lens dirtiness.

[0053] Furthermore, graph inference is performed on the standardized scene data to generate contextual semantic labels, including: Continuous monitoring of standardized scenario data within the aforementioned data topic.

[0054] In response to any significant change in the standardized scenario data, such as a vehicle entering a new geofence, the time entering a new preset period, or receiving a new weather or event alert, the standardized scenario data is mapped to a pre-built vehicle knowledge graph entity.

[0055] The entities in the vehicle knowledge graph include: spatial mapping that matches vehicle GPS coordinates with geographic entities, time mapping that matches system time with time entities, event mapping that matches event data with event entities, and environment mapping that matches environmental data such as rainfall with environmental entities.

[0056] A structured query request is generated based on the mapping result, i.e., the entity in the vehicle knowledge graph.

[0057] Based on the structured query request, graph traversal and rule matching are performed on the vehicle knowledge graph to find potential risks associated with risk paths from matching entities to risk entities, and these risks are stored in the risk type list.

[0058] Based on the risk paths, confidence scores are fused for results pointing to the same risk entity to generate a probability correction factor corresponding to the risk entity. The confidence fusion includes: when multiple paths point to the same risk entity, the inference engine fuses them according to preset weights or simple logical operations (OR) to arrive at a comprehensive risk judgment.

[0059] The risk type list and probability correction factors are packaged into a standard structured format and semantically tagged with context. The format includes context ID, timestamp, context name, contributing entity, inferred risk list containing risk type and probability modifiers, and recommended stance.

[0060] The inference engine packages the final result into a standard-format structured context semantic tag and publishes the context semantic tag to the data topic of the internal data bus.

[0061] In response to any significant change in the standardized scenario data, the previously published scenario semantic tags are republished to the data topic on the internal data bus. If, at the next time step, the key input data remains unchanged, the vehicle is still in the same area, and the time period is the same, the previous semantic tag is directly reused and published to avoid unnecessary recalculation and ensure system efficiency. The complete inference process is only re-executed when the triggering condition is met again.

[0062] In this embodiment, the system constructs a multi-dimensional, hierarchical event-driven triggering mechanism to determine whether standardized scenario data has undergone significant changes. This mechanism continuously monitors the standardized data stream, and only when changes in one or more dimensions meet preset quantization conditions will a complete semantic context re-evaluation be triggered. This mechanism is mainly divided into the following four types of quantization triggers: Spatial and geographic triggers are used to monitor changes in the location and topological relationships of vehicles in geospatial space. This includes: the system preloads a database containing multiple semantic geofences; the triggers calculate the relationship between the vehicle's current coordinates and these polygons in real time; and the triggers are activated when the vehicle's geofence state undergoes a discrete transformation. The system defines different "influence radius" thresholds for different types of POIs, which are triggered when the Euclidean distance between a vehicle and a high-priority POI first crosses its influence radius threshold. Based on the positioning of high-precision maps, the system can obtain the road attributes of the current lane, which is triggered when the road type enumeration value changes.

[0063] A time trigger, based on the system clock, monitors when time enters a predefined semantic period, including: defining a time period configuration file within the system, dividing a day into multiple semantic periods, and triggering when the system clock crosses the start or end time of any of the defined periods.

[0064] Dynamic environmental data triggers monitor continuously changing external environmental data and are triggered when the magnitude or state of the change exceeds a threshold. This includes: mapping continuous weather parameters to discrete state levels; and triggering when the mapped weather state enumeration value changes. The vehicle density around the vehicle is calculated by fusing sensor data and divided into different levels. When the traffic density level changes and the new level lasts for more than a small hysteresis window, it is triggered.

[0065] Explicit event triggers are the highest priority triggers. They are directly triggered by specific, high-value information sent by external systems, including: The system maintains a list of "high-priority message types". When a V2X or cloud message of type in this list is received and verified, semantic context reasoning is immediately and unconditionally triggered.

[0066] Therefore, through the aforementioned multi-dimensional triggering mechanism, "on-demand assessment" is achieved, which concentrates computing resources on when needed, ensuring the system's rapid response to changes in critical situations while significantly optimizing the overall computational load. The aforementioned quantitative indicators are designed as configurable, calibrable, and adjustable parameters.

[0067] In this embodiment, further, based on the adaptive digital twin environment and contextual semantic tags, a probabilistic set of behavioral hypotheses is generated for key dynamic targets in the scene, and parallel simulation and deduction are performed in combination with candidate actions to generate a decision evaluation matrix, including: subscribing to and receiving the attenuation parameters, digital twin scene snapshots and contextual semantic tags.

[0068] Based on the current vehicle state and task, sampling is performed in a two-dimensional space of acceleration and curvature to generate a set of candidate action sequences.

[0069] Based on vehicle dynamics constraints and driving tasks such as navigation objectives, sampling is performed in a two-dimensional space of acceleration and curvature to generate a set of discrete candidate action sequences that are feasible in the short future time (e.g., 3-5 seconds).

[0070] Based on the preset dynamic target basic behavior model library and predefined behavior assumptions, a set of basic behavior assumptions are generated for the key dynamic targets in the digital twin scene snapshot.

[0071] The prior probability of the basic behavioral hypothesis is adjusted based on the probability correction factor in the contextual semantic label.

[0072] The prior probabilities are normalized and merged with the basic behavioral assumptions to generate a probabilistic behavioral hypothesis set.

[0073] The parallel simulation engine loads the latest digital twin scenarios and attenuation parameters.

[0074] The candidate action sequence is combined with all elements in the probabilistic behavior hypothesis set to construct a simulation task tree, which is then distributed to each computing core of HPC (High-Performance Computing).

[0075] The computing core performs simulations at time steps far exceeding real-time (Δt = 0.1 seconds) until a preset simulation duration is reached. Within each time step, future scenarios are simulated in parallel, including: Update the vehicle status based on the candidate action sequence.

[0076] Update the target state based on a set of probabilistic behavioral assumptions.

[0077] The virtual sensor model is invoked to apply the attenuation parameters to simulate the vehicle's perception.

[0078] The computing core records key simulation data, including vehicle spacing, relative speed, and virtual sensor perception results, during the simulation process.

[0079] The cost of a single task in the simulation task tree is calculated based on the key simulation data.

[0080] The expected total cost of each candidate action is calculated by weighting the single-task cost with the corresponding probability of the probabilistic behavior hypothesis set.

[0081] The candidate action sequences and their corresponding expected total costs are combined to form a decision evaluation matrix, which is then published to the decision evaluation topic on the internal data bus for downstream decision-making to access.

[0082] Furthermore, in this embodiment, calculating the cost of a single task in the simulation task tree based on the key simulation data includes: Calculate the cost of a single task using the following cost function: Where J represents the cost per task. Indicates the weighting coefficient. Indicates cost components; The cost component include: C_safety: The safety cost calculated based on the minimum TTC (Time-to-Collision) or minimum vehicle-to-vehicle distance that occurs in the simulation. If a collision occurs, the cost is infinite.

[0083] C_efficiency: Measures the efficiency cost of the time, distance traveled, or average speed deviation required for a vehicle to reach a target point.

[0084] C_comfort: Comfort cost calculated based on longitudinal and lateral acceleration.

[0085] C_legality: Compliance cost based on whether a candidate action violates traffic rules.

[0086] C_social: The social acceptance cost generated by combining semantic context labels to perform a social evaluation of an action.

[0087] In this embodiment, further, based on the decision evaluation matrix, combined with the user's driving preferences and preset decision criteria, the optimal driving action is selected and issued, and the driving action is converted into a control command that the vehicle can execute, including: subscribing to and receiving the decision evaluation matrix.

[0088] Based on the highest risk value in the decision evaluation matrix and its corresponding scenario semantic label, and in conjunction with user preference configuration parameters, the decision criterion for this decision cycle is selected. When there is a safety cost exceeding a preset threshold in the decision evaluation matrix, or when the scenario semantic label contains a high-risk element, the decision criterion of minimizing the maximum loss is selected. When the user preference configuration parameters indicate a balanced driving mode, the decision criterion of maximizing expected utility is selected. When the user preference configuration parameters indicate a conservative driving mode, the trigger sensitivity of the decision criterion of minimizing the maximum loss (Minimax criterion) is increased.

[0089] The decision evaluation matrix is ​​calculated based on the decision criteria, and the corresponding optimal candidate action is selected.

[0090] The target trajectory corresponding to the optimal candidate action is converted into raw control commands that can be recognized by the vehicle's underlying actuators, including target acceleration and target steering angle.

[0091] The original control commands are filtered and smoothed, and then limited within the vehicle dynamics constraints to obtain the final control commands.

[0092] The final control command is issued to the vehicle motion control module to drive the physical actuator to complete the optimal driving action.

[0093] In this embodiment, further, based on the execution result of the control command, the cost function weights and decision parameters are optimized and adaptively adjusted in a closed loop, including: subscribing to and receiving the standardized scene data as real-world observation data.

[0094] Subscribe to and receive the decision evaluation matrix, probabilistic behavior hypothesis set, and control instructions as prediction data.

[0095] After aligning the real-world observation data and predicted data in time and space, an inconsistency score is calculated based on a pre-defined metric function library. The inconsistency metric function library provides various methods for calculating inconsistency.

[0096] The inconsistency score is compared with a dynamic threshold corresponding to the current scenario.

[0097] In response to the inconsistency score exceeding a dynamic threshold, case files related to the current cognitive bias event are backtracked and collected.

[0098] The case files include: Input layer: includes standardized scene data.

[0099] Twin layer: includes digital twin scene snapshots and decay parameters.

[0100] Cognitive layer: includes contextual semantic tags.

[0101] Prediction layer: includes a set of probabilistic behavioral hypotheses and prior probabilities.

[0102] Decision-making level: includes optimal candidate actions and evaluation matrix.

[0103] Results layer: The actual trajectories of all objects in the physical world.

[0104] Note: The engine adds metadata tags to this case file, such as [Event ID, Timestamp, Inconsistency Score, Bias Type: Lateral Prediction Failure, Triggering Context: Rainy Intersection].

[0105] Based on the metadata of the case file, a model optimization task is triggered, including online fine-tuning of the lightweight model and adding it to the offline training pool in the cloud.

[0106] If the inconsistency score is less than the dynamic threshold, the cache is cleared.

[0107] The beneficial technical effects of this embodiment are as follows: By combining semantic information in complex traffic scenarios with dynamic target behavior prediction through vehicle-mounted knowledge graph reasoning and probability correction mechanisms, the limitations of existing technologies that rely solely on kinematic history and lack robustness are overcome, thereby improving the perception and prediction capabilities of autonomous driving in unstructured and sudden scenarios; by utilizing an adaptive digital twin environment, multi-hypothesis simulation and closed-loop learning optimization, prediction and planning are tightly coupled, and the cost function weights and model parameters are dynamically corrected, breaking through the bottleneck of rigid decoupling of prediction and planning and simulation limited to offline verification in existing technologies, thus achieving a balance of flexibility, safety and efficiency in online decision-making.

[0108] Example 2 In conjunction with Embodiment 1, Embodiment 2 includes an autonomous driving context perception and decision-making system. This system is designed as a hardware and software collaborative system deployed on an intelligent connected vehicle, mainly composed of six core modules that interact at high speed and low latency through standardized data interfaces. These six modules are: Multi-Source Data Fusion Module (SDFM), Dynamic Digital Twin Construction Module (DDTCM), Semantic Context Reasoning Module (SCRM), Multi-Hypothesis Simulation Prediction Module (MHSPM), Optimal Behavior Decision Module (OBDM), and Closed-Loop Learning Optimization Module (CLOM). The device connection diagram of this embodiment is shown below. Figure 2 As shown, it includes: The multi-source data fusion module is used to perform time synchronization, spatial registration, and semantic fusion of multi-source heterogeneous data from vehicle sensors to generate standardized scene data.

[0109] The dynamic digital twin construction module is used to combine the standardized scene data with prior environmental constraints to generate an adaptive digital twin environment.

[0110] The semantic context reasoning module is used to perform graph reasoning on the standardized scene data and generate context semantic labels.

[0111] The multi-hypothesis simulation prediction module is used to generate a set of probabilistic behavioral hypotheses for key dynamic targets in the scene based on the adaptive digital twin environment and contextual semantic tags, and to perform parallel simulation and deduction in combination with candidate actions to generate a decision evaluation matrix. The optimal behavior decision module is used to select and issue the optimal driving action based on the decision evaluation matrix, combined with the user's driving preferences and preset decision criteria, and to convert the driving action into control commands that the vehicle can execute.

[0112] The closed-loop learning optimization module is used to perform closed-loop optimization and adaptive adjustment of the cost function weights and decision parameters based on the execution results of the control commands.

[0113] The input layer of this system consists of: vehicle-mounted sensors, V2X OBU, GNSS / IMU, and cloud platform interfaces (for weather API, traffic event API, etc.).

[0114] The system's processing layer consists of the following: SDFM receives all inputs and performs timestamp alignment and coordinate system unification. DDTCM acquires data from SDFM and internally includes a vehicle twin submodule, an environment twin submodule, and a critical sensor attenuation submodule. SCRM receives data from SDFM and DDTCM; its core is a traffic context knowledge graph. MHSPM receives outputs from DDTCM and SCRM and internally includes a behavior hypothesis generator, a parallel simulation engine, and a multi-objective evaluator. OBDM receives the evaluation results from MHSPM and outputs control commands. CLOM runs in parallel, continuously monitoring the data streams from SDFM and MHSPM; its core is an inconsistency monitoring unit.

[0115] The output layers of this system are: vehicle control commands (output to CAN / Ethernet bus) and high-value learning samples (output to cloud training platform).

[0116] In this embodiment, the multi-source data fusion module (SDFM - Sensor & Data FusionModule) is a software service running on the in-vehicle high-performance computing unit (HPC). It includes multiple data adapters corresponding to cameras, LiDAR, millimeter-wave radar, GNSS / IMU, V2X OBU, and cloud data interfaces. Its function is to transform heterogeneous data sources with different physical characteristics, sampling frequencies, coordinate systems, and communication protocols into a unified, synchronized, and formatted data stream, providing a clean, reliable, and ready-to-use data foundation for all subsequent advanced processing modules.

[0117] SDFM is designed as a multi-layered software architecture, running on an onboard high-performance computing unit (HPC) and tightly coupled to the vehicle's hardware interface. The structure diagram of the multi-source data fusion module is shown below. Figure 3 As shown, its internal structure can be divided into three layers: The Hardware Adaptation Layer (HAL) includes a series of dedicated device drivers and software adapters: Sensor driver: The hardware adaptation layer communicates directly with the ECU of the vehicle sensors, such as receiving image data from cameras, point cloud data from lidar, target list data from millimeter-wave radar, and distance data from ultrasonic radar via CAN bus or vehicle Ethernet.

[0118] GNSS / IMU adapter: It parses NMEA messages or custom format data from high-precision positioning units via serial port or UDP protocol, and extracts information such as latitude and longitude, elevation, speed, heading, and attitude angles (roll, pitch, yaw).

[0119] V2X OBU Adapter: Listens to the network interface from the Onboard Unit (OBU) and parses standard V2X messages, such as Basic Safety Messages (BSM), Map Messages (MAP), and Traffic Light Phase and Timing Messages (SPAT).

[0120] Cloud platform interface: Responsible for communicating with the cloud server via the 4G / 5G connection of the in-vehicle T-Box. It actively requests or passively receives data services from the cloud, such as weather APIs, real-time traffic event APIs, and dynamic updates of high-precision maps.

[0121] The Data Processing Core is the core logic of SDFM, consisting of three key components: Time Synchronization Manager: Responsible for maintaining a unified, high-precision time base for the system. It synchronizes with the onboard PTP (Precision Time Protocol) master clock or the PPS (Pulse Per Second) signal provided by the GNSS module to ensure the accuracy of all timestamps.

[0122] Coordinate transformation engine: It stores the vehicle's precise calibration parameters, including the precise three-dimensional position (x, y, z) and mounting attitude (roll, pitch, yaw) of each sensor relative to the vehicle coordinate system origin, which is usually the rear axle center.

[0123] Data formatting and buffer queues: Define the standard data structure for interaction between all modules in the system, and set up a fixed-length, thread-safe circular buffer for each data type.

[0124] The Data Service Publication Layer consists of an efficient data distribution mechanism implemented using a publish / subscribe pattern, including: Data topics: Define a "topic" for each type of processed data stream, such as topic_camera_front_image, topic_lidar_points, topic_fused_objects, topic_vehicle_state.

[0125] Publisher: Publishes formatted data to the corresponding topic.

[0126] Subscriber Interface: Provides an API to other modules within the system, such as DDTCM and SCRM, allowing them to subscribe to data topics of interest on demand.

[0127] SDFM runs as a continuous, high-concurrency pipeline process. The following is a typical runtime cycle: Figure 4As shown: Step 1 involves parallel data acquisition and timestamp calibration: Each adapter and driver in the HAL layer acts as an independent thread or process, receiving raw data in parallel from its respective hardware interface, such as an image frame, a point cloud packet, or a BSM message.

[0128] The moment data arrives at HAL, the time synchronization manager immediately assigns it a precise timestamp based on the system's unified time base. This step is crucial because it minimizes time errors caused by operating system scheduling delays.

[0129] Step 2 is coordinate system unification processing: All data containing spatial location information, such as lidar point clouds, radar targets, and locations in V2X messages, are fed into the coordinate transformation engine.

[0130] The engine reads the timestamp of the data frame and interpolates the vehicle's attitude at that precise moment from the IMU data stream. Then, using pre-stored sensor extrinsic parameters, including installation position and attitude, it transforms the coordinates of all points from their respective sensor coordinate systems to a unified vehicle coordinate system through standard rigid body transformation (rotation + translation) mathematical operations.

[0131] Step 3 involves preliminary data cleaning and formatting: Before entering the buffer queue, the data undergoes a preliminary cleaning process. This includes removing invalid LiDAR points, such as reflections from inside the vehicle body, and verifying the signatures of V2X messages to ensure their reliable origin.

[0132] The cleaned data is encapsulated into a predefined standard data structure. In this embodiment, a FusedObject structure contains fields such as id, timestamp, position_in_vehicle_frame, velocity_vector, dimensions, classification (vehicle / pedestrian), source_sensor_list, and confidence.

[0133] Step 4 is to write the data to the buffer queue: The formatted data objects are written to the corresponding circular buffer in the data processing core layer. The advantage of using a circular buffer is that it ensures data freshness, automatically overwrites the oldest data, and avoids the overhead and uncertainty of dynamic memory allocation.

[0134] Step 5 is data publishing: The publisher thread of the data service publishing layer retrieves the latest data from each buffer queue at a fixed high frequency (100Hz in this embodiment).

[0135] The retrieved data objects are published to their corresponding data topics. In this embodiment, the latest list of FusedObjects is published to the topic_fused_objects. At this time, all other modules that have subscribed to this topic (such as the environment twin submodule of DDTCM) will receive a data update notification almost simultaneously and can immediately access this fresh, synchronized, and formatted data.

[0136] Furthermore, in this embodiment, the Dynamic Digital Twin Construction Module (DDTCM) serves as a "high-fidelity real-time sandbox" before behavioral decisions are made. Its core task is no longer to create a general virtual world for offline testing, but rather to build and maintain a high-fidelity, adaptive online digital mirror of the physical vehicle in its specific environment at this very moment. It will reproduce the geometric and physical state of the vehicle and its environment, and can synchronize in real time the performance fluctuations of the physical vehicle caused by internal wear and tear and external environmental influences, especially the performance degradation of sensors.

[0137] DDTCM consists of three interconnected and dynamically updated sub-modules, which together construct a complete digital twin scenario that coexists with the real world. Its structure diagram is as follows: Figure 5 As shown, it includes a vehicle twin submodule, an environmental twin submodule, and a sensor attenuation submodule.

[0138] The Vehicle Twin Sub-module contains a vehicle dynamics model (VDM), a vehicle state vector (VSV), and a slow-varying parameter database (SPDB).

[0139] The vehicle dynamics model employs a simplified mathematical model, but one that is highly sensitive to key decision-making parameters, including braking distance and steering response. This model includes a single-track bicycle model and a more complex multibody dynamics model. It receives control commands (target acceleration, steering wheel angle) and outputs the vehicle's motion state.

[0140] The Vehicle State Vector (VSV) is a real-time updated data structure that stores the instantaneous vehicle state obtained from the SDFM, such as [x, y, z, yaw, pitch, roll, vx, vy, ax, ay, steering_angle, brake_pressure]. This vector is used to initialize the VDM state at the beginning of each simulation step, ensuring that the simulation starting point is completely consistent with the physical vehicle.

[0141] The Slowly Variable Parameter Database (SPDB) stores vehicle performance parameters that do not change drastically with a single drive but change over time due to wear and tear. During simulation calculations, VDM dynamically loads parameters from the SPDB, ensuring that the simulation results reflect the long-term health of the vehicle. These vehicle performance parameters include: tire_friction_coefficient: The coefficient of friction of the tire, which can be learned and updated by CLOM over a long period of time based on historical slippage data.

[0142] brake_system_delay: Braking system response delay, which can be calibrated offline.

[0143] power_consumption_model: Energy consumption model for the battery / engine.

[0144] Environment Twin Sub-module: Contains a static environment layer and a dynamic entity layer.

[0145] The static environment layer is built upon high-definition map (HD Map) data. It includes the road's geometric topology (lane lines, road boundaries), curvature, slope, and 3D models of fixed elements such as traffic signs, traffic lights, and buildings. This layer is relatively stable and is only modified when it receives updates from the HD Map.

[0146] The dynamic entity layer is the fastest-changing part of the scene. It maintains a dynamic list of targets, where each object is a data structure containing information obtained from different topics published by SDFM. This layer is refreshed in real time at a high frequency (e.g., 10-30Hz) to ensure that traffic participants in the virtual scene are fully synchronized with the physical world.

[0147] The Sensor Degradation Sub-module is comprised of a configurable Degradation Model Library (DML) and a Performance Applicator Engine (PAE). The DML pre-stores mathematical models for performance degradation of different sensors. These models map environmental factors to performance degradation coefficients. The mathematical models include: Performance degradation coefficient of rainfall mapped using the LiDAR attenuation model: in, This represents the intensity of the LiDAR signal after a propagation distance d. Indicates the initial signal strength. The extinction coefficient is a function related to rainfall intensity (mm / h) or visibility (m), which can be implemented using a lookup table or a small neural network f(rain,fog). Based on the performance degradation coefficient mapped from the camera attenuation model to the illumination and sensor operating conditions: Where K represents the camera performance degradation coefficient. This represents a (non-linear) illumination mapping function that describes the relationship between illumination intensity and image signal-to-noise ratio. These represent the weighting coefficients of the illumination mapping function. Lens dirt confidence score is derived from a visual algorithm specifically designed to detect dirt on camera lenses. The weighting coefficient represents the confidence level of lens dirtiness.

[0148] The Performance Parameter Application Engine (PAE) applies the attenuation coefficients calculated by DML to the virtual sensor models called by the subsequent Simulation Module (MHSPM). When the MHSPM performs simulations in the digital twin, it needs to simulate the "sensing" process. PAE intercepts this process and modifies the behavior of the virtual sensor. For LiDAR, PAE will adjust the attenuation coefficients accordingly. Reduce the effective detection range of virtual LiDAR point clouds and add random noise points based on the rain and fog model.

[0149] For cameras, PAE uses image processing algorithms, including reducing contrast, adding Gaussian noise, and simulating glare, to "downgrade" the images generated by the virtual camera before sending them to the virtual perception algorithm for processing.

[0150] DDTCM runs in a continuous loop to ensure a tight coupling between the digital twin and the physical world. The following is a typical running cycle: Figure 6 As shown: Each submodule of DDTCM subscribes to relevant topics published by SDFM. The vehicle status topic provides vehicle status vectors, the fusion object topic provides a dynamic list of targets, and the map update topic provides road and environmental information.

[0151] The vehicle twin module updates its internal vehicle status based on the latest VSV data received.

[0152] The environmental twin submodule updates the location, speed, and other information of traffic participants in the virtual scene based on the latest dynamic target list.

[0153] The sensor attenuation submodule extracts key environmental factors from the map update topic, including rainfall, visibility, and light intensity.

[0154] The sensor attenuation submodule inputs the extracted environmental factor vectors into the corresponding model in DML, and the model calculates the performance attenuation coefficient of each sensor at the current moment.

[0155] DDTCM publishes the completed digital twin scene snapshot, which includes the complete vehicle and environmental states, to the twin scene data topic for MHSPM to use. At the same time, it publishes the calculated sensor attenuation coefficient set to the attenuation parameter data topic.

[0156] Finally, the module returns to the subscription and reception steps, waits for the next data update from SDFM, and enters a new build loop.

[0157] In this embodiment, the fundamental task of the Semantic Context Reasoning (SCRM) module is to transform the massive, heterogeneous, low-level data stream provided by SDFM into a high-level, structured, machine-understandable semantic context. It doesn't concern itself with "there is a moving object 50 meters ahead," but rather focuses on answering "we are in a 'rainy night construction zone,' which means the moving object ahead is very likely a tired worker, and the ground is slippery, increasing braking distance." SCRM uses a pre-built knowledge base to perform deep reasoning on the current scenario, providing a "scenario qualitative" conclusion with a clear risk indication for subsequent risk prediction and decision-making modules, thus giving the vehicle's behavioral decisions similar to human "common sense" and "foresight." The core of SCRM is an in-vehicle, optimized knowledge reasoning system. It mainly consists of three tightly coupled sub-units: an in-vehicle traffic context knowledge graph, a data abstraction and query generation unit, and a reasoning engine and tag generator. Figure 7 As shown.

[0158] The in-vehicle traffic context knowledge graph is a graph database stored in the non-volatile memory of the in-vehicle HPC. It consists of entities and relations, the entities including: Geographic entities are derived from POIs on high-precision maps, including: schools, hospitals, shopping malls, bar streets, highway interchanges, and construction zones.

[0159] The time entity is a predefined time period, including: weekday morning rush hour, night, holidays, and weekend afternoon.

[0160] Events are dynamically updated via cloud services, including: sporting events, concerts, and temporary traffic control.

[0161] Environmental entities are abstracted from environmental data and include: heavy rain, dense fog, icy roads, and strong backlighting.

[0162] Risk entities are predefined typical risk patterns, including: pedestrians suddenly running out, vehicles suddenly appearing out of the way, drunk driving, and non-motorized vehicles going against traffic.

[0163] The relationship is used to define the association between entities, including: The `locatedAt` directive indicates a "located" relationship, used to describe the association between a geographic entity and a specific geographic location. For example, `locatedAt(school, GPS polygon region)` means that the entity "school" is located within a certain GPS polygon region.

[0164] hasProperty indicates an attribute relationship, used to describe an entity having certain specific attributes or characteristics. For example, hasPropert(school, {school dismissal time:'15:00-16:30'}) indicates that the school entity has an attribute, namely, the school dismissal time is from 3 pm to 4:30 pm.

[0165] `triggersRisk` indicates a risk-triggering relationship, used to describe how an entity directly triggers a certain risk event. For example, `triggersRisk(Bar Street, Drunk Driving)` means that the entity "Bar Street" is likely to trigger the risk of drunk driving.

[0166] increasesRisk indicates an increased risk relationship, used to describe how an entity increases the probability or intensity of another risk event. For example, increasesRisk(heavy rain, pedestrian suddenly running out) means that the entity in a heavy rain environment will increase the probability of the risk of a pedestrian suddenly running out.

[0167] co_occursWith indicates a relationship of coexistence or simultaneous occurrence, used to describe two entities that frequently appear at the same time or in space. For example, co_occursWith(sports event, temporary traffic control) indicates that the sports event entity usually occurs at the same time as the temporary traffic control event entity.

[0168] The basic knowledge graph is pre-installed when the vehicle leaves the factory, and event entities and some relationships can be updated regularly and incrementally through cloud services.

[0169] The data abstraction and query generation unit consists of a series of data classifiers and logical rule engines. This unit is responsible for translating the continuous, raw data stream from SDFM into discrete entities that the knowledge graph can understand. First, the vehicle's current GPS coordinates are matched with geographical entities in the knowledge graph. For example, geofencing is used to determine whether the vehicle has entered the "school" area.

[0170] Furthermore, the current system time is matched to a predefined time entity; for example, 3:30 PM is a weekday afternoon.

[0171] Furthermore, examine the event information obtained from the cloud to see if the current time and space coincide with a certain event entity.

[0172] Furthermore, data such as rainfall >25mm / h are classified as the environmental entity of heavy rain.

[0173] Based on the above mapping results, a structured query request is generated, for example: Find_Context({Location:'School', Time:'Weekday Afternoon', Weather:'Heavy Rain'}).

[0174] The inference engine and tag generator are a graph traversal and rule matching-based inference engine. Upon receiving a query request, the engine performs inference tasks on the knowledge graph, searching for paths from currently matched entities to risky entities. For example, for the matched entities (school, heavy rain), it will find two paths: Path 1: School - hasProperty({school dismissal time}) - [matches current time] - triggersRisk - pedestrians suddenly rush out.

[0175] Path 2: Heavy rain - increases risk - pedestrians suddenly rush out.

[0176] When multiple paths point to the same risk entity, the inference engine will merge them based on preset weights or simple logic to arrive at a comprehensive risk assessment.

[0177] Finally, the inference engine packages the inference results into a rich, structured semantic tag. This tag is the core output of SCRM.

[0178] SCRM operates through a process that combines event-driven mechanisms with periodic polling. The SCRM flowchart is as follows: Figure 8 As shown, it includes: Step 1: After SCRM starts, it continuously listens for relevant topics published by SDFM.

[0179] Step 2: When any key input data changes significantly, such as a vehicle entering a new geofence, time entering a new preset time period, or receiving a new weather or event alert, the data abstraction unit is triggered.

[0180] Step 3: The data abstraction unit executes its mapping logic, transforming the current raw data into a set of matched knowledge graph entities and constructing an internal query request.

[0181] Step 4: The inference engine receives the query request and performs graph traversal and rule matching on the vehicle knowledge graph to find potential risks associated with the current entity. Finally, it merges the results of multiple inference paths and quantifies their impact.

[0182] Step 5: The inference engine packages the final result into a standard-format structured semantic tag, which is published to the data topic on the internal data bus for MHSPM to subscribe to.

[0183] Step 6: If the key input data remains unchanged in the next time step (i.e., the vehicle is still in the same area and the time period remains the same), SCRM will directly reuse and publish the previous semantic tag to avoid unnecessary duplicate calculations and ensure system efficiency. The complete inference process will only be re-executed when the triggering condition is met again.

[0184] In this embodiment, the Multi-Hypothesis Simulation & Prediction Module (MHSPM) performs rapid, parallel, and multi-possibility extrapolations of the "future" in a highly realistic, real-time digital twin environment before the vehicle makes any physical actions. This module proactively generates multiple behavioral hypotheses consistent with contextual logic and evaluates the consequences for the vehicle in responding to each hypothesis. Through this analysis, MHSPM can identify driving strategies that are robust under all possibilities—that is, driving strategies with the lowest risk and highest reward—thus ensuring the vehicle's final decision-making is robust and intelligent.

[0185] The structural diagram of MHSPM is as follows: Figure 9 As shown, MHSPM is a computationally intensive module consisting of four collaborative sub-units: a candidate action generator, a probabilistic behavior hypothesis generator, a parallel simulation engine, and a multi-objective evaluator.

[0186] The candidate action generator consists of a sampler based on vehicle dynamics constraints and the driving task, responsible for generating a set of discrete driving action sequences feasible within a short future time (3-5 seconds). Actions are typically sampled in a two-dimensional space of acceleration and curvature. Its sampling strategy is lattice sampling, which uniformly generates N candidate actions in the action space, for example: Furthermore, the candidate action generator, combined with the high-level navigation task, selectively generates more task-related actions, ultimately outputting a set of candidate action sequences, such as: {A1: Maintain a constant speed in the lane, A2: Slightly reduce speed, A3: Prepare to change lanes to the right...} The probabilistic behavior hypothesis generator consists of a context-driven model library and a probability adjustment engine. Based on a preset dynamic target basic behavior model library and predefined behavior models, the probabilistic behavior hypothesis generator generates a probabilistic, discrete set of behavior hypotheses for key dynamic targets in the scene, especially those with high uncertainty, such as pedestrians and non-motorized vehicles, rather than a single trajectory. Furthermore, it combines candidate actions to perform parallel simulation and deduction to generate a decision evaluation matrix.

[0187] The model library stores a variety of basic behavioral models, including: CVM (Constant Velocity Model): a constant velocity model; and Social-LSTM / Social-GAN: deep learning prediction models that take into account interactions.

[0188] The predefined behavior macros (Maneuver Templates) include parameterized templates for typical dangerous behaviors such as "suddenly crossing" and "emergency braking".

[0189] In the hypothesis generator, a set of basic behavioral hypotheses (such as "walk along the roadside" or "cross the road") are first generated for the target. Then, it uses probability correction factors in semantic tags to adjust the prior probabilities of these hypotheses, generating a discrete set of behavioral hypotheses with probabilities.

[0190] The parallel simulation engine is a simulation kernel manager that can run on GPUs or dedicated hardware. The parallel simulation engine is the computational core of the system, responsible for performing large-scale simulations. The engine first obtains the latest digital twin scene snapshot and sensor attenuation parameters from the DDTCM; further, the engine forms a large simulation task tree by combining all combinations of the vehicle's N candidate actions with the K behavioral hypotheses of M targets and distributes them to multiple computing cores of the HPC. On each computing core, the simulator advances at time steps far exceeding real-time (Δt=0.1s), within each step: The vehicle updates its status based on its candidate action sequence.

[0191] Other targets update their states based on assumptions about their behavior.

[0192] Use a virtual sensor model with attenuation to simulate the vehicle's perception process.

[0193] Record key simulation data, such as vehicle spacing and relative speed.

[0194] The multi-objective evaluator consists of a cost function calculation unit and a risk aggregator. It is used to quantify and score the results of each simulation after the simulation ends.

[0195] For each candidate action of the vehicle, there is a set of simulation results and costs that are combined with all the target behavior assumptions. The aggregator needs to calculate the overall expected cost of the action.

[0196] The MHSPM operates in a high-frequency decision cycle, typically aligned with the vehicle's control frequency (10Hz). Its flowchart is shown below. Figure 10 As shown, it includes: Step 1: MHSPM subscribes to and receives the relevant topics of the attenuation parameters, digital twin scene snapshots, and contextual semantic tags. When a new round of input is received, the decision cycle begins.

[0197] Step 2: Receive semantic tags from SCRM, and generate a set of candidate actions {A_i} for this vehicle based on the current vehicle state and task; The probabilistic behavior hypothesis generator generates a set of probabilistic behavior hypotheses {H_j, P(H_j)} for key targets in the scene based on the latest semantic labels.

[0198] Step 3: The parallel simulation engine loads the latest digital twin scene and sensor attenuation parameters; Construct a simulation task tree and distribute the (A_i, H_j) combinations to the various computing cores of the HPC.

[0199] Step 4: All computing cores start simultaneously and rapidly simulate their assigned future scenarios until the preset simulation duration is reached.

[0200] Step 5: After each simulation task is completed, the multi-objective evaluator immediately calculates its corresponding cost Cost(A_i, H_j).

[0201] Step 6: After all simulation tasks are completed, the evaluator calculates the expected total cost for each candidate action A_i of the vehicle.

[0202] Step 7: MHSPM packages the evaluation matrix containing all candidate actions and their corresponding expected costs into a standard format structured context semantic tag, and publishes the context semantic tag to the data topic of the internal data bus for the OBDM module to make the final decision.

[0203] This process ensures that the system has fully considered various future possibilities and their probabilities in the current context before making a decision, thus enabling it to make intelligent decisions that are both safe and efficient.

[0204] In this embodiment, the Optimal Behavior Decision Module (OBDM) further selects and issues a final, optimal driving action based on the evaluation matrix provided by MHSPM and according to preset, explicit decision-making criteria. This module directly reflects the vehicle's driving characteristics and ethical orientation in extreme situations. OBDM converges complex, probabilistic evaluation results into a single, deterministic physical world control command, completing the crucial execution from thought to action.

[0205] OBDM is a lightweight but highly logical module, and its structure diagram is as follows: Figure 11 As shown, it consists of three sub-units: a user preference configuration unit, a decision criterion selection engine, and an instruction generation and smoothing unit.

[0206] The user preference configuration unit consists of a parameter set storing the user's driving style preferences, which can be set through the in-vehicle human-machine interface. The parameter set stores a set of configurable driving modes and custom cost function weights. In this embodiment, the driving modes include three modes: conservative, balanced, and sport. The custom cost function weights include three weights: safety, comfort, and fuel efficiency. These parameters directly affect the selection of subsequent decision criteria or the weighting method of the cost function. For example, selecting sport mode may make the system more inclined to choose actions with lower fuel efficiency costs, even if the comfort cost is slightly higher. The user preference configuration unit provides an API for the HMI to call, allowing the driver to switch between safety, comfort, and fuel efficiency modes.

[0207] The decision criterion selection engine incorporates several classic decision theory models. Based on the vehicle's current safety status and user preferences, it selects the most suitable decision criterion from its model library. These models include the Minimax criterion, expected utility maximization, regret minimization, and the influence of user preferences. Minimax (Minimize Maximum Loss) Criterion: This is the ultimate criterion prioritizing safety. Its logic is to select the option that minimizes our loss among all possible worst-case scenarios. It doesn't care about the average performance of an action, only its bottom line in the worst case. The calculation involves finding the highest cost value max(Cost(A_i,H_j)) across all simulations for each candidate action A_i, and then selecting the action A_i with the smallest max(Cost).

[0208] Maximize Expected Utility: This is a commonly used criterion in equilibrium. Its logic is to select the option that has the highest overall expected return among all possibilities. The calculation idea is to directly select the action A_i with the lowest ExpectedCost(A_i) calculated by MHSPM, that is, the action with the highest expected utility.

[0209] Minimize Regret: A more complex criterion that aims to select the action that is least likely to be regretted afterward. Its selection logic defaults to the expected utility maximization criterion. When the semantic label of SCRM contains high risk or when any simulation shows extremely high safety costs, the engine automatically switches to the Minimax criterion to enforce the most conservative and safest decision.

[0210] User preference impact: Users who choose Conservative mode will increase the sensitivity of triggering the Minimax criterion.

[0211] The instruction generation and smoothing unit consists of an action-to-control instruction converter and an instruction smoothing filter. Its function is to transform selected, discrete, and abstract candidate actions, such as "gentle deceleration and slight right turn," into continuous and smooth control instructions that can be understood by the vehicle's underlying actuators. The selected candidate action is essentially a short-term target trajectory or state sequence. This unit uses the vehicle's inverse dynamics model to calculate the longitudinal target acceleration and lateral target curvature required to achieve this target trajectory.

[0212] To prevent sudden changes in control commands from causing vehicle vibration or passenger discomfort, the output control command sequence is smoothed by a low-pass filter or PID controller. Simultaneously, the unit checks whether the commands are within the vehicle's physical capabilities, such as maximum acceleration and maximum steering angular velocity, and applies necessary limiting. The final commands are then output to the Vehicle Motion Control (VMC) module.

[0213] The OBDM process follows MHSPM, constituting the final stage of the decision-making cycle. Its flowchart is as follows: Figure 12 As shown, it includes: Step 1: OBDM subscribes to the decision evaluation matrix in the data topic published by MHSPM to the internal data bus and obtains the current settings of the user preference configuration unit.

[0214] Step 2: Decision Criteria Selection. The highest risk value and semantic label in the engine analysis and evaluation matrix are combined with user preferences to select the criteria to be used in this decision cycle.

[0215] Step 3: The module calculates the evaluation matrix based on the selected criteria; Step 4: The instruction generation and smoothing unit receives the selected optimal action ID and finds its corresponding target trajectory from the candidate action list.

[0216] Step 5: Calculate the original control commands required to achieve the trajectory using the inverse dynamics model, and then filter, smooth, and limit them.

[0217] Step 6: Issue the final, smooth control command to its corresponding topic, which is received by the vehicle's VMC module and drives the physical actuators, namely the throttle, brake, and steering systems, to complete the action.

[0218] This process ensures that the vehicle's final behavior is not only based on future considerations, but also conforms to preset safety strategies and user expectations.

[0219] In this embodiment, the Closed-Loop Optimization Module (CLOM) ​​serves as the reflection and evolution engine. Its core mission is to establish an intelligent feedback loop from physical world results to the digital twin model, enabling continuous self-optimization of the system. Unlike traditional offline training methods that rely on massive amounts of undifferentiated data, CLOM monitors inconsistencies between predictions and reality, capturing high-value data samples that best expose the current model's flaws. These samples are then used to specifically correct and optimize the system's core models, such as behavior prediction models and sensor attenuation models, thereby driving the iteration and evolution of the system's cognitive capabilities in the most efficient way.

[0220] CLOM is a background, event-driven module. It begins working after a physical action occurs and consists of three sub-units: a prediction-reality inconsistency monitoring unit, a high-value sample screening and labeling unit, and a model optimization task distribution unit. Its structure diagram is shown below. Figure 13 As shown.

[0221] The predictive-reality inconsistency monitoring unit consists of a data aligner, a state comparator, and a set of inconsistency metrics, and is used to output a quantified inconsistency score within a time window.

[0222] The data aligner caches the predicted data generated by the MHSPM before decision-making, as well as the vehicle's final selected action by the OBDM. After the vehicle performs the action, it continuously receives real-world data from the SDFM, i.e., the actual observed trajectory of the key target. The data aligner is responsible for accurately aligning these two sets of data in time and space. For each target under key monitoring, the comparator compares its predicted state with the actual state at each time step.

[0223] The inconsistency metric library provides various methods for calculating inconsistency, including: • Lateral / Longitudinal Error: Calculates the lateral and longitudinal distances between the predicted and actual locations in the Frenet coordinate system.

[0224] Mahalanobis Distance: A more advanced metric that takes into account the uncertainty of the prediction (covariance matrix). The Mahalanobis distance will be large if the actual location falls outside the high-confidence region of the predicted probability distribution.

[0225] • Trajectory Similarity: Such as Dynamic Time Warping (DTW) or Fréchet Distance, used to evaluate the shape similarity between the entire predicted trajectory and the actual trajectory.

[0226] The high-value sample screening and labeling unit consists of a dynamic threshold comparator and a data packaging and labeling engine. The dynamic threshold comparator determines whether the inconsistency score is large enough to be considered a significant cognitive bias. The dynamic threshold is not fixed but context-dependent. For example, in a simple scenario like following another car on a highway, the dynamic threshold will be small, and slight deviations will be labeled; while at a chaotic intersection, the dynamic threshold will be larger to tolerate the inherently higher uncertainty. The dynamic threshold can be adjusted by the semantic labels output by the SCRM.

[0227] The data packaging and annotation engine is activated when the inconsistency score exceeds a dynamic threshold. It then backtracks and collects all data related to this cognitive bias event, forming a complete case file. This case file includes: Input layer: Complete raw SDFM data fragments.

[0228] Twin layer: Scene snapshots and sensor attenuation parameters constructed by DDTCM.

[0229] Cognitive layer: Semantic tags generated by SCRM.

[0230] Prediction layer: The complete set of behavioral hypotheses and probabilities generated by MHSPM.

[0231] Decision-making level: OBDM's selected actions and final evaluation matrix.

[0232] Results layer: The actual trajectories of all objects in the physical world.

[0233] The data packaging and labeling engine adds metadata tags to this case file, such as: [Event ID, timestamp, inconsistency score, bias type: lateral prediction failure, triggering scenario: rainy intersection].

[0234] The model optimization task distribution unit consists of a sample storage manager and a model training / fine-tuning task scheduler. The sample storage manager stores packaged high-value samples in an onboard "error log" database or uploads them to a cloud-based sample library via a T-Box. The task scheduler determines how to utilize a sample based on its annotation information. If the bias is small and computational resources allow, minor parameter updates can be made directly on-vehicle for some lightweight models, such as the lookup table in a sensor attenuation model. In most cases, samples are uploaded to the cloud. The cloud-based MLOps platform automatically adds these high-value samples to the training set for the next round of retraining or reinforcement learning of heavyweight models. If predictions consistently fail in a certain scenario, the system can generate a report suggesting that developers add new entity or relation rules to the knowledge graph.

[0235] The CLOM execution process is triggered asynchronously after the decision is executed, and its flowchart is as follows: Figure 14 As shown, it includes: Step 1: While the OBDM issues control commands, the CLOM inconsistency monitoring unit begins to cache the prediction results and related context of the MHSPM.

[0236] Step 2: Continuous monitoring and data alignment: Over the next few seconds, the monitoring unit continuously receives and aligns real-world observation data from SDFM.

[0237] Step 3: At the end of the monitoring time window, the monitoring unit calls its metric function library to calculate the inconsistency score between the prediction and reality.

[0238] Step 4: The high-value sample screening unit compares the calculated inconsistency score with the dynamic threshold in the current context.

[0239] Step 5: If the inconsistency score is less than or equal to the dynamic threshold, it means that the prediction is within an acceptable range. This loop ends and the cache is cleared.

[0240] In response to an inconsistency score exceeding a dynamic threshold, the packaging and labeling engine is triggered to backtrack and collect data from the entire chain, generating a high-value sample with detailed metadata.

[0241] Step 6: The model optimization task distribution unit stores the generated high-value samples in the database or uploads them to the cloud; based on the sample's metadata, the scheduler decides whether to trigger an online fine-tuning task or add it to the offline training pool in the cloud.

[0242] This process creates an efficient, intelligent, and self-driven evolutionary mechanism that ensures the system can learn from every unexpected event and continuously improve its understanding and predictive capabilities about the world.

[0243] The six modules mentioned above do not constitute a simple linear pipeline, but rather a tightly coupled, iterative cognitive architecture. Its core operational logic can be summarized as three nested stages: generation, deduction, and evaluation. Through this cognitive cycle, the entire system elevates the vehicle from a passive reactor to an active thinker, providing quantifiable, real-time-based decision-making support for autonomous driving systems and enhancing the system's interpretability and credibility.

[0244] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for autonomous driving situational perception and decision-making, characterized in that, include: Time synchronization, spatial registration, and semantic fusion are performed on multi-source heterogeneous data from vehicle sensors to generate standardized scene data. Based on the standardized scene data and prior environmental constraints, an adaptive digital twin environment is generated; Graph reasoning is performed on the standardized scene data to generate contextual semantic tags; Based on the adaptive digital twin environment and contextual semantic tags, a set of probabilistic behavioral hypotheses and a set of candidate actions corresponding to key dynamic targets are generated; Parallel simulations are performed by combining a set of probabilistic behavioral assumptions and a set of candidate actions to generate a decision evaluation matrix; Based on the decision evaluation matrix, combined with user driving preferences and preset decision criteria, the optimal driving action is selected and issued; Generate a decision evaluation matrix, including: Based on the current vehicle state and task, sampling is performed in the two-dimensional space of acceleration and curvature to generate a set of candidate action sequences; Based on the preset dynamic target basic behavior model library and predefined behavior assumptions, a set of basic behavior assumptions are generated for the key dynamic targets in the digital twin scene snapshot; The prior probability of the basic behavioral hypothesis is adjusted based on the probability correction factor in the contextual semantic label. The prior probability is combined with the basic behavioral hypothesis to generate a probabilistic behavioral hypothesis set; Load the latest digital twin scene and decay parameters, and construct a simulation task tree by combining the candidate action sequence with all elements in the probabilistic behavior hypothesis set, and distribute it to each computing core of HPC; The computing core performs parallel simulations and extrapolates future scenarios according to a preset time step. The computing core records key simulation data during the simulation process, including vehicle spacing, relative speed between the vehicle and the key dynamic target, and virtual sensor perception results. Calculate the cost of a single task in the simulation task tree based on the key simulation data; The expected total cost of each candidate action is calculated by weighting the single-task cost with the corresponding probability of the probabilistic behavior hypothesis set. The candidate action sequences and their corresponding expected total costs are combined to form a decision evaluation matrix.

2. The autonomous driving situational perception and decision-making method according to claim 1, characterized in that, Generate standardized scene data, including: A unified timestamp is assigned to the received multi-source heterogeneous data; The timestamp of the data frame is read, and the vehicle attitude is obtained by interpolation based on the inertial measurement unit data; By combining preset sensor external parameters, multi-source heterogeneous data are unified into the vehicle coordinate system through rigid body transformation; The transformed multi-source heterogeneous data is fused to generate a fused object; The fusion object is subjected to validity detection and anomaly removal, and the cleaned fusion object is encapsulated into a data structure with unified fields.

3. The autonomous driving situational perception and decision-making method according to claim 1, characterized in that, Generate an adaptive digital twin environment, including: Subscribe to and receive standardized scenario data published to different data themes, including: vehicle status theme provides vehicle status vectors, fusion object theme provides a dynamic target list, and map update theme provides road and environmental information; The vehicle status, spatial location and movement status of traffic participants are updated based on the standardized scenario data, and an environmental factor vector is constructed. Based on the environmental factor vector, the set of performance degradation coefficients of the environmental factor vector at the current moment is calculated using an adapted performance degradation mathematical model. A digital twin scene snapshot is generated based on the vehicle state, spatial location, and motion state, and an adaptive digital twin environment is generated by combining the performance degradation coefficient set.

4. The autonomous driving situation perception and decision-making method according to claim 3, characterized in that, The performance degradation mathematical model includes: LiDAR attenuation model adapted to rainfall: in, This represents the intensity of the LiDAR signal after a propagation distance d. Indicates the initial signal strength. Indicates the extinction coefficient; Camera attenuation model adapted to lighting conditions and sensor operating status: Where K represents the camera performance degradation coefficient. This represents the illumination mapping function that describes the relationship between illumination intensity and image signal-to-noise ratio. These represent the weighting coefficients of the illumination mapping function. This represents the lens dirt confidence score output by the visual algorithm used to detect dirt on camera lenses. The weighting coefficient represents the confidence level of lens dirtiness.

5. The autonomous driving situational perception and decision-making method according to claim 1, characterized in that, Perform graph inference on the standardized scene data to generate contextual semantic labels, including: The standardized scenario data is mapped to pre-built vehicle knowledge graph entities; By performing graph traversal and rule matching on the vehicle knowledge graph, the risk path from the matching entity to the risk entity is determined; Based on the risk path, confidence scores of results pointing to the same risk entity are fused to generate a probability correction factor corresponding to the risk entity. The risk path and probability correction factor corresponding to the risk entity are packaged into a contextual semantic tag.

6. The autonomous driving context perception and decision-making method according to claim 1, characterized in that, Calculating the cost of a single task in the simulation task tree based on the key simulation data includes: Calculate the safety cost, efficiency cost, comfort cost, compliance cost, and social acceptance cost based on the key simulation data. The single-task cost is obtained by weighting and summing the security cost, efficiency cost, comfort cost, compliance cost, and social acceptance cost according to a cost function. Where J represents the cost per task. The coefficients representing the weights of the cost function. Indicates cost components.

7. The autonomous driving situational perception and decision-making method according to claim 1, characterized in that, Select and issue the optimal driving action, including: Based on the highest risk value in the decision evaluation matrix and its corresponding contextual semantic label, and combined with user preference configuration parameters, the decision criteria for this decision cycle are selected. The optimal candidate action is selected from the decision evaluation matrix according to the decision criteria.

8. The autonomous driving situational perception and decision-making method according to claim 6, characterized in that, Based on the decision evaluation matrix, and combining user driving preferences and preset decision criteria, after selecting and issuing the optimal driving action, the process further includes: The driving actions are converted into control commands that the vehicle can execute; Based on the execution results of the control commands, the cost function weights and decision criteria are optimized and adaptively adjusted in a closed loop.

9. The autonomous driving situational perception and decision-making method according to claim 8, characterized in that, Closed-loop optimization and adaptive adjustment of the cost function weights and decision criteria are performed, including: The standardized scene data is used as real-world observation data, and the decision evaluation matrix, probabilistic behavior hypothesis set, and control commands are used as prediction data. After aligning the real-world observation data and the predicted data in time and space, the inconsistency score between the real-world observation data and the predicted data is calculated according to a preset metric function library. The inconsistency score is compared with a dynamic threshold corresponding to the current scenario; In response to the inconsistency score exceeding a dynamic threshold, backtrack and collect case files containing data related to the current cognitive bias event; Fine-tune the lightweight model based on the metadata of the case file.

Images