Automatic driving risk deduction method and system based on world model and human factor coupling

Abstract

The invention provides an automatic driving risk deduction method and system based on world model and human factor coupling, and the method comprises the steps: obtaining multi-modal data which comprises the state of a passenger, the motion of a vehicle, and environment interaction data; generating long-tail scene data through world model processing according to the multi-modal data, and pre-training the world model and a human factor coupling risk assessment model through the long-tail scene data; generating synthetic training data according to the dynamic passenger state through a causal model; acquiring real-time data, taking the synthesized training data as priori knowledge, and performing risk assessment on the real-time data through a pre-trained human factor coupling risk assessment model to obtain risk probability distribution; the operation sequence is deduced through the pre-trained world model to obtain future scene evolution data, the overall loss is obtained according to the future scene evolution data, the automatic driving optimal control sequence data is generated according to the overall loss, and the accuracy and generalization of automatic driving risk deduction can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving safety, in particular to an automatic driving risk deduction method and system based on world model and human factor coupling. BACKGROUND

[0002] The existing automatic driving risk assessment technology mainly relies on static environment perception and preset rules, and has two major bottlenecks: (1) Environment deduction distortion: traditional methods are difficult to generate physically consistent future multi-modal traffic scenes, resulting in risk prediction deviation. (2) Lack of human factor modeling: ignoring the dynamic influence of passenger state on takeover behavior, poor generalization in data scarcity. Therefore, there is an urgent need for a dynamic risk deduction technology that integrates environment generation and human factor reasoning. In addition, the existing deep learning method usually needs a large amount of data for offline training in automatic driving, and relies on high-performance hardware for real-time reasoning. This leads to high data collection cost, difficulty in generalization to long-tail scenes, and high requirements for on-board computing power, hindering the wide application of the technology. SUMMARY

[0003] In view of this, the present application provides an automatic driving risk deduction method and system based on world model and human factor coupling to solve the problems existing in the prior art.

[0004] To achieve the above purpose, the present application provides an automatic driving risk deduction method and system based on world model and human factor coupling, comprising: Obtaining multi-modal data, wherein the multi-modal data includes passenger state, ego motion and environment interaction data; Building the world model and the human factor coupling risk assessment model, generating long-tail scene data through the world model processing according to the multi-modal data, and pre-training the world model and the human factor coupling risk assessment model through the long-tail scene data; Building a causal model according to the pre-trained world model and obtaining dynamic passenger state, generating synthetic training data through the causal model according to the dynamic passenger state; Obtaining real-time data, taking the synthetic training data as prior knowledge, and performing risk assessment on the real-time data through the pre-trained human factor coupling risk assessment model to obtain a risk probability distribution; Building an operation sequence, deducing the operation sequence through the pre-trained world model to obtain future scene evolution data, obtaining an overall loss according to the future scene evolution data, and generating automatic driving optimal control sequence data to control the automatic driving according to the overall loss.

[0005] Optionally, the process of obtaining multi-modal data comprises: Collect multi-dimensional data, wherein the multi-dimensional data includes a dynamic state vector, a takeover trigger timestamp, a self-vehicle motion perception data, and a dynamic target trajectory set; Perform time-space synchronization and interpolation compensation on the multi-dimensional data; and perform uniform space reference mapping on the compensated data through a differentiable rendering engine to obtain multi-modal data.

[0006] Optionally, the process of obtaining the long-tail scene data includes: Extract environment features from the multi-modal data, and generate a road topology reconstruction model through The generator reconstructs the road topology based on the environment features to obtain road grid data. Extract a human factor heat map and a dynamic target trajectory set from the multi-modal data, and generate dynamic agent trajectory prediction data through a trajectory diffusion model based on the human factor heat map and the dynamic target trajectory set. Compress the history tensor of the multi-modal data through 3D convolution to obtain latent variables, construct physical constraints, and inject the physical constraints into the latent variables through a Lie group algebra layer to obtain scene latent variable data. According to the conditional tensor, the scene latent variable data is gradually denoised through a probabilistic generation model to obtain scene latent variable data at different time steps, wherein the conditional tensor includes road topology data, dynamic agent trajectory prediction data, and a human factor heat map, and the road topology data is generated based on the road grid data. Decode the scene latent variable data at different time steps to obtain long-tail scene data.

[0007] Optionally, in the world model, a spatial gating mask is generated based on the road topology data, and a cross-modal feature alignment is performed on the spatial gating mask through a gated spatio-temporal attention mechanism to perform spatial masking.

[0008] Optionally, the process of generating the synthetic training data includes: Intervene in sampling of the dynamic occupant state to obtain synthetic training data ; wherein:

[0009] wherein, is a joint state of the vehicle and the environment. represents intervention on a human factor real-time vector , represents a distraction level, represents an environment feature, represents a vehicle operation, P represents a probability distribution, represents a risk result.

[0010] Optionally, the process of obtaining the risk probability distribution includes:​ According to the synthetic training data generation history data set, the history data set is integrated with the real-time data through a bidirectional attention mechanism to obtain an extended sample space; According to the extended sample space, position encoding is performed and encoding is performed through a Transformer encoder layer to obtain a query set final embedding; The query set final embedding is decoded through a multilayer perception machine to obtain a future risk probability distribution.

[0011] Optionally, in the encoding process of the Transformer encoder layer, attention weight calculation of the feature dimension of the same sample after position encoding is included, and then attention weight calculation of the same feature after position encoding of the sample is performed.

[0012] Optionally, the overall loss includes a risk-sensitive loss term and a trajectory and operation constraint loss term , wherein:

[0013] wherein, denotes the risk probability when performing an operation sequence at the th time step, denotes the Bayesian uncertainty at the time step; is a preset safety uncertainty threshold, , and are weighting coefficients of each term;

[0014] wherein, and are the state vector and control command of the vehicle at time. is a reference state. is a smoothness penalty term of the control command. is the weight of each term.

[0015] On the other hand, the present application provides an automatic driving risk deduction system based on a world model and human factors, which is used to perform the above method.

[0016] Compared with the prior art, the present application has the beneficial effects that, The present application first solves the inherent defect of the mutual separation of environmental deduction and human factor variables in the traditional risk assessment method. By providing a dynamic deduction based on a human factor corrected physically consistent world model scene, the present application innovatively integrates the occupant distraction state as the core generation condition into the physical deduction architecture. This makes the deduction of high-fidelity traffic scenes not only strictly follow Newton's laws of mechanics, but also accurately simulate the chain risk effect caused by human factor failure. For example, when detecting that the driver is seriously distracted, the trajectory diffusion model will strengthen the approaching speed and path conflict probability of the obstacle, and convert the takeover preparation time into a collision time constraint in the physical engine, thereby fundamentally solving the key bottleneck of the disconnection between environment and human factor variables.

[0017] Secondly, the present application constructs a deeply coupled human factor and environmental causal decision-making framework, which significantly improves the generalization ability of the model in data-scarce and long-tail scenarios. In view of the defect of decoupling occupant state and risk decision-making in the prior art, the present application innovatively models the dynamic causal chain. It uses a hidden Markov model to quantify the evolution process of distraction behavior, and injects the occupant state vector into the intervention node of the structural causal model in real time. This not only establishes a quantitative mapping chain of "distraction level→takeover delay→collision probability", but also generates a large amount of synthetic data covering extreme human factor conditions, so that the risk assessment model can still effectively generalize when real data is scarce.

[0018] Finally, the present application establishes an efficient non-autoregressive risk situation joint deduction mechanism, which avoids the error propagation of traditional serial reasoning, and significantly improves the real-time decision-making of complex human factor-environment coupled scenarios. More importantly, the present application takes the deduced risk results as a core component of the cost function of the vehicle decision planning module, so that the planning algorithm can select the path and control command that can minimize future risks. This makes the entire system truly realize a complete closed loop from perception-deduction-decision-control, and fundamentally guarantees driving safety. BRIEF DESCRIPTION OF DRAWINGS

[0019] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included only serving the purpose of illustrating the preferred embodiments and are not considered as limiting the present application. In the drawings: Figure 1 The system overall architecture and processing flowchart in the embodiment of the present application; Figure 2 The data priori pre-training flowchart based on the causal model in the embodiment of the present application; Figure 3 The parallelized prediction method flowchart in the embodiment of the present application; Figure 4 The visual interface schematic diagram in the embodiment of the present application. DETAILED DESCRIPTION

[0020] Exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art. It should be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0021] The present disclosure aims to solve the above challenges and provide an automatic driving risk situation reasoning method that balances accuracy and computational efficiency by fusing world models and human factors. The method uses a two-stage architecture based on cloud-end collaboration to decouple computationally intensive tasks and real-time inference tasks, thereby reducing the requirement for on-board computing power and improving the robustness of the system in data-scarce and long-tail scenarios.

[0022] The present disclosure includes four core features: (1) World model cloud scenario generation and training. Utilizing the powerful computing resources of the cloud, the environment state is encoded as a three-dimensional spatiotemporal tensor based on a physically differentiable rendering engine. A denoising model is used to generate future multi-modal scenarios, and a forward simulation of the latent space is used to generate multiple possible world states, including dynamic agent trajectories, road topology, and physical constraint relationships. These large-scale, diverse synthetic long-tail scenario data are used to pretrain the core world model and risk assessment module, significantly improving its generalization ability. (2) Modeling of the causal chain of human-environment coupling. A dynamic causal graph is constructed that couples the state of the occupant and the environment. The evolution of distraction behavior is quantified using a hidden Markov model, and the takeover response probability is adjusted in real time through feedback from the occupant state signal. This module is trained on the cloud along with the world model to ensure the sensitivity of risk assessment to dynamic changes in human factors. (3) Robust training based on structural causal models. The data generation framework of the structural causal model is expanded to include human intervention nodes, generating an "environment-vehicle-human" coupled dataset that covers long-tail scenarios. (4) Non-autoregressive joint risk situation reasoning and small-sample adaptation. On the vehicle side, an optimized lightweight model is deployed for real-time risk reasoning. This model design has a time-series-feature dual attention mechanism that can quickly output the risk probability distribution, Bayesian confidence interval, and high-risk area heat map for multiple steps in the future in a single forward propagation. Most importantly, the present disclosure uses a probabilistic fast network to achieve small-sample learning on the vehicle side. When the vehicle encounters a new, untrained long-tail scenario, it can quickly calibrate and adapt to the new, small amount of data collected in real time without the need for a large amount of data, providing reliable risk assessment for the vehicle and effectively addressing the poor generalization of traditional models in data-scarce scenarios.

[0023] Based on the above technical needs, the present application provides an automatic driving risk deduction method and system based on world model and human factor coupling, referring to Figure 1 , the specific implementation steps are as follows: Step 1, real-time synchronous acquisition of multi-modal data: through the cabin sensor network and roadside equipment, the passenger state, vehicle motion and environmental interaction data are synchronously acquired.

[0024] Step 2, cloud large-scale scene generation and offline training. On the cloud server cluster, a large number of long-tail scene data are generated through the world model, and the pre-training of the large world model and human factor coupling risk assessment model is carried out using these data.

[0025] Step 3, human-environment coupling risk modeling: the passenger state is injected into the causal model framework as a dynamic variable to generate synthetic training data with causal relationship.

[0026] Step 4, non-autoregressive risk situation joint output and online calibration. The vehicle-mounted model receives real-time data and quickly outputs the risk probability and confidence interval of future multiple steps through the probability inference engine. When encountering long-tail scenes with data scarcity, online fine-tuning is performed according to the small amount of real-time collected data to ensure the real-time and accuracy of risk assessment.

[0027] Step 5, risk situation driven decision planning and control. The non-autoregressive risk deduction result is used as the core input of the vehicle decision planning module to realize the risk minimization of automatic driving control.

[0028] Further, step 1, real-time synchronous acquisition of multi-modal data, a cross-modal spatio-temporal alignment "human-vehicle-environment" holographic perception system is constructed, visual behavior recognition and takeover timing modeling are fused, physical space and behavior space are unified coded, and physical consistent input is provided for world model and human factor coupling modeling. By solving the spatio-temporal synchronization of automatic driving passenger state signal, vehicle motion state and environmental dynamic target, the unified coding of heterogeneous data flow in three-dimensional spatio-temporal tensor is realized. A hierarchical sensor network and edge-cloud collaborative computing framework is adopted, and the specific steps are as follows: First, deploy a four-layer heterogeneous sensor array to realize multi-dimensional data capture: First, the driver state information is synchronously captured by the in-vehicle camera, and the behavior recognition model outputs the action state vector .

[0029] At the same time, when the automatic driving system issues a takeover request, the TOR trigger timestamp is recorded Second, the vehicle motion perception layer outputs 6-DoF pose , and vehicle bus CANFD bus data.

[0030] Then, the environment interaction perception layer fuses the roadside laser radar point cloud and V2X communication to obtain the laser radar point cloud , and generates a dynamic target trajectory set .

[0031] Secondly, the space-time synchronization, aiming at the time sequence alignment problem of millisecond-level visual signals and sub-second-level environmental data, adopts a precise clock protocol PTPv2 to build a global time reference. The Lie group interpolation compensation is applied to the asynchronously collected laser radar point cloud:

[0032] wherein, is the compensated point cloud pose (timestamp ), and are adjacent pose timestamps, is the Lie group exponential mapping, is the Lie group logarithm mapping, is a time weight factor.

[0033] The motion interpolation between adjacent timestamps is performed using the ego-vehicle pose matrix to eliminate the point cloud displacement error caused by vehicle bumping. The takeover request timestamp is aligned to the unified clock source through the time axis mapping function, ensuring the time sequence consistency of the behavior event and the environmental state.

[0034] Thirdly, the cross-modal feature fusion, through the differentiable rendering engine, maps the heterogeneous data to a unified space reference.

[0035] Firstly, the bird's eye view semantic grid is encoded into a 64-dimensional feature map , retaining the lane line topology and obstacle spatial distribution. Secondly, the trajectory set is converted into an instance grid with a speed vector , each grid cell encoding the motion state of the nearest agent, as well as the human factor data behavior vector and the takeover timing parameter.

[0036] Further, the gated feature pyramid is adopted to fuse multi-source information, introducing a learnable weight matrix to dynamically adjust the contribution degree of environmental, dynamic target, and human factor features. Through spatial broadcast multiplication, pixel-level feature enhancement is realized, and finally a spatio-temporal tensor is synthesized through three-dimensional convolution. This tensor covers a 100x100 meter physical space with a 1 second history window length, and 128 feature channels explicitly separate environmental semantics, dynamic targets, human factors, and ego-motion, forming a human-environment coupled causal traceable representation.

[0037] The process takes layered sensor network as the core, synchronously collects the cabin passenger state, vehicle motion parameters and environmental interaction information. In view of the spatio-temporal alignment challenge of millisecond biological signal and sub-second environmental data, a triple fusion mechanism is innovatively adopted: first, time synchronization is realized based on PTPv2 protocol and FPGA hardware trigger; second, Li group interpolation compensation technology is used to eliminate the motion scale difference; finally, through the differentiable rendering engine, the driver state index is projected into the spatio-temporal heat map in the virtual cockpit coordinate system, and the environmental semantics, dynamic target trajectory and other elements are jointly coded into a three-dimensional spatio-temporal tensor.

[0038] Further, step 2 world model dynamic scene generation, this step converts the three-dimensional spatio-temporal tensor generated in step 1 into a future multi-step traffic scene through an innovative diffusion-physical hybrid deduction architecture. The scene is guaranteed to be realistic through physical constraints. The specific steps are as follows: S2.1 Structured traffic element analysis S2.2 Scene diffusion deduction with physical constraints Further, S1.1 separates road structure, dynamic agent trajectory and human factor state from the input tensor, and constructs a physically interpretable generation condition. The specific steps are as follows: First, based on The generator decodes the road structure from the environmental features. Road topology reconstruction is realized by the generator from the environmental features Decoding road structure:

[0039] Wherein, is the reconstructed road grid map, is the input environmental feature, is the HDMap generator parameter, is the three-dimensional transpose convolution. This module adopts a five-layer three-dimensional transpose convolution network, and the input is the 32-dimensional environmental semantic feature generated in step 1. The original spatial resolution is restored through feature upsampling. In the output 8-channel road grid map, each channel encodes. The reconstruction process ensures lane continuity through a topology constraint loss function, and a 10 times penalty coefficient is applied to the lane line fracture area to eliminate the road structure fracture problem in traditional methods.

[0040] Second, trajectory generation with human factor correction. Dynamic agent trajectory prediction is based on human factor heat map Correct the target behavior, and adopt a trajectory diffusion model:

[0041] Wherein, is the 3D bounding box sequence, is the dynamic target trajectory set, The operator representing the concatenation of trajectory and human characteristics is used in a multilayer perceptron. Humans are caused by heat maps. The parameters for the trajectory diffusion model are used to ultimately output a sequence of 3D bounding boxes. .

[0042] Furthermore, S1.2 constructs an autonomous driving diffusion model based on physical constraint diffusion deduction, strictly adhering to Newtonian mechanics constraints when generating future scenarios. The specific steps are as follows: The first step is spatiotemporal conditional encoding, using the history tensor. Compressed into latent variables using 3D CNN Injected through a Lie group algebraic layers via physical constraints:

[0043] in Forced speed acceleration Satisfying rigid body kinematics, For physical constraint strength, This represents the latent variable at time t. Represents the latent variables of a scenario that meet physical conditions.

[0044] The second step involves using a diverse scenario extrapolation process based on a probabilistic generation model within the noise latent space. Perform multi-step noise reduction:

[0045] in, This represents the backdiffusion probability. Indicates a normal distribution. Conditional mean prediction For diffusion time step, Let C represent the variance of the diffusion process, and C represent the conditional tensor. Specifically, the conditional tensor... It integrates road, agent, and human factors status.

[0046] In addition, spatial gating masks Road topology The generated spatial mask suppresses the generation of non-passable regions. Specifically, cross-modal features are aligned using a gated spatiotemporal attention mechanism:

[0047] in, These are the query matrix, key matrix, and value matrix, respectively. Scaling factor This indicates element-wise multiplication.

[0048] Further, the physical micro-rendering is performed, a dynamic regular term is added in a loss function, and a differentiable collision detection is used to ensure that the generated scene is physically feasible.

[0049] This step realizes the physically consistent generation of dynamic scenes through a diffusion-physical hybrid inference architecture. Firstly, the low-dimensional semantic features are decoded into a structured road network by the HDMap generator, and a topological continuity loss is introduced to eliminate lane breaks. Then, a trajectory diffusion model is embedded into a Newtonian mechanics regular term, and a differentiable collision detection and a tire force model are used to ensure the kinematic rationality of the vehicle. Furthermore, a takeover preparation time weighting mechanism is introduced through the dynamic coupling of human factors and the scene, so that the driver's distraction state directly regulates the risk level of the generated scene, and the human factor error scene is accurately simulated.

[0050] Further, as Figure 2 Step 3: Human-Environment Coupling Risk Modeling. The occupant state is injected into the causal model framework as a dynamic variable to generate synthetic training data with causal relationships. The present invention proposes a human-enhanced dynamic Bayesian causal modeling to solve the inherent problem of traditional risk inference models, i.e., the mutual isolation of occupant state and environmental dynamic evolution.

[0051] Specifically, we inject the real-time human factor vector stream into the structured causal graph framework as an intervention variable. This framework models the dynamic causal chain between human factor states (such as distraction level, reaction delay) and environmental elements (such as obstacle speed, path conflict probability), achieving accurate generation and inference of future risk scenarios. This method fundamentally guarantees the physical consistency of the inferred scene and the authenticity of human factor behavior.

[0052] This method defines a causal intervention operation in the human-enhanced structured causal model to perform intervention sampling and generate a large-scale, causally related counterfactual synthetic dataset . These data focus on covering long-tail and extreme working conditions caused by human factor errors, significantly improving the generalization ability of risk assessment models in real data scarce scenarios.

[0053] Human factor intervention data generation generates a dataset containing risk results by intervention sampling on the human factor vector .

[0054]

[0055] where, is the joint state of the vehicle and the environment. represents the real-time vector of human factors Interventions (e.g., setting a distraction level ). represent environmental features. represent vehicle operations, represent a risk mapping function of the state vector.

[0056] Further, as Figure 3 shown, step 4 outputs a non-autoregressive risk scenario jointly. This step integrates the structural causal model-driven prior knowledge transfer mechanism and the bidirectional attention architecture. The model unifies the entire historical dataset and the new test sample as a single sample space, and realizes multi-step risk joint prediction through parallelization prediction method. The specific steps are as follows: The synthetic data generated in the pre-training phase has encoded the complex causal dependence of occupant behavior-vehicle dynamics-environment interaction. In the inference phase, the model integrates the historical dataset and the new test sample into an extended sample space through a bidirectional attention mechanism. This space marks the sample temporal attribute through position encoding and inputs into the Transformer encoder layer Z:

[0057] where, is the sample, is the sample to be predicted, in each layer of Transformer, first perform intra-row feature interaction: calculate the attention weight matrix :

[0058] where, is the input embedding, is the sample index, is the feature dimension index, is the query / key projection matrix, is the key dimension. This operation enables the occupant distraction state to dynamically associate with the acceleration of the ego vehicle and the distance to the front obstacle and other heterogeneous features. Then perform intra-column temporal interaction: calculate the attention :

[0059] where, ​​​​is the time window for the attention weight. By capturing the evolution of key indicators such as occupant state, surrounding vehicle state, etc. within the time window, its decaying attention distribution can identify the precursor patterns of sudden takeover behavior. Through a strict attention masking mechanism, the query set can only absorb information from the support set, making the historical risk label is transferred to the test sample representation step by step through 12 layers of attention propagation. The final updated query set embedding is decoded by two layers of MLP to obtain the future risk probability :

[0060] where, is the final embedding of the query set, are the first / second layer weights of the MLP, respectively, are the first / second layer biases of the MLP, respectively, is the sigmoid function, and the output is the future step risk probability distribution .

[0061] This enables the model to capture the evolution pattern of each key feature over time and its potential long-term impact on future multi-step risk. This enables the model to calculate and save its internal state when it first processes the training set. In addition, the model's optimized memory management enables it to process massive amounts of time series units under limited resources. This combination of multi-step joint prediction, real-time efficient inference, and Bayesian uncertainty output is very suitable for building an automatic driving risk assessment system that takes into account the dynamic changes of occupant state and is highly responsive.

[0062] Further, step 5 risk situation driven decision planning and control, this step takes the non-autoregressive risk deduction result output by step 5 as the core input of the vehicle decision planning module to realize the risk minimization of automatic driving control. This makes the whole system form an end-to-end closed loop from perception to control, ensuring that the vehicle can generate safe and efficient control instructions in real time based on the dynamically changing risk situation. The specific steps are as follows: S5.1 Risk Situation and Cost Function Fusion S5.2 World Model Assisted Control Instruction Generation Further, S5.1 risk situation and cost function fusion. The invention dynamically maps the results of human-environment joint deduction to the cost function of the planning module, achieving active quantification and real-time minimization of potential danger.

[0063] Based on the principle of rolling horizon optimization, this decision planning task is reconstructed as a nonlinear optimization problem with time-varying constraints. Our goal is to solve a set of optimal operation sequences such that the overall loss function Min.

[0064]

[0065] where the total loss function is composed of two parts:

[0066] where, a risk-sensitive loss term, which directly maps the derived probability risk and uncertainty into a non-linear penalty, guiding the vehicle to choose safer behaviors.

[0067]

[0068] where, denotes the risk probability at the th time step of performing the operation sequence . denotes the Bayesian uncertainty at this time step (i.e., the size of the confidence interval). is a pre-defined safety uncertainty threshold. , and are the weighting coefficients for each term.

[0069] a trajectory tracking and operation constraint loss term, which is the traditional trajectory tracking cost function part, ensuring the kinematic properties of the vehicle are satisfied and optimizing the smoothness of the operation.

[0070]

[0071] where, and are the state vector and control command of the vehicle at time instant, respectively. is the reference state. is the smoothness penalty term for the control command. are the weights for each term.

[0072] By minimizing this cost function with the embedded non-linear risk term, the planning module can automatically trade-off between risk aversion and tracking efficiency, prioritizing the operation with the lowest risk.

[0073] Further, S5.2 world model assisted control command generation, in a higher level implementation, the present application utilizes the human-in-the-loop environment evolution model to perform forward simulation to predict the future consequences of different control command sequences.

[0074] First, operation sequence sampling: the planning module first generates a set of candidate operation sequences , representing different potential behaviors.

[0075] Second, joint state forward simulation: for each candidate sequence , the system inputs the current state and the sequence into the human-in-the-loop environment evolution model to simulate the future traffic scenario evolution.

[0076]

[0077] where, is the predicted future joint state (including vehicles and environment) at time instant. is the action command at the previous time instant. is the environment state at the previous time instant. The model outputs the simulated future scenario and vehicle trajectory for multiple steps.

[0078] Finally, loss evaluation and optimal selection: the future state sequence of each simulated trajectory is input into the risk inference model and tracking loss model to calculate its corresponding overall loss . Finally, the planning module selects the control sequence with the smallest overall loss.

[0079]

[0080] Figure 4 The display diagram of the automatic driving risk inference method and system based on the world model and human-in-the-loop environment of the present application is shown. It should be noted that the diagram is only an interface example diagram, only contains necessary functions, and can be improved according to specific needs in the future, which is not a constraint condition of the present application.

[0081] Referring to Figure 4, specifically explains the implementation function of the system, block ① shows the function interface as an autonomous driving risk decision platform interface, including four function modules of overall situation, risk situation, risk decision and intelligent panel; block ② overall situation module contains three parts of current road section traffic flow, vehicle information list, according to real-time traffic flow and historical data, predicts the traffic condition in a period of time in the future, displays the traffic flow trend chart in the next few minutes or hours, helps the driver to make the optimal decision. Block ③ is the risk situation module, which displays the real-time risk assessment results of multiple dimensions on the current road, including collision risk, traffic congestion, road condition, weather change and the like. According to real-time data update, it provides instant risk warning and decision suggestion; block ④ corresponds to the risk decision module, when detecting an emergency (such as imminent collision, emergency brake and the like), it outputs real-time control instruction suggestion to avoid risks. At the same time, it also supports providing suggestions under different driving modes and states; block ⑤ is the button of intelligent assistant, which can input intelligent query content and obtain intelligent risk decision suggestion by clicking; block ⑥ is a real-time simulation scene, which can real-time view the risk situation of the autonomous driving vehicle placed on the road section.

[0082] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit it, although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that: the specific embodiments of the present application can still be modified or replaced by the equivalent, without departing from the spirit and scope of the present application, any modification or equivalent replacement, which should be covered within the protection scope of the claims of the present application.

Claims

1. A method for extrapolating autonomous driving risks based on the coupling of a world model and human factors, characterized in that, include: Acquire multimodal data, wherein the multimodal data includes occupant status, vehicle motion, and environmental interaction data; Construct the world model and the human-factor coupled risk assessment model, generate long-tail scenario data by processing the world model based on the multimodal data, and pre-train the world model and the human-factor coupled risk assessment model using the long-tail scenario data; A causal model is constructed and the dynamic occupant state is obtained. Synthetic training data is generated based on the dynamic occupant state using the causal model. Real-time data is acquired, and the synthetic training data is used as prior knowledge. A pre-trained human-factor coupling risk assessment model is used to assess the risk of the real-time data to obtain the risk probability distribution. An operation sequence is constructed, and the operation sequence is extrapolated through a pre-trained world model to obtain future scene evolution data. The overall loss is obtained based on the future scene evolution data, and the optimal control sequence data for autonomous driving is generated based on the overall loss to control autonomous driving.

2. The method according to claim 1, characterized in that, The process of acquiring multimodal data includes: Collect multi-dimensional data, including dynamic state vectors, takeover trigger timestamps, vehicle motion perception data, and dynamic target trajectory sets; The multi-dimensional data is subjected to spatiotemporal synchronization and interpolation compensation; the compensated data is then mapped to a unified spatial reference using a differentiable rendering engine to obtain multimodal data.

3. The method according to claim 1, characterized in that, The process of acquiring long-tail scenario data includes: Environmental features are extracted from the multimodal data, through... The generator performs road topology reconstruction on the environmental features to obtain road raster data; Human factor heatmaps and dynamic target trajectory sets are extracted from the multimodal data. Based on the human factor heatmaps and dynamic target trajectory sets, dynamic proxy trajectory prediction data is generated through a trajectory diffusion model. The historical tensors of the multimodal data are compressed using 3D convolution to obtain latent variables, physical constraints are constructed, and the physical constraints are injected into the latent variables through a Lie group algebra layer to obtain scene latent variable data. Based on the conditional tensor, the scene latent variable data is gradually denoised through a probabilistic generative model to obtain scene latent variable data at different time steps. The conditional tensor includes road topology data, dynamic agent trajectory prediction data, and human factors heatmap. The road topology data is generated based on the road grid data. Long-tail scene data is obtained by decoding the latent variable data of the scene at different time steps.

4. The method according to claim 3, characterized in that, In the world model, a spatial gating mask is generated based on road topology data. The spatial gating mask is then aligned with cross-modal features using a gating spatiotemporal attention mechanism to perform spatial masking.

5. The method according to claim 1, characterized in that, The process of generating synthetic training data includes: Construct a causal model, wherein the causal model is used to model the dynamic causal chain between human state and environmental factors, wherein the causal model is constructed by a dynamic Bayesian causal modeling method; Interventional sampling of the dynamic occupant state is performed to obtain synthetic training data. ;in: ， in, This refers to the combined state of the vehicle and the environment. Represents the real-time vector of human factors Intervention should be carried out. Indicates the level of distraction. Indicates environmental characteristics, Let P represent vehicle operation, and let P represent probability distribution. Indicates risk outcome, The risk mapping function represents the state vector.

6. The method according to claim 1, characterized in that, The process of obtaining the risk probability distribution includes: A historical dataset is generated based on the synthetic training data, and the historical dataset is integrated with the real-time data through a bidirectional attention mechanism to obtain an expanded sample space; The query set is finally embedded by encoding the position of the extended sample space and then encoding it through the Transformer encoder layer. The query set is finally embedded and decoded using a multilayer perceptron to obtain the probability distribution of future risks.

7. The method according to claim 1, characterized in that, The encoding process in the Transformer encoder layer includes calculating attention weights for the same sample after positional encoding, and then calculating attention weights for the same sample after positional encoding.

8. The method according to claim 1, characterized in that, Overall losses include risk-sensitive loss items. Trajectory and Operational Constraint Loss Term ,in: ， in, Indicates the execution of the operation sequence The The probability of risk at each time step This represents the Bayesian uncertainty at that time step; As a preset threshold for security uncertainty, , and These are the weighting coefficients for each item; ， in, and The vehicles are respectively in The state vector and control command at each moment For reference only. To control the smoothness of commands, The weights of each item.

9. An autonomous driving risk simulation system based on the coupling of a world model and human factors, characterized in that, The method described in any one of 1-8 is required to exercise the above rights.

Images