Automatic driving training method and system based on differentiable world model simulation system, computer device and medium

Abstract

The application relates to an automatic driving training method and system based on a differentiable world model simulation system, a computer device and a medium. The method comprises the following steps: constructing a differentiable world model simulation system; the differentiable world model simulation system comprises a differentiable perception module, a dynamics model and a strategy network; freezing the strategy network, pre-training the differentiable world model simulation system by using first training data; unfreezing the strategy network, performing end-to-end joint optimization on the differentiable world model simulation system by using second training data and adopting a curriculum learning strategy; and performing safety reinforcement training on the differentiable world model simulation system by using third training data. The method realizes end-to-end differentiation of automatic driving full links, i.e. perception, prediction and planning, improves training efficiency and improves the generalization of long-tail scenes.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to an autonomous driving training method, system, computer equipment, and medium based on a differentiable world model simulation system. Background Technology

[0002] The training of current autonomous driving systems mainly relies on the following technical paths: (1) Modular training framework: Decouple the perception, prediction and planning modules, train them separately and then integrate them; (2) Reinforcement learning scheme: Based on the simulation system of the non-differentiable world model (such as the traditional simulator), optimize the driving strategy through trial and error interaction; (3) Imitation learning scheme: The policy network is trained by expert demonstration data, but it lacks the ability to model the dynamic environment.

[0003] The above technical approach has the following shortcomings: The nondifferentiability of environmental conditions on vehicle actions makes it impossible to optimize the decision-control link end-to-end. Sampling-based reinforcement learning is difficult to cover low-probability dangerous scenarios (such as emergency obstacle avoidance scenario coverage <3%). Furthermore, existing world model simulation systems have poor dynamic rationality in predicting trajectories (such as collision false alarm rate >15%). In addition, the above technical approaches require training with over one million kilometers of road test data, resulting in low training efficiency. Summary of the Invention

[0004] To address the aforementioned shortcomings or drawbacks, this application provides an autonomous driving training method, system, computer equipment, and medium based on a differentiable world model simulation system. This method enables end-to-end differentiation across the entire autonomous driving process (perception → prediction → planning), improving training efficiency and generalization capabilities for long-tail scenarios.

[0005] This application provides a method for training autonomous driving based on a differentiable world model simulation system according to a first aspect, the method comprising: Training data is constructed based on pre-collected multimodal sensor data; the training data includes first training data, second training data, and third training data; A differentiable world model simulation system is constructed. This system includes a differentiable perception module, a dynamics model, and a policy network. The perception module generates dynamic scene decoupling data based on the input multimodal sensor data sequence. The dynamic scene decoupling data includes road structure, current state of traffic participants, and environmental conditions. The dynamics model generates future trajectories of traffic participants based on their current state and initial control commands. The policy network generates safety control commands based on road structure, future trajectories of traffic participants, and environmental conditions. The strategy network is frozen and pre-trained on the differentiable world model simulation system using the first training data; the strategy network is unfrozen and jointly optimized end-to-end using the second training data and a course learning strategy; the simulation system is then subjected to security reinforcement training using the third training data.

[0006] In some embodiments, the perception module includes a neural field construction module and a dynamic scene decoupler; the neural field construction module is used to construct a spatiotemporally aligned multimodal neural field based on the input multimodal sensor data sequence; the dynamic scene decoupler is used to extract equivariant features from the multimodal neural field using a three-dimensional Euclidean group equivariant convolutional neural network, and generate structured dynamic scene decoupling data based on the extracted equivariant features.

[0007] In some embodiments, the policy network includes a high-level decision-maker and a low-level controller; the high-level decision-maker is used to predict the intentions of traffic participants based on their future trajectories to obtain their intentions; the low-level controller is used to perform differential flat spatial projection based on the traffic participants' intentions to obtain the vehicle's initial trajectory parameters, apply physical and rule constraints to the initial trajectory parameters based on preset dynamic constraint information and road structure to obtain the original control command, and perform risk field modulation on the original control command based on the environmental conditions, risk field gradient, and weights of the traffic participants' intentions output by the perception module to obtain the safety control command.

[0008] In some embodiments, the dynamic model is used to construct input data based on the current state of the traffic participant and the initial control command, input the input data into a pre-constructed neural ordinary differential equation to obtain the state derivative, and use a specified solver to integrate the state derivative over a specified time interval to obtain the future trajectory of the traffic participant.

[0009] In some embodiments, during pre-training, the perception loss and dynamics loss are combined to update the parameters of the perception module and the dynamics model, but the parameters of the policy network are not updated; during end-to-end joint optimization, the simulation loss, the real vehicle loss, and the consistency loss are combined to update the parameters of the perception module, the dynamics model, and the policy network; during safety enhancement training, the collision loss and the comfort loss are combined to update the parameters of the perception module, the dynamics model, and the policy network.

[0010] In some embodiments, the perception loss is calculated based on the intersection-union ratio between the predicted semantic segmentation result and the ground truth semantic segmentation result, and the Euclidean distance between the predicted traffic participant position and the ground truth traffic participant position; the dynamics loss is calculated based on the mean square error between the predicted state change rate and the ground truth state change rate; the simulation loss is calculated based on the Euclidean distance between the predicted trajectory and the ground truth trajectory in the simulation environment, and the Euclidean distance between the predicted control command and the ground truth control command; the real vehicle loss is calculated based on the real vehicle intervention frequency and the vehicle longitudinal acceleration change rate; the consistency loss is calculated based on the maximum mean difference between the simulation feature distribution and the real vehicle feature distribution; the collision loss is calculated based on the minimum distance between the vehicle and the nearest obstacle; and the comfort loss is calculated based on the vehicle longitudinal acceleration change rate and the steering wheel angular velocity.

[0011] In some embodiments, during the backpropagation process of end-to-end joint optimization, the vehicle Jacobian matrix is ​​calculated on the neural ordinary differential equation of the dynamic model by automatic differentiation, an attention diagonal matrix is ​​constructed based on the interaction attention weights of the vehicle and other traffic participants, a target gradient is calculated based on the vehicle Jacobian matrix and the attention diagonal matrix, and gradient information for updating the parameters of the perception module, the dynamic model and the policy network is generated based on the target gradient.

[0012] According to a second aspect, this application provides an autonomous driving training system based on a differentiable world model simulation system, the system comprising: The training data construction module is used to construct training data based on pre-acquired multimodal sensor data; the training data includes first training data, second training data, and third training data; The world model simulation system construction module is used to build a differentiable world model simulation system. This system includes a differentiable perception module, a dynamics model, and a policy network. The perception module generates dynamic scene decoupling data based on the input multimodal sensor data sequence. This decoupling data includes road structure, current states of traffic participants, and environmental conditions. The dynamics model generates future trajectories of traffic participants based on their current states and initial control commands. The policy network generates safety control commands based on road structure, future trajectories of traffic participants, and environmental conditions. The phased optimization module is used to freeze the policy network and pre-train the differentiable world model simulation system using the first training data; unfreeze the policy network and use the second training data and a course learning strategy to perform end-to-end joint optimization of the differentiable world model simulation system; and use the third training data to perform security reinforcement training on the differentiable world model simulation system.

[0013] According to a third aspect, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above embodiments of the autonomous driving training method based on a differentiable world model simulation system.

[0014] According to a fourth aspect, this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executed, implements the steps of any of the above embodiments of the autonomous driving training method based on a differentiable world model simulation system.

[0015] The embodiments described above in this application construct training data, including first training data, second training data, and third training data, based on pre-collected multimodal sensor data. Next, a differentiable world model simulation system is constructed. This system includes a differentiable perception module, a dynamics model, and a policy network. The perception module generates dynamic scene decoupling data based on the input multimodal sensor data sequence. The dynamic scene decoupling data includes road structure, current state of traffic participants, and environmental conditions. The dynamics model generates future trajectories of traffic participants based on their current state and initial control commands. The policy network generates safety control commands based on road structure, future trajectories of traffic participants, and environmental conditions. Finally, the differentiable world model simulation system is optimized in stages. The first stage includes freezing the policy network and pre-training the system using the first training data. The second stage includes unfreezing the policy network and performing end-to-end joint optimization using the second training data and a course learning strategy. The third stage includes performing safety reinforcement training on the system using the third training data. This application's embodiments transform the discrete decision-making links in the traditional pipeline into continuous optimization problems through a differentiable world model simulation system architecture, providing a new paradigm for autonomous driving system training. It can achieve end-to-end differentiation of the entire autonomous driving link (perception → prediction → planning), improving training efficiency and generalization in long-tail scenarios. Attached Figure Description

[0016] Figure 1 This is a flowchart of an autonomous driving training method based on a differentiable world model simulation system, which is one or more embodiments of this application. Figure 2 This is a schematic diagram of the architecture of a differentiable world model simulation system in one or more embodiments of this application; Figure 3 This is a schematic diagram illustrating the motion prediction of a neural ODE in one or more embodiments of this application; Figure 4This is a schematic diagram of the architecture of the policy network in one or more embodiments of this application; Figure 5 This is a schematic diagram illustrating the principle of differential flat control transformation in one or more embodiments of this application; Figure 6 This is a schematic diagram of the virtual-reality gradient alignment process in one or more embodiments of this application; Figure 7 This is a schematic diagram of the structure of an autonomous driving training system based on a differentiable world model simulation system in one or more embodiments of this application; Figure 8 This is a schematic diagram of the internal structure of a computer device according to one or more embodiments of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the described embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0018] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0019] In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0020] To address the shortcomings or defects of related technologies, this application provides an autonomous driving training method based on a differentiable world model simulation system. This method transforms the discrete decision-making links in the traditional pipeline into continuous optimization problems through the architecture of a differentiable world model simulation system, providing a new paradigm for autonomous driving system training. It can achieve end-to-end differentiation of the entire autonomous driving link (perception → prediction → planning), improve training efficiency, and enhance the generalization of long-tail scenarios.

[0021] In some exemplary embodiments of this application, the method includes as follows: Figure 1 The steps shown are explained in detail below.

[0022] S110: Construct training data based on pre-acquired multimodal sensor data; the training data includes first training data, second training data, and third training data.

[0023] In autonomous driving systems based on differentiable world model simulation systems, multimodal sensor data refers to raw data collected by multiple sensors. These sensors may include LiDAR (Light Laser Detection and Ranging), cameras, millimeter-wave radar, IMU (Inertial Measurement Unit), and GPS (or RTK). GPS stands for Global Positioning System, and RTK stands for Real-time kinematic. Multimodal sensor data can include 3D point cloud sequences acquired by LiDAR (sampling frequency can be 10 to 20 Hz, which can be used for high-precision geometric modeling and obstacle detection), multi-view RGB (Red-Green-Blue) image sequences acquired by cameras (sampling frequency can be 30 to 60 Hz, which can be used for semantic segmentation and traffic sign recognition), 4D point clouds acquired by millimeter-wave radar (sampling frequency can be 10 to 25 Hz, which can be used for moving object tracking and speed measurement), 6-axis inertial measurement data acquired by IMU (sampling frequency can be 100 to 1000 Hz, which can be used for vehicle pose estimation and motion compensation), and high-precision global positioning data acquired by GPS or RTK (sampling frequency can be 5 to 10 Hz, which can be used for global path planning and map matching). Among these, the world model simulation system is a key component of the world model; specifically, it is a state transition model capable of predicting the future distribution of environmental states based on current environmental data and current policy data.

[0024] Training data is a data stream obtained by slicing multimedia sensor data (e.g., generating a 5-frame sequence according to a 100-millisecond sliding window), time synchronization, spatial alignment, annotation, and other processes; that is, a multimodal sensor data sequence.

[0025] Multimodal sensor data can be real vehicle data extracted from normal driving segments of a real vehicle in various scenarios, or simulation data generated using a simulation platform.

[0026] The first, second, and third training data are training data used in different training stages. They are all multimodal sensor data sequences. However, different training stages will use multimedia sensor data from different scenarios and sources to construct the training data. For example, the first training data can be a multimodal sensor data sequence continuously collected in a static scene or a simulation environment containing only simple dynamic objects with uniform motion. The second training data can be a multimodal sensor data sequence continuously collected in a dynamic urban scene or highway scene containing traffic lights, intersections, and adversarial traffic flow. 90% of this data can come from real-time rendering on a high-fidelity simulation platform, and 10% can come from anonymized real vehicle logs. The third training data can be a multimodal sensor data sequence continuously collected in a high-adversarial scene containing sudden braking, rapid cutting in, and oncoming vehicles, as well as under extreme conditions such as heavy rain, night, and dense fog. 95% of this data can be reconstructed by a simulation adversarial generator based on real accident data or generated by GAN, and 5% can come from extreme working condition segments of real vehicles.

[0027] S120: Construct a simulation system for a differentiable world model; the simulation system for a differentiable world model includes a differentiable perception module, a dynamic model, and a policy network.

[0028] A differentiable world model simulation system refers to an autonomous driving world model simulation system in which all computational units are continuously differentiable functions, enabling end-to-end backpropagation from raw sensor inputs to final control commands. It integrates the traditional four discrete modules of "perception-prediction-planning-control" into three differentiable sub-networks (i.e., perception, dynamics model, and policy network), and achieves end-to-end gradient flow through automatic differentiation (Autograd). Compared to related technologies that decouple the perception, prediction, and planning modules, train them separately, and then integrate them, this approach effectively improves the training efficiency and policy safety of the world model simulation system in complex scenarios. Figure 2 The diagram shown is a schematic of the architecture of a differentiable world model simulation system. The following section will combine... Figure 2 The simulation system for the differentiable world model is described.

[0029] The perception module generates dynamic scene decoupling data based on the input multimodal sensor data sequence; this data includes road structure, current state of traffic participants, and environmental conditions. The dynamic model generates future trajectories of traffic participants based on their current states and initial control commands. The policy network generates safety control commands based on road structure, future trajectories of traffic participants, and environmental conditions.

[0030] In some embodiments, the perception module includes a neural field construction module and a dynamic scene decoupler.

[0031] The neural field construction module is used to construct a spatiotemporally aligned multimodal neural field based on the input multimodal sensor data sequence. For example, ego-motion interpolation can be performed on RGB images acquired by a camera and 4D point clouds acquired by radar, using a LiDAR frame as the reference clock, to unify these data into the current LiDAR coordinate system. Furthermore, 6-DoF pose data acquired by an IMU can be used for compensation to eliminate ghosting of the vehicle's motion. For RGB images, distortion correction can be performed, such as using intrinsic parameters and distortion coefficients to undistort each frame, resulting in a distortion-free RGB image. Next, voxel rasterization is performed, for example, establishing a 256×256×32 voxel grid with a resolution of 0.1m. The fill channels can include: occupancy probability (LiDAR point density), reflection intensity (LiDAR), Doppler velocity (Radar interpolation), and RGB features (which can be obtained by combining camera backprojection and trilinear interpolation). Then, the voxel features are temporally aligned. For example, the voxel features of the most recent k=5 frames (0.1-second intervals) are concatenated along the time dimension to obtain a 4D tensor. This is then compressed using a 1×1×1×3 convolution to obtain the final tensor F∈ℝ^(B,C,X,Y,Z), where B refers to the batch size, C refers to the number of channels, and "X,Y,Z" are the spatial dimensions. The final tensor F(x,y,z,t) is the spatiotemporally aligned multimodal neural field.

[0032] The dynamic scene decoupler is used to extract equivariant features from a multimodal neural field using a 3D Euclidean group equivariant convolutional neural network (SE(3)-Equivariant CNN), and generate structured dynamic scene decoupling data based on the extracted equivariant features. Among them, the equivariant features have rotation and translation invariance. For example, the state representation of the vehicle point cloud remains consistent regardless of the viewing angle.

[0033] Dynamic scene decoupling (which can be represented as) The input is a spatiotemporally aligned multimodal neural field.

[0034] A multimodal neural field can be represented as F(x,y,z,t), where x,y,z,t represent the following data: (1) LiDAR point cloud can be a geometric structure in three-dimensional space (such as voxelization or original point cloud). (2) The RGB image from the camera can be generated by cross-modal fusion of RGB features (such as semantic information from the BEV perspective); BEV refers to the bird's-eye view. (3) Radar point cloud data, including position information and velocity information (such as Doppler radar point cloud); (4) Timestamps can ensure the temporal correlation of consecutive frames (such as optical flow or scene flow).

[0035] The output of the dynamic scene decoupler is the semantic and state decomposition of the dynamic scene, which consists of three core elements: (1) Road structure (static environment) can be a vector representation of lane lines, curbs and traffic signs. Its output form can be a BEV segmentation map (i.e., (H×W) grid, where each pixel corresponds to a semantic category). (2) Current state of traffic participants (dynamic objects), which can be the current state of traffic participants such as vehicles, pedestrians, and cyclists (such as position, speed, acceleration and heading angle); its output form can be a structured list (where each traffic participant contains 6DoF pose and motion state); (3) Environmental conditions (which can be light and / or weather), which can be quantitative parameters of light intensity, rain, snow and fog (such as visibility coefficient), and the output form can be scalar or low-dimensional vector (such as weather influence factors in the interval [0,1]).

[0036] The dynamic scene decoupler inputs the multimodal neural field into a three-dimensional Euclidean group equivariant convolutional neural network (SE(3)-Equivariant CNN, where the convolutional kernels of each layer can be 64→128→256→512 respectively), and obtains the equivariant feature map output by the three-dimensional Euclidean group equivariant convolutional neural network. The equivariant feature map can be represented as Φ, where Φ∈ℝ^{B,512,X,Y,Z}, thus ensuring that the features are consistent after rotation or translation transformation. Then, a 1×1×1 convolution is performed on the equivariant feature map to obtain a three-way parallel tensor, namely: (1) Φ_sem, i.e. semantic logic (7 categories); (2) Φ_inst, i.e. instance embedding (32-dimensional); (3) Φ_motion, i.e., motion vector (v_x,v_y,v_z).

[0037] Semantic decoding of Φ_sem yields the road structure (specifically, the road structure raster, represented as road_structure); instance segmentation of Φ_inst produces a list of traffic participant IDs (including the IDs of each traffic participant); motion estimation is performed on each traffic participant ID based on Φ_motion to obtain a 6-DoF pose sequence (i.e., the current state of the traffic participant) for each ID; and global average pooling is applied to the isotropic feature map to obtain the environmental conditions (represented as environment); the environmental conditions can be a three-dimensional vector, such as [light, rain, fog]∈[0,1]. Finally, the road structure, the current state of traffic participants, and the environmental conditions can be output in JSON format, for example, {road_structure, agents, environment}.

[0038] In some embodiments, the dynamic model is used to construct input data based on the current state of the traffic participant and the initial control command, input the input data into a pre-constructed neural ordinary differential equation to obtain the state derivative, and use a specified solver to integrate the state derivative over a specified time interval to obtain the future trajectory of the traffic participant.

[0039] The dynamic model introduces a continuous-time neural ODE (i.e., neural frequent differential equation). The neural ODE can model continuously (which can overcome the error accumulation of discrete time steps, especially suitable for high-frequency control), can realize interactive perception (specifically, it can learn the interaction of complex multi-agents, such as the game behavior between vehicles), and can also achieve physical consistency (i.e., integrate explicit dynamic priors, such as bicycle models, with data-driven complements). Figure 3 The diagram shown below illustrates the motor prediction of neural ODEs. Figure 3 Explain the neural ODE.

[0040] The neural ODE is modeled as follows: ; The physical meaning and mathematical definition of all parameters in the above formula are explained below: This refers to the current state of a traffic participant, representing the state vector of the traffic participant at time (t), and its dimension (d) is typically [missing value]. (i.e., basic 6DoF + extended features), specifically including: Pose (3DoF): (i.e., horizontal position and heading angle); Motion status (3DoF): (i.e., linear velocity and angular velocity).

[0041] Extended features ( Dimension: can be acceleration (represented as...) ), steering angle (expressed as One or more of the following: interaction features (such as attention weights).

[0042] This refers to the initial control command, which is specifically the control input to the autonomous vehicle (or ego vehicle, self-driving vehicle) (it can be the output of a policy network or from expert data). Its form can be: Longitudinal control: acceleration (which can be expressed as) , ); Lateral control: Front wheel steering angle (which can be expressed as...) , ); It can be represented as: (Two-dimensional vector).

[0043] This represents the dynamic function, which signifies the rate of change of state in a neural network model (continuous-time dynamics). Parameters Represents the weights of a neural network (such as an MLP or Transformer). The inputs are the current state of the traffic participants and the initial control commands, and the output is the state derivative (expressed as...). ).

[0044] After obtaining the state derivative, a specified solver (such as a 5th-order implicit Runge-Kutta solver) can be used to integrate the state derivative over a specified time interval to obtain the future trajectory of the traffic participants.

[0045] In some embodiments, the policy network, also known as a hierarchical policy network, has an architecture that includes an input layer, a high-level decision network (also known as a high-level decision maker), a low-level control network (also known as a low-level controller), and an output layer. Figure 4 The diagram shown below illustrates the architecture of a policy network. The following section will discuss this in conjunction with... Figure 4 The various parts of the policy network are explained in detail.

[0046] (1) Input layer The sources of input data include: BEV (Bird's-eye View) semantic information: that is, static road structures (such as lane lines, curbs, traffic signs, etc.). Instance sports field: that is, the current state of dynamic traffic participants (including position, speed, heading angle).

[0047] The input layer preprocesses the input data, such as aligning and encoding multimodal sensor data (e.g., LiDAR point clouds, camera RGB image sequences) to convert them into a unified vector representation for subsequent network processing.

[0048] (2) High-level decision-making network The high-level decision network is used to predict the intentions of traffic participants based on their future trajectories, thus obtaining their intentions. The high-level decision network includes a graph construction module, a graph attention layer, and an intention classifier.

[0049] The graph construction module is used to model traffic scenarios as a graph structure. The nodes of the graph represent the state information (such as position and speed) of each traffic participant (e.g., ego vehicles, surrounding vehicles, pedestrians), while the edges represent the spatial relationships (such as distance and relative speed) between traffic participants. The input to the graph construction module is the output of the input layer, and the output of the graph construction module is a dynamically updated graph that reflects the interaction topology of the current traffic scenario.

[0050] The graph attention layer is used to identify key interactive objects in the graph output by the graph construction module. It calculates the importance weights between nodes in the graph through an attention mechanism (for example, an ego vehicle will pay more attention to a vehicle braking suddenly in front of it, while ignoring a stationary vehicle in the distance, thus giving the vehicle braking suddenly a higher importance weight than a stationary vehicle in the distance). The output of the graph attention layer is a weighted graph representation, which can highlight high-risk or highly relevant interactions by assigning importance weights to nodes.

[0051] The intent classifier is used to predict the intents of traffic participants based on the weighted graph output from the graph attention layer. Intent types can include changing lanes left or right, following another vehicle, emergency braking, turning at intersections, etc. The output of the intent classifier is the probability distribution of the intents of each traffic participant (e.g., "changing lanes left: 70%, following another vehicle: 30%").

[0052] (III) Lower-level control network The underlying control network is used to perform differential flat space projection based on the intentions of traffic participants to obtain the initial trajectory parameters of the vehicle. Based on the preset dynamic constraint information and road structure, physical and rule constraints are applied to the initial trajectory parameters to obtain the original control command. Based on the environmental conditions, risk field gradient and the weights of the traffic participants' intentions output by the perception module, the original control command is modulated by the risk field to obtain the safety control command.

[0053] The underlying control network includes a differential flat spatial projection module, a differentiable optimization layer, and a risk field modulation module.

[0054] The Differential Flat Space Projection module transforms the traffic participant intentions output by the high-level decision network into physically feasible trajectories. It leverages vehicle dynamics (such as position and heading angle, which fully describe vehicle state) to simplify complex control problems into path optimization in a flat space. Its output is the initial trajectory parameters of the ego vehicle (such as target position and velocity profile). The principle of differential flat control transformation can be found in [link to relevant documentation]. Figure 5 As shown.

[0055] The differentiable optimization layer applies physical and rule constraints to the initial trajectory parameters output by the differential flat space projection module. Specifically, the differentiable optimization layer applies physical constraints to the initial trajectory parameters based on preset dynamic constraint information (such as maximum acceleration and curvature limit information), thereby obtaining the original control commands (such as steering angle). Physical constraints refer to dynamic constraints, such as maximum acceleration and curvature limits (to avoid sharp turns). The differentiable optimization layer further applies rule constraints to the physically constrained initial trajectory parameters based on the road structure output by the dynamic scene decoupler. Rule constraints refer to traffic rules, such as lane keeping and traffic light compliance. The differentiable optimization layer specifically ensures that these constraints are not violated through differentiable operations (such as gradient clipping).

[0056] The risk field modulation module is used to dynamically adjust the safety of the original control commands output by the differentiable optimization layer. The risk field modulation module can modulate the risk field based on the original control commands, the risk field gradient (used to identify the degree of danger of obstacles around the vehicle), and the weights of traffic participant intentions output by the high-level decision network (such as "lane change" intentions requiring higher lateral action sensitivity), thereby obtaining safe control commands (i.e., vehicle control commands constrained by safety boundaries).

[0057] In some embodiments, risk field modulation can be implemented using the following formula, which dynamically adjusts the original control commands output by the differentiable optimization layer. Through risk field gradient weighting, it ensures that the actions of the final output control commands conform to the safety boundary.

[0058] ; The parameters in the formula are explained below: This refers to safety control commands, i.e., control commands that have been modified for safety (such as throttle and steering angle). Its dimensions can be two, namely longitudinal acceleration and lateral steering angle.

[0059] This refers to the policy network (with parameters as follows) According to the current status of traffic participants ( The raw control commands generated may be beyond the safe limits.

[0060] For example, longitudinal: acceleration ( m / s 2 Lateral: Steering angle ( ).

[0061] This refers to the risk field ( ) for state ( The gradient of the surrounding environment can indicate the direction and intensity of danger.

[0062] Risk field ( The definition of ) is as follows: Dynamic risk: inversely proportional to the distance and relative speed of traffic participants (e.g., the closer the vehicle in front, the higher the risk value).

[0063] Static risks: distance to road boundaries and obstacles (risk increases when deviating from the lane).

[0064] It can reflect the direction of the fastest change in danger, and can be calculated using the following formula: ; This refers to the gradient norm, which represents the magnitude of the risk gradient and is used to quantify the overall degree of danger in the current state. A larger value indicates that even slight changes in action can lead to a sharp increase in risk (such as when approaching an obstacle). It can be calculated using the following formula: ; It is the Sigmoid function, which is used to compress the gradient norm to the [0,1] interval as a safety factor.

[0065] (iv) Output layer The output layer is used to control the vehicle based on control commands output from the underlying control network. These control commands can include two types: longitudinal control commands, such as throttle / brake commands (acceleration values), and lateral control commands, such as steering wheel angle (which can be calculated based on the trajectory curvature). The output layer can convert these control commands into vehicle CAN bus signals, which are then used to directly control the actuators.

[0066] S130: Freeze the policy network to pre-train the differentiable world model simulation system using the first training data; unfreeze the policy network to perform end-to-end joint optimization of the differentiable world model simulation system using the second training data and a course learning strategy; and perform security reinforcement training of the differentiable world model simulation system using the third training data.

[0067] This step is used to optimize the differentiable world model simulation system in stages, that is, to train the differentiable world model simulation system in stages one, two, and three in sequence. The training of each stage is explained below.

[0068] The first stage is the pre-training stage, which specifically involves pre-training the perception module and dynamics model in the differentiable world model simulation system. The training goal of this stage is to enable the perception module to learn the accurate mapping from multimodal sensor data to scene understanding, and to enable the dynamics model to master the motion patterns and interaction effects of traffic participants.

[0069] The first training data used in this stage may include: (1) simulation data of 100,000 sets of calibration scenarios (such as sunny and / or rainy days, different traffic densities) generated by the simulation platform; and (2) real vehicle data extracted from 5,000 kilometers of normal driving segments that have been collected in advance and from which sensitive information has been removed.

[0070] Data augmentation can be performed on the first training data. For example, randomly discarding 0-20% of the LiDAR point cloud sequence can simulate sensor occlusion, reflectivity attenuation, and hardware failure in real-world scenarios. Adding fog and haze noise to the RGB images from the camera can reproduce low visibility conditions such as rain, fog, and backlighting. This allows the model to learn invariant representations of missing and degraded inputs, thereby improving the generalization boundary of the subsequent second stage, i.e., end-to-end training. Furthermore, sunny, rainy, and constant-speed traffic scenarios do not adequately cover extreme weather and occlusion conditions. Therefore, through the above data augmentation strategies, the data density of edge scenarios can be artificially expanded without additional acquisition costs, reducing the dependence of the subsequent third stage, i.e., the safety enhancement stage, on rare events and achieving optimal allocation of training resources.

[0071] Before training, policy parameters need to be frozen, which means fixing the parameters of the policy network. Gradient truncation can also be performed by limiting the gradient norm of the perception module (e.g., setting max_grad_norm=1.0), so that only the parameters of the perception module and the dynamics model are updated, without updating the parameters of the policy network.

[0072] Regarding the settings for training parameters, they can be: Optimizer: AdamW (lr=3e-4); Batch Size: 32 (simulation) + 8 (real vehicles); Training duration: 50 epochs (approximately 72 hours); Hardware configuration: 8×A100 GPUs.

[0073] In addition, the validation metric for the perception module can be mIoU > 75% (BEV segmentation); the validation metric for the dynamic model can be predicted position error < 0.3m (1s horizon).

[0074] During pre-training, the perceptual loss and dynamic loss can be combined to update the parameters of the perceptual module and the dynamic model, without updating the parameters of the policy network. For example, the loss function used in this stage could be: ; The above loss function applies to perceived loss (i.e.) ) and dynamic loss (i.e. We perform a weighted summation to obtain the total loss (i.e., Finally, the parameters of the perception module and the dynamics model are updated based on the total loss. The weight values ​​of the perception loss and dynamics loss in the above formula are only examples, and their specific values ​​can be adjusted according to actual needs.

[0075] The perceptual loss described above is calculated based on the intersection-union ratio (IoU) between the predicted and ground truth semantic segmentation results, and the Euclidean distance between the predicted and ground truth traffic participant positions. The dynamic loss is calculated based on the mean square error between the predicted and ground truth rate of change.

[0076] For example, the perceptual loss can be calculated using the following formula: ; in, This refers to the perception loss. pred_seg refers to the BEV (Bird's-Eye-View) semantic segmentation prediction map output by the perception module, for example, with a size of 256×256×7, where each pixel represents the probability distribution of categories such as lane lines, vehicles, and pedestrians. Is with The ground truth semantic segmentation map of the same resolution can be derived from HD maps or self-supervised pseudo-labels. It is the center coordinate (x, y) of each traffic participant in the BEV plane predicted by the perception module. Is with The corresponding truth center coordinates can be given by the LiDAR truth box or instance annotation.

[0077] The kinetic loss can be calculated using the following formula: ; in, This refers to dynamic loss. It refers to the state derivative of the neural ODE output. This refers to the change in the true state, and the calculation method can be... That is, it is obtained by the difference of the true state of two consecutive frames.

[0078] The second stage is the end-to-end joint optimization stage. The training objective of this stage is to achieve end-to-end gradient propagation from the original sensor input to the final control command, thereby optimizing the overall policy. Before starting training, the policy network can be unfrozen by setting its learning rate to 10 times that of the sensor module, for example, setting lr_policy=3e-3. Then, a Hessian-Free optimizer is introduced to handle the non-convex loss surface.

[0079] This stage can employ a course-based learning strategy for training, where the scenario complexity and the proportion of adversarial vehicles can be increased according to the training progress. For example, the corresponding scenario complexity and adversarial vehicle proportions for different training progress are shown in Table 1: Table 1: The optimizer can be configured as follows: Hessian-Free: initial_trust_radius: 0.1; max_hessian_updates: 50; damping: 0.01; use_gauss_newton: True.

[0080] The convergence criterion for this stage can be to conduct 10 consecutive validation set evaluations. If the tracking error of the simulation scenario is less than 1.0 meter and the intervention rate of the real vehicle data is less than 5%, it is considered convergent; otherwise, it is considered non-converged.

[0081] When performing end-to-end joint optimization, simulation loss, real-vehicle loss, and consistency loss can be combined to update the parameters of the perception module, dynamics model, and policy network. For example, the following loss function can be used in this stage: ; The above loss function affects the simulation loss (i.e.) ), actual vehicle loss (i.e. ) and consistency loss (i.e. We perform a weighted summation to obtain the total loss (i.e., Finally, the parameters of the perception module, dynamics model, and policy network are updated based on the total loss. The weight values ​​of simulation loss, real vehicle loss, and consistency loss in the above formula are only examples, and their specific values ​​can be adjusted according to actual needs.

[0082] The above loss function can be used to achieve virtual-real gradient alignment. The virtual-real gradient alignment process can be found in [link to documentation]. Figure 6 As shown.

[0083] The simulation loss mentioned above is calculated based on the Euclidean distance between the predicted trajectory and the true trajectory in the simulation environment, as well as the Euclidean distance between the predicted control command and the true control command; the real vehicle loss is calculated based on the real vehicle intervention frequency and the rate of change of the vehicle's longitudinal acceleration; and the consistency loss is calculated based on the maximum mean difference between the simulation feature distribution and the real vehicle feature distribution.

[0084] For example, the simulation loss can be calculated using the following formula: ; in, This refers to simulation loss. It refers to the future trajectories of traffic participants (specifically, coordinate sequences) output by the dynamic model in the simulation environment. This refers to the true trajectory values ​​(also coordinate sequences) generated by expert strategies or HD-maps in a simulation environment, which are related to... Same dimension. This refers to the safety control commands (such as acceleration and steering wheel angle) output by the policy network in the simulation environment. It refers to the true value of the control instructions given by experts or rule controllers in the simulation environment.

[0085] The actual vehicle damage can be calculated using the following formula: ; in, This refers to actual vehicle damage. This refers to the ratio of the number of times a safety driver takes over the steering wheel or brake pedal during real-world vehicle testing to the total mileage driven. It refers to the first derivative of the vehicle's longitudinal acceleration with respect to time, and the unit can be m / s³. It is calculated differentially from the 100Hz signal of the IMU.

[0086] The consistency loss can be calculated using the following formula: ; in, This refers to the loss of consistency. It refers to the feature distribution extracted by the perception module from the simulation data (which is a set of samples, and the dimension can be 256). This refers to the feature distribution extracted by the perception module from real vehicle data, and... Same size in the same batch.

[0087] In some embodiments, during the backpropagation process of end-to-end joint optimization, the vehicle Jacobian matrix is ​​calculated on the neural ordinary differential equation of the dynamic model by automatic differentiation, an attention diagonal matrix is ​​constructed based on the interaction attention weights of the vehicle and other traffic participants, a target gradient is calculated based on the vehicle Jacobian matrix and the attention diagonal matrix, and gradient information for updating the parameters of the perception module, the dynamic model and the policy network is generated based on the target gradient.

[0088] The dynamic model in this embodiment incorporates interactively-aware Jacobian propagation, and the relevant calculation formulas are as follows: ; The above formula is used to quantify the actions of the ego vehicle (i.e., the self-driving vehicle). ) regarding its own future state ( The model indirectly influences the states of surrounding traffic participants. During gradient backpropagation, interactive attention weights can be used to focus on key interaction objects, suppressing irrelevant vehicles and avoiding interference from unrelated participants. Therefore, during training, the model can automatically focus on key interaction scenarios (such as lane-changing games and emergency obstacle avoidance), improving the robustness of policies in complex scenarios. For example, when an ego vehicle turns, it only strengthens gradient updates for the states of nearby vehicles, ignoring distant vehicles.

[0089] For example, if the interaction attention weight matrix W is: ; During gradient propagation, 90% of the gradient of ego's own state is retained, only 10% for car A, and it is ignored for car B.

[0090] The parameters in the formula are explained below: This refers to the target gradient, representing the ego vehicle's action ( ) regarding its own future state ( The direct impact of ego on vehicle braking (i.e., How do changes affect its position, velocity, and other states? Its dimension is ℝ^d×m, where d is the state dimension and m is the action dimension. and From the policy network.

[0091] This refers to the Jacobian matrix of the vehicle, i.e., the dynamic function. The Jacobian matrix of the ego vehicle state reflects the state derivative ( ) for the current state ( The local sensitivity of ) has a dimension of ℝ^d×d, where d is the state dimension. The dynamic function of the neural ODE can be obtained through automatic differentiation (Autograd). It is calculated using the following formula: ; It refers to the transposed Jacobian matrix, used for gradient backpropagation (chain rule). In backpropagation automatic differentiation, gradient propagation requires left multiplication by the Jacobian transpose. The upstream gradient (the gradient of the loss function with respect to the state) can be passed to the action space.

[0092] This refers to the attention diagonal matrix, whose dimensions and sum The dimensions are the same. The diagonal elements of the attention diagonal matrix are the interaction attention weights of the ego vehicle with other traffic participants. The higher the interaction attention weight of the ego vehicle with other traffic participants, the greater the influence of that traffic participant on the ego vehicle's state (e.g., a vehicle braking suddenly in front is more critical than a vehicle further away). These weights can be calculated by the interaction perception module (such as GNN or Transformer) using the following formula: ; This refers to the interaction attention weight between the ego vehicle and the j-th traffic participant. This refers to the query vector (Query) for ego vehicles. It refers to the key vector (Key) of the j-th traffic participant. This refers to the vector dimension scaling factor.

[0093] The third stage is the safety enhancement stage. The training objective of this stage is to improve the robustness of long-tail scenarios by training the model, which already has basic driving capabilities, through high-intensity and extreme scenario training.

[0094] This phase can introduce various types of confrontations, such as vehicles braking suddenly (e.g., deceleration of 8 m / s²), malicious cutting in (e.g., lateral speed of 3 m / s, cutting angle of 45°), and pedestrians running red lights (running speed of 2.5 m / s). The generation method can be: { def generate_adversarial(scene): if random() < 0.05: # 5% probability agent = choice(scene.vehicles) agent.apply_emergency_behavior() return scene}

[0095] In the training process, forward propagation can use a batch (referring to a set of samples used to train the model in one iteration) that mixes normal and adversarial scenarios. When calculating gradients, it is necessary to separate the gradients of normal samples and adversarial samples. When updating parameters, standard SGD can be used for normal samples, while adversarial gradients after PGD attack (ε=0.1) can be used for adversarial samples.

[0096] During safety enhancement training, collision loss and comfort loss can be combined to update the parameters of the perception module, dynamics model, and policy network. For example, the loss function used in this stage could be: ; The above loss function applies to collision loss (i.e.) ) and loss of comfort (i.e. We perform a weighted summation to obtain the total loss (i.e., Finally, the parameters of the perception module, dynamics model, and policy network are updated based on the total loss. The weight values ​​for collision loss and comfort loss in the above formula are only examples, and their specific values ​​can be adjusted according to actual needs.

[0097] The collision loss mentioned above is calculated based on the minimum distance between the vehicle and the nearest obstacle, while the comfort loss is calculated based on the vehicle's longitudinal acceleration rate of change and the steering wheel angular velocity.

[0098] For example, the collision loss can be calculated using the following formula: ; in, This refers to collision damage. This refers to the minimum distance between the vehicle and the nearest obstacle (vehicle, pedestrian, stationary object), measured in meters. In this formula, when... Penalties begin when the distance is less than 1.5 meters.

[0099] The loss of comfort can be calculated using the following formula: ; in, This refers to the loss of comfort. It refers to the first derivative of the vehicle's longitudinal acceleration with respect to time, i.e., the rate of change of acceleration, with units of m / s³, which can be obtained by differential analysis of the 100Hz signal from the IMU. It refers to the steering wheel angular velocity, which is the first derivative of the steering angle with respect to time, and is measured in rad / s. It can be obtained by differential analysis of the 100Hz vehicle bus signal.

[0100] It should be noted that, regarding the various steps included in the autonomous driving training method based on a differentiable world model simulation system provided in any of the above embodiments, unless explicitly stated herein, there is no strict order restriction on the execution of these steps; they can be executed in other orders. Furthermore, at least some of these steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is also not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0101] Based on the same inventive concept, this application also provides an autonomous driving training device based on a differentiable world model simulation system. In some embodiments, such as Figure 7 As shown, the autonomous driving training device based on the differentiable world model simulation system includes the following modules: The training data construction module 110 is used to construct training data based on pre-acquired multimodal sensor data; the training data includes first training data, second training data, and third training data; The world model simulation system construction module 120 is used to construct a differentiable world model simulation system. The differentiable world model simulation system includes a differentiable perception module, a dynamic model, and a policy network. The perception module generates dynamic scene decoupling data based on the input multimodal sensor data sequence. The dynamic scene decoupling data includes road structure, current state of traffic participants, and environmental conditions. The dynamic model generates future trajectories of traffic participants based on their current states and initial control commands. The policy network generates safety control commands based on road structure, future trajectories of traffic participants, and environmental conditions. The phased optimization module 130 is used to freeze the policy network, pre-train the differentiable world model simulation system using the first training data; unfreeze the policy network, use the second training data, and adopt a course learning strategy to perform end-to-end joint optimization of the differentiable world model simulation system; and use the third training data to perform security reinforcement training on the differentiable world model simulation system.

[0102] In some embodiments, the perception module includes a neural field construction module and a dynamic scene decoupler; the neural field construction module is used to construct a spatiotemporally aligned multimodal neural field based on the input multimodal sensor data sequence; the dynamic scene decoupler is used to extract equivariant features from the multimodal neural field using a three-dimensional Euclidean group equivariant convolutional neural network, and generate structured dynamic scene decoupling data based on the extracted equivariant features.

[0103] In some embodiments, the policy network includes a high-level decision-maker and a low-level controller; the high-level decision-maker is used to predict the intentions of traffic participants based on their future trajectories to obtain their intentions; the low-level controller is used to perform differential flat spatial projection based on the traffic participants' intentions to obtain the vehicle's initial trajectory parameters, apply physical and rule constraints to the initial trajectory parameters based on preset dynamic constraint information and road structure to obtain the original control command, and perform risk field modulation on the original control command based on the environmental conditions, risk field gradient, and weights of the traffic participants' intentions output by the perception module to obtain the safety control command.

[0104] In some embodiments, the dynamic model is used to construct input data based on the current state of the traffic participant and the initial control command, input the input data into a pre-constructed neural ordinary differential equation to obtain the state derivative, and use a specified solver to integrate the state derivative over a specified time interval to obtain the future trajectory of the traffic participant.

[0105] In some embodiments, during pre-training, the perception loss and dynamics loss are combined to update the parameters of the perception module and the dynamics model, but the parameters of the policy network are not updated; during end-to-end joint optimization, the simulation loss, the real vehicle loss, and the consistency loss are combined to update the parameters of the perception module, the dynamics model, and the policy network; during safety enhancement training, the collision loss and the comfort loss are combined to update the parameters of the perception module, the dynamics model, and the policy network.

[0106] In some embodiments, the perception loss is calculated based on the intersection-union ratio between the predicted semantic segmentation result and the ground truth semantic segmentation result, and the Euclidean distance between the predicted traffic participant position and the ground truth traffic participant position; the dynamics loss is calculated based on the mean square error between the predicted state change rate and the ground truth state change rate; the simulation loss is calculated based on the Euclidean distance between the predicted trajectory and the ground truth trajectory in the simulation environment, and the Euclidean distance between the predicted control command and the ground truth control command; the real vehicle loss is calculated based on the real vehicle intervention frequency and the vehicle longitudinal acceleration change rate; the consistency loss is calculated based on the maximum mean difference between the simulation feature distribution and the real vehicle feature distribution; the collision loss is calculated based on the minimum distance between the vehicle and the nearest obstacle; and the comfort loss is calculated based on the vehicle longitudinal acceleration change rate and the steering wheel angular velocity.

[0107] In some embodiments, during the backpropagation process of end-to-end joint optimization, the vehicle Jacobian matrix is ​​calculated on the neural ordinary differential equation of the dynamic model by automatic differentiation, an attention diagonal matrix is ​​constructed based on the interaction attention weights of the vehicle and other traffic participants, a target gradient is calculated based on the vehicle Jacobian matrix and the attention diagonal matrix, and gradient information for updating the parameters of the perception module, the dynamic model and the policy network is generated based on the target gradient.

[0108] Specific limitations regarding the autonomous driving training device based on a differentiable world model simulation system can be found in the above-mentioned limitations on the autonomous driving training method based on a differentiable world model simulation system, and will not be repeated here. Each module in the aforementioned autonomous driving training device based on a differentiable world model simulation system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0109] This application also provides a computer device. In some embodiments, the computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can implement the steps of the autonomous driving training method based on the differentiable world model simulation system provided in any of the above embodiments.

[0110] In some embodiments, the internal structure diagram of a computer device may be as follows: Figure 8 As shown, the computer device includes a processor, memory, and network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores training data, dynamic constraint information, and other data; the specific data stored can also be found in the limitations defined in the above method embodiments. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an autonomous driving training method based on a differentiable world model simulation system.

[0111] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0112] This application also provides a computer-readable storage medium, in some embodiments of which a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps of the autonomous driving training method based on the differentiable world model simulation system provided in any of the above embodiments.

[0113] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0114] Those skilled in the art will understand that implementing all or part of the processes in the above method embodiments can be accomplished by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink, DRAM (SLDRAM), memory bus, direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0115] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0116] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An autonomous driving training method based on a differentiable world model simulation system, characterized in that, The method includes: Training data is constructed based on pre-collected multimodal sensor data; the training data includes first training data, second training data, and third training data; the first training data is a sequence of multimodal sensor data continuously collected in a static scene or a simulation environment containing only simple dynamic objects moving at a constant speed; the second training data is a sequence of multimodal sensor data continuously collected in a dynamic urban scene or highway scene containing traffic lights, intersections, and adversarial traffic flow; the third training data is a sequence of multimodal sensor data continuously collected in a high-adversarial scene containing sudden braking, rapid cutting in, and oncoming vehicles, as well as under extreme conditions such as heavy rain, night, and dense fog. A differentiable world model simulation system is constructed. This system includes a differentiable perception module, a dynamics model, and a policy network. The perception module generates dynamic scene decoupling data based on input multimodal sensor data sequences. This dynamic scene decoupling data includes road structure, current state of traffic participants, and environmental conditions. The dynamics model generates future trajectories of traffic participants based on their current states and initial control commands. The policy network generates safety control commands based on the road structure, future trajectories, and environmental conditions. The policy network includes a high-level decision-maker and a low-level controller. The high-level decision-maker predicts the intentions of traffic participants based on their future trajectories. The low-level controller performs differential flat space projection based on the participants' intentions to obtain initial trajectory parameters. It then applies physical and rule constraints to these initial trajectory parameters based on preset dynamics constraints and the road structure to obtain original control commands. Finally, it modulates the original control commands using the environmental conditions, risk field gradient, and weights of the participants' intentions output by the perception module to obtain safety control commands. The policy network is frozen, and the differentiable world model simulation system is pre-trained using the first training data; the policy network is unfrozen, and the differentiable world model simulation system is jointly optimized end-to-end using the second training data and a course learning strategy; the differentiable world model simulation system is then subjected to safety enhancement training using the third training data; during the pre-training, the perception loss and dynamics loss are combined to update the parameters of the perception module and the dynamics model, but the parameters of the policy network are not updated; during the end-to-end joint optimization, the simulation loss, the real vehicle loss, and the consistency loss are combined to update the parameters of the perception module, the dynamics model, and the policy network; during the safety enhancement training, the collision loss and the comfort loss are combined to update the parameters of the perception module, the dynamics model, and the policy network.

2. The method according to claim 1, characterized in that, The perception module includes a neural field construction module and a dynamic scene decoupler; the neural field construction module is used to construct a spatiotemporally aligned multimodal neural field based on the input multimodal sensor data sequence; the dynamic scene decoupler is used to extract equivariant features from the multimodal neural field using a three-dimensional Euclidean group equivariant convolutional neural network, and generate structured dynamic scene decoupling data based on the extracted equivariant features.

3. The method according to claim 1, characterized in that, The dynamic model is used to construct input data based on the current state of the traffic participant and the initial control command. The input data is then input into a pre-constructed neural ordinary differential equation to obtain the state derivative. A specified solver is used to integrate the state derivative over a specified time interval to obtain the future trajectory of the traffic participant.

4. The method according to claim 1, characterized in that, The perception loss is calculated based on the intersection-union ratio (IU) between the predicted semantic segmentation result and the ground truth semantic segmentation result, and the Euclidean distance between the predicted traffic participant position and the ground truth traffic participant position; the dynamics loss is calculated based on the mean square error between the predicted state change rate and the ground truth state change rate; the simulation loss is calculated based on the Euclidean distance between the predicted trajectory and the ground truth trajectory in the simulation environment, and the Euclidean distance between the predicted control command and the ground truth control command; the real vehicle loss is calculated based on the real vehicle intervention frequency and the vehicle longitudinal acceleration change rate; the consistency loss is calculated based on the maximum mean difference between the simulation feature distribution and the real vehicle feature distribution; the collision loss is calculated based on the minimum distance between the vehicle and the nearest obstacle; and the comfort loss is calculated based on the vehicle longitudinal acceleration change rate and the steering wheel angular velocity.

5. The method according to claim 1, characterized in that, During the backpropagation process of the end-to-end joint optimization, the vehicle Jacobian matrix is ​​calculated on the neural ordinary differential equation of the dynamic model by automatic differentiation. An attention diagonal matrix is ​​constructed based on the interaction attention weights of the vehicle and other traffic participants. The target gradient is calculated based on the vehicle Jacobian matrix and the attention diagonal matrix. Gradient information for updating the parameters of the perception module, the dynamic model and the policy network is generated based on the target gradient.

6. An autonomous driving training system based on a differentiable world model simulation system, characterized in that, The system includes: The training data construction module is used to construct training data based on pre-collected multimodal sensor data. The training data includes first training data, second training data, and third training data. The first training data is a sequence of multimodal sensor data continuously collected in a static scene or a simulation environment containing only simple dynamic objects moving at a constant speed. The second training data is a sequence of multimodal sensor data continuously collected in a dynamic urban scene or highway scene containing traffic lights, intersections, and adversarial traffic flow. The third training data is a sequence of multimodal sensor data continuously collected in a high-adversarial scene containing sudden braking, rapid cutting in, and oncoming vehicles, as well as under extreme conditions such as heavy rain, night, and dense fog. A world model simulation system construction module is used to construct a differentiable world model simulation system. The differentiable world model simulation system includes a differentiable perception module, a dynamic model, and a policy network. The perception module generates dynamic scene decoupling data based on input multimodal sensor data sequences. The dynamic scene decoupling data includes road structure, current state of traffic participants, and environmental conditions. The dynamic model generates future trajectories of traffic participants based on their current states and initial control commands. The policy network generates safety control commands based on the road structure, future trajectories, and environmental conditions. The policy network includes a high-level decision-maker and a low-level controller. The high-level decision-maker predicts the intentions of traffic participants based on their future trajectories. The low-level controller performs differential flat space projection based on the traffic participants' intentions to obtain initial trajectory parameters. It then applies physical and rule constraints to these initial trajectory parameters based on preset dynamic constraints and the road structure to obtain original control commands. Finally, it modulates the original control commands based on the environmental conditions, risk field gradient, and the weights of the traffic participants' intentions output by the perception module to obtain safety control commands. The phased optimization module is used to freeze the policy network and pre-train the differentiable world model simulation system using the first training data; unfreeze the policy network and perform end-to-end joint optimization of the differentiable world model simulation system using the second training data and a course learning strategy; perform safety reinforcement training of the differentiable world model simulation system using the third training data; during the pre-training, the perception loss and dynamics loss are combined to update the parameters of the perception module and the dynamics model, but the parameters of the policy network are not updated; during the end-to-end joint optimization, the simulation loss, real vehicle loss, and consistency loss are combined to update the parameters of the perception module, the dynamics model, and the policy network; during the safety reinforcement training, the collision loss and comfort loss are combined to update the parameters of the perception module, the dynamics model, and the policy network.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

Images