Differential world model simulation system-based automatic driving training method and system, computer equipment and medium

Abstract

The invention relates to an automatic driving training method and system based on a differentiable world model simulation system, computer equipment and a medium. The method comprises the following steps: constructing the differentiable world model simulation system; the differentiable world model simulation system comprises a differentiable sensing module, a dynamic model and a strategy network; freezing the strategy network, and pre-training the differentiable world model simulation system by using the first training data; unfreezing the strategy network, and performing end-to-end joint optimization on the differentiable world model simulation system by utilizing the second training data and adopting a course learning strategy; and carrying out safety strengthening training on the differentiable world model simulation system by utilizing the third training data. According to the method, the full link is automatically driven, namely, perception, prediction and planning end-to-end differential is achieved, the training efficiency is improved, and the generalization of a long-tail scene is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of autonomous driving, in particular to an autonomous driving training method and system based on a differentiable world model simulation system, a computer device and a medium. BACKGROUND

[0002] The current training of autonomous driving systems mainly relies on the following technical paths: (1) Modular training framework: decoupling perception, prediction and planning modules, training them separately and then integrating them; (2) Reinforcement learning scheme: based on a non-differentiable world model simulation system (such as a traditional simulator), optimizing the driving strategy through trial and error interaction; (3) Imitation learning scheme: training the strategy network through expert demonstration data, but lacking the ability to dynamically model the environment.

[0003] The above technical paths have the following disadvantages: The non-differentiability of the environment state to the vehicle action leads to the inability to optimize the decision-control link end-to-end, the sampled reinforcement learning is difficult to cover low-probability dangerous scenarios (such as emergency obstacle avoidance scenario coverage <3%), and the existing world model simulation system has poor dynamics rationality in predicting trajectories (such as collision false alarm rate >15%), in addition, the above technical paths require the use of more than one million kilometers of road test data for training, which is low in training efficiency. SUMMARY

[0004] In view of the above deficiencies or shortcomings, the present application provides an autonomous driving training method and system based on a differentiable world model simulation system, a computer device and a medium. The method can realize end-to-end differentiation of the autonomous driving full link (perception → prediction → planning), improve the training efficiency and improve the generalization of long-tail scenarios.

[0005] According to a first aspect, the present application provides an autonomous driving training method based on a differentiable world model simulation system, the method comprising: constructing training data based on pre-acquired multi-modal sensor data; the training data comprising first training data, second training data and third training data; constructing a differentiable world model simulation system; the differentiable world model simulation system comprising a differentiable perception module, a dynamics model and a strategy network; the perception module is used to generate dynamic scene decoupling data according to the input multi-modal sensor data sequence; the dynamic scene decoupling data comprises road structure, current state of traffic participants and environmental conditions; the dynamics model is used to generate future trajectories of traffic participants according to the current state of traffic participants and initial control instructions; the strategy network is used to generate safe control instructions according to the road structure, the future trajectories of traffic participants and the environmental conditions; The strategy network is frozen, the differentiable world model simulation system is pre-trained by using first training data; the strategy network is unfrozen, the differentiable world model simulation system is optimized end-to-end by using second training data and a curriculum learning strategy; and the differentiable world model simulation system is safety reinforced by using third training data.

[0006] In some embodiments, the perception module comprises a neural field construction module and a dynamic scene decoupler; the neural field construction module is configured to construct a spatio-temporally aligned multi-modal neural field according to an input multi-modal sensor data sequence; and the dynamic scene decoupler is configured to extract equivariant features from the multi-modal neural field using a three-dimensional Euclidean group equivariant convolutional neural network, and generate structured dynamic scene decoupled data according to the extracted equivariant features.

[0007] In some embodiments, the strategy network comprises a high-level decision maker and a low-level controller; the high-level decision maker is configured to perform intention prediction according to a future trajectory of a traffic participant, and obtain an intention of the traffic participant; and the low-level controller is configured to perform differential flatness space projection according to the intention of the traffic participant, and obtain preliminary trajectory parameters of a host vehicle, impose physical and rule constraints on the preliminary trajectory parameters according to preset dynamic constraint information and road structure, and obtain original control instructions, and perform risk field modulation on the original control instructions according to weights of environmental conditions, risk field gradients and the intention of the traffic participant output by the perception module, and obtain safety control instructions.

[0008] In some embodiments, the dynamic model is configured to construct input data according to a current state of the traffic participant and initial control instructions, input the input data into a pre-constructed neural ordinary differential equation, obtain a state derivative, and use a specified solver to integrate the state derivative within a specified time interval, and obtain a future trajectory of the traffic participant.

[0009] In some embodiments, during pre-training, a perception loss and a dynamic loss are combined to update parameters of the perception module and the dynamic model, and parameters of the strategy network are not updated; during end-to-end joint optimization, a simulation loss, a real vehicle loss and a consistency loss are combined to update parameters of the perception module, the dynamic model and the strategy network; and during safety reinforcement training, a collision loss and a comfort loss are combined to update parameters of the perception module, the dynamic model and the strategy network.

[0010] In some embodiments, the perception loss is calculated according to an intersection over union between a predicted semantic segmentation result and a ground truth semantic segmentation result and an Euclidean distance between a predicted traffic participant position and a ground truth traffic participant position; the dynamics loss is calculated according to a mean square error between a predicted state rate of change and a ground truth state rate of change; the simulation loss is calculated according to an Euclidean distance between a predicted trajectory and a ground truth trajectory in a simulation environment and an Euclidean distance between a predicted control instruction and a ground truth control instruction; the real vehicle loss is calculated according to a real vehicle intervention frequency and a vehicle longitudinal acceleration rate of change; the consistency loss is calculated according to a maximum mean difference between a simulation feature distribution and a real vehicle feature distribution; the collision loss is calculated according to a minimum distance value between the ego vehicle and a nearest obstacle; and the comfort loss is calculated according to a vehicle longitudinal acceleration rate of change and a steering wheel angular velocity.

[0011] In some embodiments, in the back propagation process of the end-to-end joint optimization, the ego vehicle Jacobian matrix is calculated on the neural differential equation of the dynamics model through automatic differentiation, the attention diagonal matrix is constructed according to the interaction attention weights of the ego vehicle and other traffic participants, the target gradient is calculated according to the ego vehicle Jacobian matrix and the attention diagonal matrix, and the gradient information for updating the parameters of the perception module, the dynamics model and the policy network is generated respectively according to the target gradient.

[0012] The application provides an automatic driving training system based on a differentiable world model simulation system according to a second aspect, and the system comprises: a training data construction module configured to construct training data based on pre-acquired multi-modal sensor data; the training data comprises first training data, second training data and third training data; a world model simulation system construction module configured to construct a differentiable world model simulation system; the differentiable world model simulation system comprises a differentiable perception module, a dynamics model and a policy network; the perception module is configured to generate dynamic scene decoupling data according to an input multi-modal sensor data sequence; the dynamic scene decoupling data comprises road structure, traffic participant current state and environmental condition; the dynamics model is configured to generate traffic participant future trajectory according to the traffic participant current state and initial control instruction; and the policy network is configured to generate safe control instruction according to the road structure, the traffic participant future trajectory and the environmental condition; a staged optimization module configured to freeze the policy network, pre-train the differentiable world model simulation system using the first training data, unfreeze the policy network, perform end-to-end joint optimization on the differentiable world model simulation system using the second training data and adopting a curriculum learning strategy, and perform safe reinforcement training on the differentiable world model simulation system using the third training data.

[0013] The application provides a computer readable storage medium according to a third aspect, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the automatic driving training method based on the differentiable world model simulation system in any one of the above embodiments.

[0014] The application provides a computer device according to a fourth aspect, and the computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the automatic driving training method based on the differentiable world model simulation system in any one of the above embodiments when executed.

[0015] The above embodiments of the application construct training data including first training data, second training data and third training data based on pre-collected multi-modal sensor data; then, a differentiable world model simulation system is constructed; the differentiable world model simulation system comprises a differentiable perception module, a dynamics model and a policy network; the perception module is used to generate dynamic scene decoupling data according to an input multi-modal sensor data sequence; the dynamic scene decoupling data comprises road structure, current state of a traffic participant and environmental conditions; the dynamics model is used to generate a future trajectory of the traffic participant according to the current state of the traffic participant and an initial control instruction; the policy network is used to generate a safe control instruction according to the road structure, the future trajectory of the traffic participant and the environmental conditions; finally, the differentiable world model simulation system is optimized in stages, the first stage comprises freezing the policy network, and the differentiable world model simulation system is pre-trained by using the first training data; the second stage comprises unfreezing the policy network, and the differentiable world model simulation system is end-to-end jointly optimized by using the second training data and adopting a curriculum learning strategy; the third stage comprises performing safe reinforcement training on the differentiable world model simulation system by using the third training data. The embodiments of the application convert discrete decision-making links in a traditional pipeline into a continuous optimization problem by using the differentiable world model simulation system architecture, provide a new paradigm for automatic driving system training, can realize end-to-end differentiation of an automatic driving full link (perception→prediction→planning), improve training efficiency and improve generalization of long-tail scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A flowchart of an automatic driving training method based on a differentiable world model simulation system in one or more embodiments of the application; Figure 2 An architecture schematic diagram of a differentiable world model simulation system in one or more embodiments of the application; Figure 3 A motion prediction schematic diagram of a neural ODE in one or more embodiments of the application; Figure 4An architecture schematic diagram of a policy network in one or more embodiments of the present application; Figure 5 A schematic diagram of a differential flatness control transformation principle in one or more embodiments of the present application; Figure 6 A virtual-real gradient alignment process schematic diagram in one or more embodiments of the present application; Figure 7 A structure schematic diagram of an automatic driving training system based on a differentiable world model simulation system in one or more embodiments of the present application; Figure 8 An internal structure schematic diagram of a computer device in one or more embodiments of the present application. DETAILED DESCRIPTION

[0017] In order to make the purposes, technical solutions and advantages of the present application clearer, the following will further describe the embodiments of the present application with reference to the accompanying drawings. It should be clear that the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0018] The following description refers to the accompanying drawings. Unless otherwise indicated, same numbers in different drawings indicate same or similar elements. The implementations described in the following example embodiments are not meant to represent all implementations consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0019] In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not necessarily mean a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, "multiple" means two or more, unless otherwise specified. "And / or", which describes the relationship between the associated objects, means that there can be three relationships, for example, A and / or B can mean that there are three cases of A alone, A and B together, and B alone. The character " / " generally represents that the associated objects before and after are in an "or" relationship.

[0020] To address the deficiencies or shortcomings of the prior art, the present application provides an automatic driving training method based on a differentiable world model simulation system, which converts the discrete decision-making link in the traditional pipeline into a continuous optimization problem through a differentiable world model simulation system architecture, provides a new paradigm for automatic driving system training, can realize end-to-end differentiation of the full link (perception → prediction → planning) of automatic driving, and improves the training efficiency and the generalization of long-tail scenarios.

[0021] In some exemplary embodiments of the present application, the method includes the steps as shown in Figure 1 The various steps are described in detail below.

[0022] S110: constructing training data based on pre-acquired multi-modal sensor data; the training data includes first training data, second training data and third training data.

[0023] In the automatic driving system based on the differentiable world model simulation system, the multi-modal sensor data refers to the original data collected by multiple sensors. These sensors can include LiDAR (Light Laser Detection and Ranging), camera, millimeter wave radar, IMU (Inertial Measurement Unit), and GPS (or RTK). GPS refers to Global Positioning System, RTK refers to Real-time kinematic, which is a real-time dynamic carrier phase difference technology. The multi-modal sensor data can include 3D (three-dimensional) point cloud sequence collected by LiDAR (the sampling frequency can be 10 to 20 Hz, which can be used for high-precision geometric modeling and obstacle detection), multi-view RGB (Red-Green-Blue) image sequence collected by camera (the sampling frequency can be 30 to 60 Hz, which can be used for semantic segmentation and traffic sign recognition), 4D (four-dimensional) point cloud collected by millimeter wave radar (the sampling frequency can be 10 to 25 Hz, which can be used for moving object tracking and speed measurement), 6-axis inertial measurement data collected by IMU (the sampling frequency can be 100 to 1000 Hz, which can be used for vehicle pose estimation and motion compensation), and high-precision global positioning data collected by GPS or RTK (the sampling frequency can be 5 to 10 Hz, which can be used for global path planning and map matching). Among them, the world model simulation system is a key component of the world model, which is specifically a state transition model capable of predicting the future environment state distribution based on the current environment data and the current policy data.

[0024] The training data is a data stream obtained by slicing (such as generating a 5-frame sequence according to a 100-millisecond sliding window), time synchronization, spatial alignment, labeling and a series of processes on multimedia sensor data, that is, a multi-modal sensor data sequence.

[0025] The multi-modal sensor data can be real vehicle data extracted from normal driving segments of a real vehicle in various scenes, or simulation data generated by a simulation platform.

[0026] The first training data, the second training data and the third training data are training data used in different training stages, and they are all multi-modal sensor data sequences. However, different training stages use multi-modal sensor data of different scenes and different sources to construct the training data. Exemplarily, the first training data can be a multi-modal sensor data sequence continuously collected in a static scene or a simulation environment containing only simple dynamic objects moving at a constant speed, the second training data can be a multi-modal sensor data sequence continuously collected in a dynamic urban scene or a high-speed scene containing traffic lights, intersections and traffic flow, 90% of which can be from a high-fidelity simulation platform rendered in real time, and 10% of which can be from a desensitized real vehicle log; the third training data can be a multi-modal sensor data sequence continuously collected in a high-adversarial scene containing sudden braking, rapid cutting and reverse vehicles, and in extreme conditions such as heavy rain, night and heavy fog, 95% of which can be reconstructed based on real accident data or generated by a GAN, and 5% of which can be from a real vehicle extreme condition segment.

[0027] S120: constructing a differentiable world model simulation system; the differentiable world model simulation system includes a differentiable perception module, a dynamics model and a strategy network.

[0028] The differentiable world model simulation system refers to an automatic driving world model simulation system in which all computing units are continuous and differentiable functions, and can support end-to-end backpropagation from original sensor input to final control instruction. It integrates the traditional four discrete modules of “perception-prediction-planning-control” into three differentiable sub-networks (i.e., perception, dynamics model and strategy network), and realizes full-link gradient flow through automatic differentiation (Autograd). Compared with the related art in which the perception, prediction and planning modules are decoupled, trained separately and then integrated, the training efficiency and strategy safety of the world model simulation system in complex scenes can be effectively improved. Figure 2 The figure shows an architectural schematic diagram of the differentiable world model simulation system, and the following will be described in combination with Figure 2 The differentiable world model simulation system is described.

[0029] The perception module generates dynamic scene decoupling data based on the input multimodal sensor data sequence; this data includes road structure, current state of traffic participants, and environmental conditions. The dynamic model generates future trajectories of traffic participants based on their current states and initial control commands. The policy network generates safety control commands based on road structure, future trajectories of traffic participants, and environmental conditions.

[0030] In some embodiments, the perception module includes a neural field construction module and a dynamic scene decoupler.

[0031] The neural field construction module is used to construct a spatiotemporally aligned multimodal neural field based on the input multimodal sensor data sequence. For example, ego-motion interpolation can be performed on RGB images acquired by a camera and 4D point clouds acquired by radar, using a LiDAR frame as the reference clock, to unify these data into the current LiDAR coordinate system. Furthermore, 6-DoF pose data acquired by an IMU can be used for compensation to eliminate ghosting of the vehicle's motion. For RGB images, distortion correction can be performed, such as using intrinsic parameters and distortion coefficients to undistort each frame, resulting in a distortion-free RGB image. Next, voxel rasterization is performed, for example, establishing a 256×256×32 voxel grid with a resolution of 0.1m. The fill channels can include: occupancy probability (LiDAR point density), reflection intensity (LiDAR), Doppler velocity (Radar interpolation), and RGB features (which can be obtained by combining camera backprojection and trilinear interpolation). Then, the voxel features are temporally aligned. For example, the voxel features from the most recent k=5 frames (0.1-second intervals) are concatenated along the time dimension to obtain a 4D tensor. This is then compressed using a 1×1×1×3 convolution to obtain the final tensor F∈ ^(B,C,X,Y,Z), where B refers to the batch size, C refers to the number of channels, and "X,Y,Z" are the spatial dimensions. The final tensor F(x,y,z,t) is the spatiotemporally aligned multimodal neural field.

[0032] The dynamic scene decoupler is used to extract equivariant features from a multimodal neural field using a 3D Euclidean group equivariant convolutional neural network (SE(3)-Equivariant CNN), and generate structured dynamic scene decoupling data based on the extracted equivariant features. Among them, the equivariant features have rotation and translation invariance. For example, the state representation of the vehicle point cloud remains consistent regardless of the viewing angle.

[0033] Dynamic scene decoupler (which can be represented as) The input is a spatiotemporally aligned multimodal neural field.

[0034] A multimodal neural field can be represented as F(x,y,z,t), where x,y,z,t represent the following data: (1) LiDAR point cloud can be a geometric structure in three-dimensional space (such as voxelization or original point cloud). (2) The RGB image from the camera can be generated by cross-modal fusion of RGB features (such as semantic information from the BEV perspective); BEV refers to the bird's-eye view. (3) Radar point cloud data, including position information and velocity information (such as Doppler radar point cloud); (4) Timestamps can ensure the temporal correlation of consecutive frames (such as optical flow or scene flow).

[0035] The output of the dynamic scene decoupler is the semantic and state decomposition of the dynamic scene, which consists of three core elements: (1) Road structure (static environment) can be a vector representation of lane lines, curbs and traffic signs. Its output form can be a BEV segmentation map (i.e., (H×W) grid, where each pixel corresponds to a semantic category). (2) Current state of traffic participants (dynamic objects), which can be the current state of traffic participants such as vehicles, pedestrians, and cyclists (such as position, speed, acceleration and heading angle); its output form can be a structured list (where each traffic participant contains 6DoF pose and motion state); (3) Environmental conditions (which can be light and / or weather), which can be quantitative parameters of light intensity, rain, snow and fog (such as visibility coefficient), and the output form can be scalar or low-dimensional vector (such as weather influence factors in the interval [0,1]).

[0036] The dynamic scene decoupler inputs the multimodal neural field into a three-dimensional Euclidean group equivariant convolutional neural network (SE(3)-Equivariant CNN, where the convolutional kernels of each layer can be 64→128→256→512 respectively), and obtains the equivariant feature map output by the three-dimensional Euclidean group equivariant convolutional neural network. The equivariant feature map can be represented as Φ, where Φ∈ ^{B,512,X,Y,Z} ensures feature consistency after rotation or translation transformation; next, a 1×1×1 convolution is performed on the isomorphic feature map to obtain a three-way parallel tensor, i.e.: (1) Φ_sem, i.e. semantic logic (7 categories); (2) Φ_inst, i.e. instance embedding (32-dimensional); (3) Φ_motion, i.e., motion vector (v_x,v_y,v_z).

[0037] Semantic decoding of Φ_sem yields the road structure (specifically, the road structure raster, represented as road_structure); instance segmentation of Φ_inst produces a list of traffic participant IDs (including the IDs of each traffic participant); motion estimation is performed on each traffic participant ID based on Φ_motion to obtain a 6-DoF pose sequence (i.e., the current state of the traffic participant) for each ID; and global average pooling is applied to the isotropic feature map to obtain the environmental conditions (represented as environment); the environmental conditions can be a three-dimensional vector, such as [light, rain, fog]∈[0,1]. Finally, the road structure, the current state of traffic participants, and the environmental conditions can be output in JSON format, for example, {road_structure, agents, environment}.

[0038] In some embodiments, the dynamic model is used to construct input data based on the current state of the traffic participant and the initial control command, input the input data into a pre-constructed neural ordinary differential equation to obtain the state derivative, and use a specified solver to integrate the state derivative over a specified time interval to obtain the future trajectory of the traffic participant.

[0039] The dynamic model introduces a continuous-time neural ODE (i.e., neural frequent differential equation). The neural ODE can model continuously (which can overcome the error accumulation of discrete time steps, especially suitable for high-frequency control), can realize interactive perception (specifically, it can learn the interaction of complex multi-agents, such as the game behavior between vehicles), and can also achieve physical consistency (i.e., integrate explicit dynamic priors, such as bicycle models, with data-driven complements). Figure 3 The diagram shown below illustrates the motor prediction of neural ODEs. Figure 3 Explain the neural ODE.

[0040] The neural ODE is modeled as follows: ; The physical meaning and mathematical definition of all parameters in the above formula are explained below: This refers to the current state of a traffic participant, representing the state vector of the traffic participant at time (t), and its dimension (d) is typically [missing value]. (i.e., basic 6DoF + extended features), specifically including: Pose (3DoF): (i.e., horizontal position and heading angle); Motion status (3DoF): (i.e., linear velocity and angular velocity).

[0041] Extended features ( Dimension: can be acceleration (represented as...) ), steering angle (expressed as One or more of the following: interaction features (such as attention weights).

[0042] This refers to the initial control command, which is specifically the control input to the autonomous vehicle (or ego vehicle, self-driving vehicle) (it can be the output of a policy network or from expert data). Its form can be: Longitudinal control: acceleration (which can be expressed as) , ); Lateral control: Front wheel steering angle (which can be expressed as...) , ); It can be represented as: (Two-dimensional vector).

[0043] This represents the dynamic function, which signifies the rate of change of state in a neural network model (continuous-time dynamics). Parameters Represents the weights of a neural network (such as an MLP or Transformer). The inputs are the current state of the traffic participants and the initial control commands, and the output is the state derivative (expressed as...). ).

[0044] After obtaining the state derivative, a specified solver (such as a 5th-order implicit Runge-Kutta solver) can be used to integrate the state derivative over a specified time interval to obtain the future trajectory of the traffic participants.

[0045] In some embodiments, the policy network, also known as a hierarchical policy network, has an architecture that includes an input layer, a high-level decision network (also known as a high-level decision maker), a low-level control network (also known as a low-level controller), and an output layer. Figure 4 The diagram shown below illustrates the architecture of a policy network. The following section will discuss this in conjunction with... Figure 4 The various parts of the policy network are explained in detail.

[0046] (1) Input layer The sources of input data include: BEV (Bird's-eye View) semantic information: that is, static road structures (such as lane lines, curbs, traffic signs, etc.). Instance sports field: that is, the current state of dynamic traffic participants (including position, speed, heading angle).

[0047] The input layer preprocesses the input data, such as aligning and encoding multimodal sensor data (e.g., LiDAR point clouds, camera RGB image sequences) to convert them into a unified vector representation for subsequent network processing.

[0048] (2) High-level decision-making network The high-level decision network is used to predict the intentions of traffic participants based on their future trajectories, thus obtaining their intentions. The high-level decision network includes a graph construction module, a graph attention layer, and an intention classifier.

[0049] The graph construction module is used to model traffic scenarios as a graph structure. The nodes of the graph represent the state information (such as position and speed) of each traffic participant (e.g., ego vehicles, surrounding vehicles, pedestrians), while the edges represent the spatial relationships (such as distance and relative speed) between traffic participants. The input to the graph construction module is the output of the input layer, and the output of the graph construction module is a dynamically updated graph that reflects the interaction topology of the current traffic scenario.

[0050] The graph attention layer is used to identify key interactive objects in the graph output by the graph construction module. It calculates the importance weights between nodes in the graph through an attention mechanism (for example, an ego vehicle will pay more attention to a vehicle braking suddenly in front of it, while ignoring a stationary vehicle in the distance, thus giving the vehicle braking suddenly a higher importance weight than a stationary vehicle in the distance). The output of the graph attention layer is a weighted graph representation, which can highlight high-risk or highly relevant interactions by assigning importance weights to nodes.

[0051] The intent classifier is used to predict the intents of traffic participants based on the weighted graph output from the graph attention layer. Intent types can include changing lanes left or right, following another vehicle, emergency braking, turning at intersections, etc. The output of the intent classifier is the probability distribution of the intents of each traffic participant (e.g., "changing lanes left: 70%, following another vehicle: 30%").

[0052] (III) Lower-level control network The underlying control network is used to perform differential flat space projection based on the intentions of traffic participants to obtain the initial trajectory parameters of the vehicle. Based on the preset dynamic constraint information and road structure, physical and rule constraints are applied to the initial trajectory parameters to obtain the original control command. Based on the environmental conditions, risk field gradient and the weights of the traffic participants' intentions output by the perception module, the original control command is modulated by the risk field to obtain the safety control command.

[0053] The underlying control network includes a differential flat spatial projection module, a differentiable optimization layer, and a risk field modulation module.

[0054] The Differential Flat Space Projection module transforms the traffic participant intentions output by the high-level decision network into physically feasible trajectories. It leverages vehicle dynamics (such as position and heading angle, which fully describe vehicle state) to simplify complex control problems into path optimization in a flat space. Its output is the initial trajectory parameters of the ego vehicle (such as target position and velocity profile). The principle of differential flat control transformation can be found in [link to relevant documentation]. Figure 5 As shown.

[0055] The differentiable optimization layer applies physical and rule constraints to the initial trajectory parameters output by the differential flat space projection module. Specifically, the differentiable optimization layer applies physical constraints to the initial trajectory parameters based on preset dynamic constraint information (such as maximum acceleration and curvature limit information), thereby obtaining the original control commands (such as steering angle). Physical constraints refer to dynamic constraints, such as maximum acceleration and curvature limits (to avoid sharp turns). The differentiable optimization layer further applies rule constraints to the physically constrained initial trajectory parameters based on the road structure output by the dynamic scene decoupler. Rule constraints refer to traffic rules, such as lane keeping and traffic light compliance. The differentiable optimization layer specifically ensures that these constraints are not violated through differentiable operations (such as gradient clipping).

[0056] The risk field modulation module is used to dynamically adjust the safety of the original control commands output by the differentiable optimization layer. The risk field modulation module can modulate the risk field based on the original control commands, the risk field gradient (used to identify the degree of danger of obstacles around the vehicle), and the weights of traffic participant intentions output by the high-level decision network (such as "lane change" intentions requiring higher lateral action sensitivity), thereby obtaining safe control commands (i.e., vehicle control commands constrained by safety boundaries).

[0057] In some embodiments, risk field modulation can be implemented using the following formula, which dynamically adjusts the original control commands output by the differentiable optimization layer. Through risk field gradient weighting, it ensures that the actions of the final output control commands conform to the safety boundary.

[0058] ; The parameters in the formula are explained below: This refers to safety control commands, i.e., control commands that have been modified for safety (such as throttle and steering angle). Its dimensions can be two, namely longitudinal acceleration and lateral steering angle.

[0059] This refers to the policy network (with parameters as follows) According to the current status of traffic participants ( The raw control commands generated may be beyond the safe limits.

[0060] For example, longitudinal: acceleration ( m / s 2 Lateral: Steering angle ( ).

[0061] This refers to the risk field ( ) for state ( The gradient of the surrounding environment can indicate the direction and intensity of danger.

[0062] Risk field ( The definition of ) is as follows: Dynamic risk: inversely proportional to the distance and relative speed of traffic participants (e.g., the closer the vehicle in front, the higher the risk value).

[0063] Static risks: distance to road boundaries and obstacles (risk increases when deviating from the lane).

[0064] It can reflect the direction of the fastest change in danger, and can be calculated using the following formula: ; This refers to the gradient norm, which represents the magnitude of the risk gradient and is used to quantify the overall degree of danger in the current state. A larger value indicates that even slight changes in action can lead to a sharp increase in risk (such as when approaching an obstacle). It can be calculated using the following formula: ; It is the Sigmoid function, which is used to compress the gradient norm to the [0,1] interval as a safety factor.

[0065] (iv) Output layer The output layer is used to control the vehicle based on control commands output from the underlying control network. These control commands can include two types: longitudinal control commands, such as throttle / brake commands (acceleration values), and lateral control commands, such as steering wheel angle (which can be calculated based on the trajectory curvature). The output layer can convert these control commands into vehicle CAN bus signals, which are then used to directly control the actuators.

[0066] S130: Freeze the policy network to pre-train the differentiable world model simulation system using the first training data; unfreeze the policy network to perform end-to-end joint optimization of the differentiable world model simulation system using the second training data and a course learning strategy; and perform security reinforcement training of the differentiable world model simulation system using the third training data.

[0067] This step is used to optimize the differentiable world model simulation system in stages, that is, to train the differentiable world model simulation system in stages one, two, and three in sequence. The training of each stage is explained below.

[0068] The first stage is the pre-training stage, which specifically involves pre-training the perception module and dynamics model in the differentiable world model simulation system. The training goal of this stage is to enable the perception module to learn the accurate mapping from multimodal sensor data to scene understanding, and to enable the dynamics model to master the motion patterns and interaction effects of traffic participants.

[0069] The first training data used in this stage may include: (1) simulation data of 100,000 sets of calibration scenarios (such as sunny and / or rainy days, different traffic densities) generated by the simulation platform; and (2) real vehicle data extracted from 5,000 kilometers of normal driving segments that have been collected in advance and from which sensitive information has been removed.

[0070] Data augmentation can be performed on the first training data. For example, randomly discarding 0-20% of the LiDAR point cloud sequence can simulate sensor occlusion, reflectivity attenuation, and hardware failure in real-world scenarios. Adding fog and haze noise to the RGB images from the camera can reproduce low visibility conditions such as rain, fog, and backlighting. This allows the model to learn invariant representations of missing and degraded inputs, thereby improving the generalization boundary of the subsequent second stage, i.e., end-to-end training. Furthermore, sunny, rainy, and constant-speed traffic scenarios do not adequately cover extreme weather and occlusion conditions. Therefore, through the above data augmentation strategies, the data density of edge scenarios can be artificially expanded without additional acquisition costs, reducing the dependence of the subsequent third stage, i.e., the safety enhancement stage, on rare events and achieving optimal allocation of training resources.

[0071] Before training, policy parameters need to be frozen, which means fixing the parameters of the policy network. Gradient truncation can also be performed by limiting the gradient norm of the perception module (e.g., setting max_grad_norm=1.0), so that only the parameters of the perception module and the dynamics model are updated, without updating the parameters of the policy network.

[0072] Regarding the settings for training parameters, they can be: Optimizer: AdamW (lr=3e-4); Batch Size: 32 (simulation) + 8 (real vehicles); Training duration: 50 epochs (approximately 72 hours); Hardware configuration: 8×A100 GPUs.

[0073] In addition, the validation metric for the perception module can be mIoU > 75% (BEV segmentation); the validation metric for the dynamic model can be predicted position error < 0.3m (1s horizon).

[0074] During pre-training, the perceptual loss and dynamic loss can be combined to update the parameters of the perceptual module and the dynamic model, without updating the parameters of the policy network. For example, the loss function used in this stage could be: ; The above loss function applies to perceived loss (i.e.) ) and dynamic loss (i.e. We perform a weighted summation to obtain the total loss (i.e., Finally, the parameters of the perception module and the dynamics model are updated based on the total loss. The weight values ​​of the perception loss and dynamics loss in the above formula are only examples, and their specific values ​​can be adjusted according to actual needs.

[0075] The perceptual loss described above is calculated based on the intersection-union ratio (IoU) between the predicted and ground truth semantic segmentation results, and the Euclidean distance between the predicted and ground truth traffic participant positions. The dynamic loss is calculated based on the mean square error between the predicted and ground truth rate of change.

[0076] For example, the perceptual loss can be calculated using the following formula: ; in, This refers to the perception loss. pred_seg refers to the BEV (Bird's-Eye-View) semantic segmentation prediction map output by the perception module, for example, with a size of 256×256×7, where each pixel represents the probability distribution of categories such as lane lines, vehicles, and pedestrians. Is with The ground truth semantic segmentation map of the same resolution can be derived from HD maps or self-supervised pseudo-labels. It is the center coordinate (x, y) of each traffic participant in the BEV plane predicted by the perception module. Is with The corresponding truth center coordinates can be given by the LiDAR truth box or instance annotation.

[0077] The kinetic loss can be calculated using the following formula: ; in, This refers to dynamic loss. It refers to the state derivative of the neural ODE output. This refers to the change in the true state, and the calculation method can be... That is, it is obtained by the difference of the true state of two consecutive frames.

[0078] The second stage is the end-to-end joint optimization stage. The training objective of this stage is to achieve end-to-end gradient propagation from the original sensor input to the final control command, thereby optimizing the overall policy. Before starting training, the policy network can be unfrozen by setting its learning rate to 10 times that of the sensor module, for example, setting lr_policy=3e-3. Then, a Hessian-Free optimizer is introduced to handle the non-convex loss surface.

[0079] This stage can employ a course-based learning strategy for training, where the scenario complexity and the proportion of adversarial vehicles can be increased according to the training progress. For example, the corresponding scenario complexity and adversarial vehicle proportions for different training progress are shown in Table 1: Table 1: The optimizer can be configured as follows: Hessian-Free: initial_trust_radius: 0.1; max_hessian_updates: 50; damping: 0.01; use_gauss_newton: True.

[0080] The convergence criterion for this stage can be to conduct 10 consecutive validation set evaluations. If the tracking error of the simulation scenario is less than 1.0 meter and the intervention rate of the real vehicle data is less than 5%, it is considered convergent; otherwise, it is considered non-converged.

[0081] When performing end-to-end joint optimization, simulation loss, real-vehicle loss, and consistency loss can be combined to update the parameters of the perception module, dynamics model, and policy network. For example, the following loss function can be used in this stage: ; The above loss function affects the simulation loss (i.e.) ), actual vehicle loss (i.e. ) and consistency loss (i.e. We perform a weighted summation to obtain the total loss (i.e., Finally, the parameters of the perception module, dynamics model, and policy network are updated based on the total loss. The weight values ​​of simulation loss, real vehicle loss, and consistency loss in the above formula are only examples, and their specific values ​​can be adjusted according to actual needs.

[0082] The above loss function can be used to achieve virtual-real gradient alignment. The virtual-real gradient alignment process can be found in [link to documentation]. Figure 6 As shown.

[0083] The simulation loss mentioned above is calculated based on the Euclidean distance between the predicted trajectory and the true trajectory in the simulation environment, as well as the Euclidean distance between the predicted control command and the true control command; the real vehicle loss is calculated based on the real vehicle intervention frequency and the rate of change of the vehicle's longitudinal acceleration; and the consistency loss is calculated based on the maximum mean difference between the simulation feature distribution and the real vehicle feature distribution.

[0084] For example, the simulation loss can be calculated using the following formula: ; in, This refers to simulation loss. It refers to the future trajectories of traffic participants (specifically, coordinate sequences) output by the dynamic model in the simulation environment. This refers to the true trajectory values ​​(also coordinate sequences) generated by expert strategies or HD-maps in a simulation environment, which are related to... Same dimension. This refers to the safety control commands (such as acceleration and steering wheel angle) output by the policy network in the simulation environment. It refers to the true value of the control instructions given by experts or rule controllers in the simulation environment.

[0085] The actual vehicle damage can be calculated using the following formula: ; in, This refers to actual vehicle damage. This refers to the ratio of the number of times a safety driver takes over the steering wheel or brake pedal during real-world vehicle testing to the total mileage driven. It refers to the first derivative of the vehicle's longitudinal acceleration with respect to time, and the unit can be m / s³. It is calculated differentially from the 100Hz signal of the IMU.

[0086] The consistency loss can be calculated using the following formula: ; in, This refers to the loss of consistency. It refers to the feature distribution extracted by the perception module from the simulation data (which is a set of samples, and the dimension can be 256). This refers to the feature distribution extracted by the perception module from real vehicle data, and... Same batch size.

[0087] In some embodiments, during the backpropagation process of end-to-end joint optimization, the vehicle Jacobian matrix is ​​calculated on the neural ordinary differential equation of the dynamic model by automatic differentiation, an attention diagonal matrix is ​​constructed based on the interaction attention weights of the vehicle and other traffic participants, a target gradient is calculated based on the vehicle Jacobian matrix and the attention diagonal matrix, and gradient information for updating the parameters of the perception module, the dynamic model and the policy network is generated based on the target gradient.

[0088] The dynamic model in this embodiment incorporates interactively-aware Jacobian propagation, and the relevant calculation formulas are as follows: ; The above formula is used to quantify the actions of the ego vehicle (i.e., the self-driving vehicle). ) regarding its own future state ( The model indirectly influences the states of surrounding traffic participants. During gradient backpropagation, interactive attention weights can be used to focus on key interaction objects, suppressing irrelevant vehicles and avoiding interference from unrelated participants. Therefore, during training, the model can automatically focus on key interaction scenarios (such as lane-changing games and emergency obstacle avoidance), improving the robustness of policies in complex scenarios. For example, when an ego vehicle turns, it only strengthens gradient updates for the states of nearby vehicles, ignoring distant vehicles.

[0089] For example, if the interaction attention weight matrix W is: ; During gradient propagation, 90% of the gradient of ego's own state is retained, only 10% for car A, and ignored for car B.

[0090] The parameters in the formula are explained below: This refers to the target gradient, representing the ego vehicle's action ( ) regarding its own future state ( The direct impact of ego on vehicle braking (i.e., How do changes affect its position, velocity, and other states? Its dimensions are ^d×m, where d is the state dimension and m is the action dimension. and From the policy network.

[0091] This refers to the Jacobian matrix of the vehicle, i.e., the dynamic function. The Jacobian matrix of the ego vehicle state reflects the state derivative ( ) for the current state ( The local sensitivity of ) has the following dimensions. ^d×d, where d is the state dimension. The dynamic function of the neural ODE can be obtained through automatic differentiation (Autograd). It is calculated using the following formula: ; It refers to the transposed Jacobian matrix, used for gradient backpropagation (chain rule). In backpropagation automatic differentiation, gradient propagation requires left multiplication by the Jacobian transpose. The upstream gradient (the gradient of the loss function with respect to the state) can be passed to the action space.

[0092] This refers to the attention diagonal matrix, whose dimensions and sum The dimensions are the same. The diagonal elements of the attention diagonal matrix are the interaction attention weights of the ego vehicle with other traffic participants. The higher the interaction attention weight of the ego vehicle with other traffic participants, the greater the influence of that traffic participant on the ego vehicle's state (e.g., a vehicle braking suddenly in front is more critical than a vehicle further away). These weights can be calculated by the interaction perception module (such as GNN or Transformer) using the following formula: ; This refers to the interaction attention weight between the ego vehicle and the j-th traffic participant. This refers to the query vector (Query) for ego vehicles. It refers to the key vector (Key) of the j-th traffic participant. This refers to the vector dimension scaling factor.

[0093] The third stage is the safety enhancement stage. The training objective of this stage is to improve the robustness of long-tail scenarios by training the model, which already has basic driving capabilities, through high-intensity and extreme scenario training.

[0094] This phase can introduce various types of confrontations, such as vehicles braking suddenly (e.g., deceleration of 8 m / s²), malicious cutting in (e.g., lateral speed of 3 m / s, cutting angle of 45°), and pedestrians running red lights (running speed of 2.5 m / s). The generation method can be: { def generate_adversarial(scene): if random() < 0.05: # 5% probability agent = choice(scene.vehicles) agent.apply_emergency_behavior() return scene }

[0095] In the training process, forward propagation can use a batch (referring to a set of samples used to train the model in one iteration) that mixes normal and adversarial scenarios. When calculating gradients, it is necessary to separate the gradients of normal samples and adversarial samples. When updating parameters, standard SGD can be used for normal samples, while adversarial gradients after PGD attack (ε=0.1) can be used for adversarial samples.

[0096] During safety enhancement training, collision loss and comfort loss can be combined to update the parameters of the perception module, dynamics model, and policy network. For example, the loss function used in this stage could be: ; The above loss function applies to collision loss (i.e.) ) and loss of comfort (i.e. We perform a weighted summation to obtain the total loss (i.e., Finally, the parameters of the perception module, dynamics model, and policy network are updated based on the total loss. The weight values ​​for collision loss and comfort loss in the above formula are only examples, and their specific values ​​can be adjusted according to actual needs.

[0097] The collision loss mentioned above is calculated based on the minimum distance between the vehicle and the nearest obstacle, while the comfort loss is calculated based on the vehicle's longitudinal acceleration rate of change and the steering wheel angular velocity.

[0098] For example, the collision loss can be calculated using the following formula: ; in, This refers to collision damage. This refers to the minimum distance between the vehicle and the nearest obstacle (vehicle, pedestrian, stationary object), measured in meters. In this formula, when... Penalties begin when the distance is less than 1.5 meters.

[0099] The loss of comfort can be calculated using the following formula: ; in, This refers to the loss of comfort. It refers to the first derivative of the vehicle's longitudinal acceleration with respect to time, i.e., the rate of change of acceleration, with units of m / s³, which can be obtained by differential analysis of the 100Hz signal from the IMU. It refers to the steering wheel angular velocity, which is the first derivative of the steering angle with respect to time, and is measured in rad / s. It can be obtained by differential analysis of the 100Hz vehicle bus signal.

[0100] It should be noted that, regarding the various steps included in the autonomous driving training method based on a differentiable world model simulation system provided in any of the above embodiments, unless explicitly stated herein, there is no strict order restriction on the execution of these steps; they can be executed in other orders. Furthermore, at least some of these steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is also not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0101] Based on the same inventive concept, this application also provides an autonomous driving training device based on a differentiable world model simulation system. In some embodiments, such as Figure 7 As shown, the autonomous driving training device based on the differentiable world model simulation system includes the following modules: The training data construction module 110 is used to construct training data based on pre-acquired multimodal sensor data; the training data includes first training data, second training data, and third training data; The world model simulation system construction module 120 is used to construct a differentiable world model simulation system. The differentiable world model simulation system includes a differentiable perception module, a dynamic model, and a policy network. The perception module generates dynamic scene decoupling data based on the input multimodal sensor data sequence. The dynamic scene decoupling data includes road structure, current state of traffic participants, and environmental conditions. The dynamic model generates future trajectories of traffic participants based on their current states and initial control commands. The policy network generates safety control commands based on road structure, future trajectories of traffic participants, and environmental conditions. The phased optimization module 130 is used to freeze the policy network, pre-train the differentiable world model simulation system using the first training data; unfreeze the policy network, use the second training data, and adopt a course learning strategy to perform end-to-end joint optimization of the differentiable world model simulation system; and use the third training data to perform security reinforcement training on the differentiable world model simulation system.

[0102] In some embodiments, the perception module includes a neural field construction module and a dynamic scene decoupler; the neural field construction module is used to construct a spatiotemporally aligned multimodal neural field based on the input multimodal sensor data sequence; the dynamic scene decoupler is used to extract equivariant features from the multimodal neural field using a three-dimensional Euclidean group equivariant convolutional neural network, and generate structured dynamic scene decoupling data based on the extracted equivariant features.

[0103] In some embodiments, the policy network includes a high-level decision-maker and a low-level controller; the high-level decision-maker is used to predict the intentions of traffic participants based on their future trajectories to obtain their intentions; the low-level controller is used to perform differential flat spatial projection based on the traffic participants' intentions to obtain the vehicle's initial trajectory parameters, apply physical and rule constraints to the initial trajectory parameters based on preset dynamic constraint information and road structure to obtain the original control command, and perform risk field modulation on the original control command based on the environmental conditions, risk field gradient, and weights of the traffic participants' intentions output by the perception module to obtain the safety control command.

[0104] In some embodiments, the dynamic model is used to construct input data based on the current state of the traffic participant and the initial control command, input the input data into a pre-constructed neural ordinary differential equation to obtain the state derivative, and use a specified solver to integrate the state derivative over a specified time interval to obtain the future trajectory of the traffic participant.

[0105] In some embodiments, during pre-training, the perception loss and dynamics loss are combined to update the parameters of the perception module and the dynamics model, but the parameters of the policy network are not updated; during end-to-end joint optimization, the simulation loss, the real vehicle loss, and the consistency loss are combined to update the parameters of the perception module, the dynamics model, and the policy network; during safety enhancement training, the collision loss and the comfort loss are combined to update the parameters of the perception module, the dynamics model, and the policy network.

[0106] In some embodiments, the perception loss is calculated based on the intersection-union ratio between the predicted semantic segmentation result and the ground truth semantic segmentation result, and the Euclidean distance between the predicted traffic participant position and the ground truth traffic participant position; the dynamics loss is calculated based on the mean square error between the predicted state change rate and the ground truth state change rate; the simulation loss is calculated based on the Euclidean distance between the predicted trajectory and the ground truth trajectory in the simulation environment, and the Euclidean distance between the predicted control command and the ground truth control command; the real vehicle loss is calculated based on the real vehicle intervention frequency and the vehicle longitudinal acceleration change rate; the consistency loss is calculated based on the maximum mean difference between the simulation feature distribution and the real vehicle feature distribution; the collision loss is calculated based on the minimum distance between the vehicle and the nearest obstacle; and the comfort loss is calculated based on the vehicle longitudinal acceleration change rate and the steering wheel angular velocity.

[0107] In some embodiments, during the backpropagation process of end-to-end joint optimization, the vehicle Jacobian matrix is ​​calculated on the neural ordinary differential equation of the dynamic model by automatic differentiation, an attention diagonal matrix is ​​constructed based on the interaction attention weights of the vehicle and other traffic participants, a target gradient is calculated based on the vehicle Jacobian matrix and the attention diagonal matrix, and gradient information for updating the parameters of the perception module, the dynamic model and the policy network is generated based on the target gradient.

[0108] Specific limitations regarding the autonomous driving training device based on a differentiable world model simulation system can be found in the above-mentioned limitations on the autonomous driving training method based on a differentiable world model simulation system, and will not be repeated here. Each module in the aforementioned autonomous driving training device based on a differentiable world model simulation system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0109] This application also provides a computer device. In some embodiments, the computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can implement the steps of the autonomous driving training method based on the differentiable world model simulation system provided in any of the above embodiments.

[0110] In some embodiments, the internal structure diagram of a computer device may be as follows: Figure 8 As shown, the computer device includes a processor, memory, and network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores training data, dynamic constraint information, and other data; the specific data stored can also be found in the limitations defined in the above method embodiments. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an autonomous driving training method based on a differentiable world model simulation system.

[0111] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0112] This application also provides a computer-readable storage medium, in some embodiments of which a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps of the autonomous driving training method based on the differentiable world model simulation system provided in any of the above embodiments.

[0113] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0114] Those skilled in the art will understand that implementing all or part of the processes in the above method embodiments can be accomplished by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink, DRAM (SLDRAM), memory bus, direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0115] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0116] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An automatic driving training method based on a differentiable world model simulation system, characterized in that, The method comprises: constructing training data based on pre-acquired multi-modal sensor data; the training data comprises first training data, second training data and third training data; constructing a differentiable world model simulation system; the differentiable world model simulation system comprises a differentiable perception module, a dynamics model and a policy network; the perception module is used to generate dynamic scene decoupling data according to an input multi-modal sensor data sequence; the dynamic scene decoupling data comprises road structure, current state of a traffic participant and environmental conditions; the dynamics model is used to generate a future trajectory of the traffic participant according to the current state of the traffic participant and initial control instructions; the policy network is used to generate safe control instructions according to the road structure, the future trajectory of the traffic participant and the environmental conditions; freezing the policy network, pre-training the differentiable world model simulation system using the first training data; unfreezing the policy network, performing end-to-end joint optimization of the differentiable world model simulation system using the second training data and adopting a curriculum learning strategy; performing safe reinforcement training of the differentiable world model simulation system using the third training data.

2. The method of claim 1, wherein, The perception module comprises a neural field construction module and a dynamic scene decoupler; the neural field construction module is used to construct a spatio-temporally aligned multi-modal neural field according to an input multi-modal sensor data sequence; the dynamic scene decoupler is used to extract equivariant features from the multi-modal neural field using a three-dimensional Euclidean group equivariant convolutional neural network, and generate structured dynamic scene decoupling data according to the extracted equivariant features.

3. The method of claim 1, wherein, The policy network comprises a high-level decision maker and a low-level controller; the high-level decision maker is used to perform intention prediction according to the future trajectory of the traffic participant, to obtain an intention of the traffic participant; The low-level controller is used to perform differential flatness space projection according to the intention of the traffic participant, to obtain preliminary trajectory parameters of the ego vehicle, to impose physical and rule constraints on the preliminary trajectory parameters according to preset dynamics constraint information and the road structure, to obtain original control instructions, and to perform risk field modulation on the original control instructions according to weights of the environmental conditions, risk field gradients and the intention of the traffic participant output by the perception module, to obtain safe control instructions.

4. The method of claim 1, wherein, The dynamics model is used to construct input data according to the current state of the traffic participant and initial control instructions, to input the input data into a pre-constructed neural ordinary differential equation, to obtain a state derivative, to use a specified solver to integrate the state derivative within a specified time interval, and to obtain a future trajectory of the traffic participant.

5. The method of claim 1, wherein, When the pre-training is performed, the parameters of the perception module and the dynamics model are updated by combining a perception loss and a dynamics loss, and the parameters of the policy network are not updated; when the end-to-end joint optimization is performed, the parameters of the perception module, the dynamics model and the policy network are updated by combining a simulation loss, a real vehicle loss and a consistency loss; when the safe reinforcement training is performed, the parameters of the perception module, the dynamics model and the policy network are updated by combining a collision loss and a comfort loss.

6. The method of claim 5, wherein, The perception loss is calculated according to the intersection over union between the predicted semantic segmentation result and the ground truth semantic segmentation result and the Euclidean distance between the predicted traffic participant position and the ground truth traffic participant position; the dynamics loss is calculated according to the mean square error between the predicted state change rate and the ground truth state change rate; the simulation loss is calculated according to the Euclidean distance between the predicted trajectory and the ground truth trajectory in the simulation environment and the Euclidean distance between the predicted control instruction and the ground truth control instruction; the real vehicle loss is calculated according to the real vehicle intervention frequency and the vehicle longitudinal acceleration change rate; the consistency loss is calculated according to the maximum mean difference between the simulation feature distribution and the real vehicle feature distribution; the collision loss is calculated according to the minimum distance value between the ego vehicle and the nearest obstacle; and the comfort loss is calculated according to the vehicle longitudinal acceleration change rate and the steering wheel angular velocity.

7. The method of claim 5, wherein, In the back propagation process of the end-to-end joint optimization, the ego vehicle Jacobian matrix is calculated on the neural differential equation of the dynamics model through automatic differentiation, the attention diagonal matrix is constructed according to the interaction attention weight of the ego vehicle and other traffic participants, the target gradient is calculated according to the ego vehicle Jacobian matrix and the attention diagonal matrix, and the gradient information for updating the parameters of the perception module, the dynamics model and the policy network is generated respectively according to the target gradient.

8. An automatic driving training system based on a differentiable world model simulation system, characterized by, The system comprises: a training data construction module configured to construct training data based on pre-acquired multi-modal sensor data; the training data comprises first training data, second training data and third training data; a world model simulation system construction module configured to construct a differentiable world model simulation system; the differentiable world model simulation system comprises a differentiable perception module, a dynamics model and a policy network; the perception module is configured to generate dynamic scene decoupling data according to an input multi-modal sensor data sequence; the dynamic scene decoupling data comprises road structure, traffic participant current state and environmental condition; the dynamics model is configured to generate traffic participant future trajectory according to the traffic participant current state and initial control instruction; and the policy network is configured to generate safe control instruction according to the road structure, the traffic participant future trajectory and the environmental condition; a staged optimization module configured to freeze the policy network, pre-train the differentiable world model simulation system using the first training data, unfreeze the policy network, perform end-to-end joint optimization of the differentiable world model simulation system using the second training data and adopting a curriculum learning strategy, and perform safe reinforcement training of the differentiable world model simulation system using the third training data.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 7.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 7.

Images