Driving decision model optimization method and electronic device

Abstract

The application discloses a driving decision model optimization method and electronic equipment, and relates to the technical field of automatic driving. The first driving path point sequence of the future moment of the first driving perception data is generated by denoising the first driving perception data and the first noisy data according to the trained diffusion model, so as to generate a rich and diverse driving trajectory by using the powerful complex multi-modal distribution modeling capability of the diffusion model. The evaluator training sample is constructed by generating a plurality of second driving path point sequences of the future moment according to the second driving perception data by using the first diffusion model. The evaluator model is trained to select the optimal driving path point sequence from the plurality of second driving path point sequences. The driving decision model is constructed by using the trained diffusion model and the evaluator model. The decision-making ability of the driving decision model to generate the candidate driving trajectory and select the optimal driving trajectory is strengthened by the joint training of the evaluator model and the diffusion model, and the safety of the automatic driving is improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a driving decision model optimization method and electronic device. Background Technology

[0002] End-to-end driving solutions have become an important tool in autonomous driving in recent years due to their ability to learn driving decisions directly from sensor data. However, driving behavior is inherently highly multimodal, and solutions relying on behavior cloning (BC) struggle to effectively capture and reproduce the complex multimodal data distribution exhibited by human drivers. This results in poor generalization ability of driving trajectories generated by driving decision models in real-world scenarios outside the training data distribution, making them unsuitable for application in practical autonomous driving environments.

[0003] Improving the generalization level of driving trajectory generation by driving decision-making models is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] This invention provides a driving decision model optimization method and electronic device to at least solve the problem of poor generalization level of driving trajectory generation by driving decision models in related technologies.

[0005] This invention provides a method for optimizing a driving decision-making model, comprising:

[0006] Obtain driving data samples;

[0007] Using the driving data samples, a diffusion model is trained to perform denoising processing on the input first driving perception data and first noisy data to generate a first driving path point sequence of the first driving perception data at future times, thus obtaining the first diffusion model;

[0008] The first diffusion model is used to generate multiple sequences of second driving path points for future moments based on the second driving perception data. Evaluator training samples are constructed based on the second driving perception data and the corresponding sequences of second driving path points.

[0009] Using the evaluator training samples, the evaluator model is trained to select the optimal driving path point sequence from multiple second driving path point sequences to obtain the first evaluator model.

[0010] A driving decision model is constructed using the first diffusion model and the first evaluator model.

[0011] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described driving decision model optimization methods.

[0012] This invention utilizes a diffusion model to generate a sequence of first driving path points for future moments of the first driving perception data by denoising the input first driving perception data and first noisy data. This leverages the powerful complex multimodal distribution modeling capabilities of the diffusion model to generate diverse driving trajectories, improving the generalization ability of generated trajectories compared to traditional driving policy networks. Furthermore, it saves computational overhead compared to traditional methods that use diffusion models to predict future scenarios. The first diffusion model generates multiple sequences of second driving path points for future moments of the second driving perception data. Evaluator training samples are constructed based on the second driving perception data and the corresponding second driving path point sequences. Using these training samples, an evaluator model is trained to select the optimal driving path point sequence from multiple second driving path point sequences, resulting in a first evaluator model. A driving decision model is constructed using the trained diffusion model and the first evaluator model. Through joint training of the evaluator model and the diffusion model, the driving decision model's ability to generate candidate driving trajectories and select the optimal driving trajectory is enhanced, improving the safety and reliability of autonomous driving decisions and enhancing the autonomous driving performance of vehicles in complex traffic environments. Attached Figure Description

[0013] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart of a driving decision model optimization method provided in an embodiment of the present invention;

[0015] Figure 2 This is a schematic diagram of the structure of a driving decision model optimization system provided in an embodiment of the present invention;

[0016] Figure 3 A flowchart illustrating the offline training process of a diffusion model provided in an embodiment of the present invention;

[0017] Figure 4 This is a flowchart illustrating the training process of an evaluator model provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0019] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0020] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] The embodiments of the present invention provide a driving decision model optimization method. The method is described in detail below in conjunction with the execution flow of the driving decision model optimization method.

[0022] Figure 1 A flowchart of a driving decision model optimization method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a driving decision model optimization system provided in an embodiment of the present invention.

[0023] like Figure 1 As shown, the driving decision model optimization method provided in this embodiment of the invention may include: S101: obtaining driving data samples.

[0024] S102: Using driving data samples, a diffusion model is trained. The input first driving perception data and first noisy data are denoised to generate a sequence of first driving path points for future moments of the first driving perception data, thus obtaining the first diffusion model.

[0025] S103: Using the first diffusion model, generate multiple second driving path point sequences for future moments of the second driving perception data based on the second driving perception data, and construct evaluator training samples based on the second driving perception data and the corresponding second driving path point sequences.

[0026] S104: Using the evaluator training samples, train the evaluator model to select the optimal driving path point sequence from multiple second driving path point sequences to obtain the first evaluator model.

[0027] S105: Construct a driving decision model using the first diffusion model and the first evaluator model.

[0028] It should be noted that, in this embodiment of the invention, the path point sequence refers to a sequence of path points at multiple consecutive time points calculated by the model, which can also be called a driving trajectory. Path points can be represented by coordinates, so the path point sequence can be represented as a coordinate time series. Alternatively, the path point sequence can also be represented in the form of a graph.

[0029] like Figure 2 As shown, the model training step in the driving decision model optimization method provided in this embodiment of the invention can be applied to a model training device. The model training device can use one or more artificial intelligence servers, which can include basic settings such as artificial intelligence processors, storage, input / output devices, communication buses, and communication interfaces, and is composed of software environments and application software such as operating systems, databases, and middleware.

[0030] For S101, driving data samples are acquired, which can be driving video samples obtained based on real driving scenarios. Specifically, the vehicle is used as the main intelligent agent, and the vehicle is controlled by a human driver. During this process, driving data is collected using various onboard sensor modules, and data preprocessing, such as timestamp alignment, is performed on the onboard edge server. Considering the limited computing and storage resources of the onboard edge server, the data is only temporarily cached on the onboard edge server. After preprocessing, it is synchronously uploaded to the model training device to build a dataset for training the world model. Onboard sensor modules can include, but are not limited to, radar, cameras, onboard control units, inertial measurement units, and navigation systems. Radar and cameras are used to collect environmental perception data, vehicle control signals are obtained from the onboard control unit, and inertial measurement units and navigation systems are used to collect vehicle motion states. The environmental perception data, vehicle motion status data, and other information are input into the data preprocessing module of the onboard edge server. This module timestamps the multimodal data and detects structured traffic condition-related data (such as targets and lane lines). Multi-view images, road topology images, traffic target data, and vehicle control information are then sent to the model training device and stored in its storage module. The model training device receives manual annotations of the driving dataset or uses an automatic annotation module to add scene description information, thus constructing the driving dataset. Driving data samples can then originate from this driving dataset.

[0031] In some optional embodiments of the present invention, the driving data sample may include driving perception data and the corresponding sample path point sequence at each time.

[0032] In some optional embodiments of the present invention, the driving data sample may include the following information: ① Multi-view images: Multi-view image information acquired in real time by multiple vehicle-mounted cameras during vehicle driving, and the acquired images may be in Red, Green, Blue, RGB format. For example, if 6 vehicle-mounted cameras are used, they can correspond to 6 views of the vehicle: front (F), front left (FL), front right (FR), rear (B), rear left (BL), and rear right (BR). Each image is in [format missing]. The image is stored in the form of a graph, where H and W represent the height and width of the image, respectively. In addition, corresponding viewpoint and time step markers are included along with the stored image.

[0033] ② Road topology images: Road topology information, including lane boundaries, lane dividers, and pedestrian crossings, is extracted from multi-view images through manual annotation or related methods. Each image is... The images are stored in a format that is aligned with time step markers and multi-view images.

[0034] ③ Traffic target data: Road target and category information, including pedestrians, vehicles, bicycles, animals, etc., are extracted from multi-view images through manual annotation or related methods. Each target is labeled with a 3D bounding box and projected onto the image plane. Stored in the form of, where For each target category, each image is aligned with the multi-view images according to the time step marker.

[0035] ④ Vehicle control information: The vehicle control actions at each time step are recorded by the onboard controller, including two control quantities: steering angle and speed. The images are stored in a format that is aligned with the multi-view images according to time step markers.

[0036] ⑤ Scene description information: Add scene description information, including weather, time, etc., to the multi-view images through manual annotation or related methods. Store the information in text form and align it with the multi-view images according to the time step mark.

[0037] During the data acquisition phase, image data and other data can be acquired and processed at a frequency of 12Hz to obtain an image dataset. Further, multiple image data samples are aggregated into video data within a certain period of time according to a set number of frames, thereby dividing the image dataset into several video segment samples to obtain a driving video dataset. Each driving video sample contains a series of time-step aligned multi-view images, road topology images, traffic target data, vehicle control data, and scene description text data.

[0038] For S102, such as Figure 2As shown, the diffusion model before offline optimization can be denoted as the initial diffusion model, and the diffusion model after offline optimization using driving data samples can be denoted as the first diffusion model.

[0039] In some optional embodiments of the present invention, step S102, using driving data samples, trains a diffusion model to perform denoising processing on the input first driving perception data and first noisy data to generate a first driving path point sequence for future times of the first driving perception data, thereby obtaining a first diffusion model. This may include: acquiring the first driving perception data from the driving data samples; using the diffusion model to perform denoising processing on the first driving perception data and random noise to obtain a first driving path point sequence for future times of the first driving perception data; calculating a first model learning loss value for the diffusion model based on the first driving path point sequence and the first sample path point sequence in the driving data samples at the corresponding times; updating the model parameters of the diffusion model using the first model learning loss value until the first iteration termination condition is met, thereby obtaining the first diffusion model.

[0040] However, reconstructing the sequence of driving path points for future moments from random noise using a diffusion model requires a large number of denoising iterations. In addition, the dynamic randomness of real-world open environments leads to insufficient real-time performance in generating diverse driving trajectories.

[0041] To address this, this invention provides a truncated diffusion strategy that combines the prior distribution of multimodal key trajectory points with the truncated diffusion process, allowing the model to directly denoise from the key trajectory point distribution to the multimodal driving trajectory distribution. This requires only a few denoising iterations to complete high-quality trajectory inference, avoiding lengthy iterations starting from pure random noise.

[0042] In some optional embodiments of the present invention, S102, using driving data samples, training a diffusion model to denoise the input first driving perception data and first noisy data to generate a first driving path point sequence for future moments of the first driving perception data, thereby obtaining a first diffusion model, may also include: extracting a first sample path point sequence corresponding to the future moments of the first driving perception data from the driving data samples; adding noise to the first sample path point sequence to obtain first noisy data; inputting the first driving perception data and the corresponding first noisy data into the diffusion model and outputting a first driving path point sequence; calculating a first model learning loss value of the diffusion model based on the first driving path point sequence and the corresponding first sample path point sequence; updating the model parameters of the diffusion model using the first model learning loss value until the first iteration termination condition is reached, thereby obtaining the first diffusion model.

[0043] Specifically, during training, a small amount of noise is added to the sequence of key path points to form a more physically plausible initial distribution. Compared to traditional diffusion models that learn driving behavior directly from random noise, this approach significantly reduces the need for denoising steps, allowing the model to be optimized from a trajectory distribution that is closer to real driving behavior.

[0044] For S103, such as Figure 2 As shown, the evaluator model is trained using the first diffusion model optimized offline.

[0045] In this embodiment of the invention, the evaluator model is used to evaluate the quality of the driving path point sequences predicted by the diffusion model. In S103, the first diffusion model generates multiple second driving path point sequences for future moments based on the second driving perception data. Evaluator training samples are constructed based on the second driving perception data and the corresponding second driving path point sequences. In S104, the evaluator model is trained using the evaluator training samples to select the optimal driving path point sequence from the multiple second driving path point sequences, thus obtaining the first evaluator model and optimizing the evaluation function of the evaluator model.

[0046] In this embodiment of the invention, the evaluator model may include a perception data prediction module and a reward module. The perception data prediction module is used to generate driving perception data prediction results at corresponding times based on the second driving path point sequence output by the first diffusion model. The reward module is used to generate reward values ​​based on the driving perception data prediction results to select the optimal driving path point sequence. The perception data prediction module may be constructed using one or more generative models such as Long Short-Term Memory (LSTM) networks, Gated Recurrent Units (GRUs) based on temporal prediction, or Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), etc., or it may be constructed using a world model.

[0047] In the iterative training described in the various embodiments of the present invention, the iteration termination condition can be reaching a preset number of iterations or satisfying a preset convergence condition (loss value is less than a preset loss).

[0048] For S105, a driving decision model is constructed using the first diffusion model and the first evaluator model. The first diffusion model can be directly used as a generator to predict and generate multiple driving path point sequences for future moments, and the first evaluator model can be used as an evaluator to select the optimal driving path point sequence from multiple driving path point sequences.

[0049] In other optional embodiments of the present invention, such as Figure 2 As shown, the first diffusion model, or the first diffusion model and the first evaluator model, can also be put into a simulation environment for online training, thereby further improving the generalization ability of the diffusion model (and the evaluator model).

[0050] The driving decision model obtained by the driving decision model optimization method provided in this embodiment of the invention loads the network parameters of the diffusion strategy to the vehicle edge server through Ethernet communication technology.

[0051] During the actual operation of the autonomous driving system, the vehicle interacts with the real environment in real time. It collects environmental perception data from the real environment through the on-board sensor module and transmits the perception data to the on-board edge server for processing through the data bus. At the same time, the vehicle receives action commands from the on-board edge server through the Controller Area Network (CAN) bus and hands them over to the vehicle control drive module to control the vehicle to perform the corresponding driving actions.

[0052] After receiving multi-view images, radar point clouds, and other perception data sent by the vehicle, the vehicle edge server first performs preprocessing such as format conversion on the raw sensor data through the data preprocessing module. The resulting multimodal data is then sent to the feature encoding module to be processed into low-dimensional latent state features. These features are then input into the diffusion model in the driving decision model, which outputs multiple driving path point sequences for future time moments. The evaluator model in the driving decision model selects the optimal driving path point sequence from these sequences, and the driving action is output based on the optimal driving path point sequence.

[0053] The driving decision model optimization method provided in this invention trains a diffusion model to denoise the input first driving perception data and first noisy data to generate a sequence of first driving path points for future moments of the first driving perception data. This leverages the powerful complex multimodal distribution modeling capability of the diffusion model to generate diverse driving trajectories, improving the generalization ability of generated trajectories compared to traditional driving policy networks, and saving computational overhead compared to traditional methods using diffusion models to predict future scenarios. The method then uses the first diffusion model to generate multiple sequences of second driving path points for future moments of the second driving perception data. Based on the second driving perception data and the corresponding second driving path point sequences, an evaluator training sample is constructed. Using the evaluator training sample, an evaluator model is trained to select the optimal driving path point sequence from the multiple second driving path point sequences, resulting in a first evaluator model. The trained diffusion model and the first evaluator model are then used to construct a driving decision model. Through joint training of the evaluator model and the diffusion model, the driving decision model's ability to generate candidate driving trajectories and select the optimal driving trajectory is enhanced, improving the safety and reliability of autonomous driving decisions and enhancing the autonomous driving performance of vehicles in complex traffic environments.

[0054] Figure 3 This is a flowchart illustrating the offline training process of a diffusion model provided in an embodiment of the present invention.

[0055] The above embodiments introduce a truncation diffusion strategy to improve the inference efficiency of diffusion models. Based on this, the embodiments of the present invention further provide specific implementation methods.

[0056] In this embodiment of the invention, inputting the first driving perception data and the corresponding first noisy data into the diffusion model and outputting the first driving path point sequence may include: inputting the first driving perception data and the corresponding first noisy data into the diffusion model and outputting multiple first driving path point sequences and their corresponding predicted classification scores.

[0057] The calculation of the first model learning loss value of the diffusion model based on the first driving path point sequence and the corresponding first sample path point sequence may include: matching multiple first driving path point sequences with the first sample path point sequence respectively, determining the first driving path point sequence that is closest to the first sample path point sequence as a positive sample, and taking the other first driving path point sequences as negative samples, using the positive and negative samples as the sample labels of the first driving path point sequences; determining the first distance loss between the first driving path point sequence and the corresponding first sample path point sequence, and calculating the trajectory recovery loss value based on the sample labels of the first driving path point sequence and the first distance loss; calculating the classification confidence based on the corresponding predicted classification score of the first driving path point sequence and the sample labels of the first driving path point sequence; and calculating the first model learning loss value based on the trajectory recovery loss value and the classification confidence.

[0058] In this embodiment of the invention, when training the diffusion model, a small amount of noise is added to the sequence of sample path points provided by the driving data samples to form a more physically plausible initial distribution. Compared with the traditional diffusion model, which learns driving behavior directly from random noise, this method significantly reduces the need for denoising steps, allowing the model to be optimized from a trajectory distribution that more closely resembles real driving behavior. The training process of the diffusion strategy can be divided into the following steps:

[0059] ① Extract sample path point sequences: Cluster the ground truth values ​​of the vehicle's future driving trajectory on the driving data samples to obtain a set of representative key path point sequences. ,in, For the first A sequence of key path points, which can cover typical trajectory patterns in different driving scenarios, is denoted as the sample path point sequence.

[0060] ② Constructing a noisy distribution of the sample path point sequence: Gaussian noise is added to the sample path point sequence to obtain a noisy critical path point distribution:

[0061] (1)

[0062] in, Indicates the number of iterations with added noise;

[0063] ③ Iteratively denoise the first noisy data: [This refers to the process of denoising the data after adding noise.] A noisy trajectory Denoising input to the diffusion model Among them, the predicted classification score and denoised driving path point sequences :

[0064] (2)

[0065] in, To provide control condition information for the denoiser, this embodiment of the invention inputs driving perception features (derived from first driving perception data) to perform denoising control;

[0066] ④ Using truth-value-supervised denoising results: for the denoised driving path point sequence Compare it with the actual trajectory Match the first sample path point sequence to obtain the closest trajectory. Then the trajectory Surrounding noise trajectories as positive samples The others are used as negative samples. ;

[0067] ⑤ For the denoised first driving path point sequence, a loss function is designed from two aspects: trajectory recovery and classification confidence. The trajectory recovery loss value is obtained by analyzing the first driving path point sequence. With the first sample path point sequence The classification confidence score is measured by the distance between the predicted classification scores and the actual positive and negative sample labels.

[0068] (3)

[0069] ⑥ Update denoiser parameters: Based on the loss value calculated in step ⑤, update the denoiser using a gradient optimization method. Network parameters Repeat steps ③ to ⑥ until the first iteration termination condition is met (such as reaching the maximum number of training iterations).

[0070] This invention also provides a network structure for a diffusion model, such as... Figure 3 As shown, the diffusion model provided in this embodiment of the invention may include N cascaded denoising networks, each of which may include a first cross-attention module, a second cross-attention module, a feedforward network, a time-step modulation layer, and a multilayer perceptron.

[0071] like Figure 3 As shown, the input is first a noisy trajectory sampled from a noisy Gaussian distribution. Then, a deformable spatial cross-attention mechanism is used to interact with bird's-eye view features or perspective features based on trajectory coordinates. Subsequently, cross-attention calculation is performed between trajectory features and vehicle or map query information, and then a feedforward network is used for feature fusion.

[0072] For the diffusion time step information, a time step modulation layer is used for encoding, followed by a multilayer perceptron to predict the confidence level and the offset relative to the initial noise trajectory coordinates.

[0073] The aforementioned network layers are constructed as a denoising module. The output of the current denoising module is used as the input of subsequent denoising modules with the same network structure to build a cascaded diffusion decoder. During inference, the trajectory is iteratively denoised, and the trajectory quality is optimized in a progressive manner. During this process, the parameters of the denoising module are shared across time steps.

[0074] Applying the network structure of the diffusion model provided in this embodiment of the invention, inputting the first driving perception data and the corresponding first noisy data into the diffusion model, and outputting the first driving path point sequence, may include: inputting the first driving perception data and the corresponding first noisy data into the diffusion model to perform multi-level denoising calculation, and using the denoised driving path point sequence obtained from the last level of denoising calculation as the first driving path point sequence.

[0075] The first-level denoising calculation may include: performing cross-attention calculation on the input noisy data and the perception map features in the first driving perception data to obtain the first attention calculation result; wherein, corresponding to the first-level denoising calculation, the noisy data is the first noisy data; corresponding to the second-level and subsequent denoising calculations, the noisy data is the denoised driving path point sequence output by the previous-level denoising network; the perception map features include at least one of bird's-eye view features and perspective view features; performing cross-attention calculation on the first attention calculation result and the driving data query information in the first driving perception data to output the second attention calculation result; the driving data query information includes at least one of vehicle query information and map query information; performing encoding calculation based on the second attention calculation result to obtain the first feature code; and predicting and generating the denoised driving path point sequence based on the first feature code.

[0076] In related technologies, driving world models use diffusion models to predict future scene images, a time-consuming process unsuitable for real-time driving. While some studies render free-viewpoint future videos in real time, this typically requires external reconstruction, making it impractical for in-vehicle driving environments. Furthermore, there is a lack of supervision over future states, as world models need to imagine multiple future states based on multiple trajectory candidates, while real-world datasets usually only have one available future state.

[0077] In this regard, the embodiment of the present invention takes the construction of the world model as an example of the perception data generation module of the evaluator model.

[0078] In this embodiment of the invention, step S104, which uses evaluator training samples to train an evaluator model to select the optimal driving path point sequence from multiple second driving path point sequences to obtain a first evaluator model, may include: inputting second driving perception data and corresponding multiple second driving path point sequences into a world model to obtain driving perception data prediction results corresponding to each of the multiple second driving path point sequences; calling the reward module to calculate the predicted reward value of the corresponding second driving path point sequence based on the driving perception data prediction results; selecting the second driving path point sequence with the highest predicted reward value among the multiple second driving path point sequences as the optimal driving path point sequence; calculating the second model learning loss value of the world model based on the optimal driving path point sequence and the second sample path point sequence at the future time corresponding to the second driving perception data; updating the model parameters of the world model using the second model learning loss value until the second iteration termination condition is reached to obtain a first world model; and constructing a first evaluator model using the first world model and the reward module.

[0079] This invention uses a world model to predict future driving perception data, specifically to predict future bird's-eye view features, thereby achieving efficient and real-time trajectory evaluation. Specifically, the second driving perception data corresponding to the current moment and the second driving path point sequence output by the first diffusion model (as a candidate driving path point sequence) are combined to form the input of the world model. Then, the world model repeatedly predicts the future states corresponding to different candidate trajectories at multiple time steps.

[0080] In this embodiment of the invention, the generation of multiple second driving path point sequences at future moments of the second driving perception data using a first diffusion model, and the construction of evaluator training samples based on the second driving perception data and the corresponding second driving path point sequences, may include: determining second driving state features from the second driving perception data; generating multiple second driving path point sequences using the first diffusion model based on the second driving state features; encoding the second driving path point sequences into action embedding features using a trajectory encoder; and constructing multiple state-action pairs from the second driving state features and the corresponding multiple action embedding features to obtain evaluator training samples.

[0081] The second driving perception data and the corresponding multiple second driving path point sequences are input into the world model to obtain the driving perception data prediction results corresponding to each of the multiple second driving path point sequences. This can include: obtaining a state-action pair, flattening the second driving state features and connecting them with the action embedding features to obtain a first feature vector; inputting the first feature vector into the world model so that the world model recursively outputs the driving perception data prediction results and the action prediction values ​​for the next time step by step until the driving perception data prediction results corresponding to the second driving path point sequences are obtained.

[0082] Specifically, taking the type of second driving perception data as a bird's-eye view as an example. Based on the current bird's-eye view state... Generate using the first diffusion model Candidate trajectories Then a trajectory encoder is used to convert the candidate trajectories. Encoding as an action embedding , build State-action pairs As input to the world model;

[0083] Then, future predictions are made based on candidate trajectories. Given a state-action pair, a world model is used to predict the corresponding future state. Given a state-action pair... ,Will Flattened and embedded with motion Connect them to obtain the feature vector. Then The input is fed into the world model. In this embodiment of the invention, the world model can be constructed using a Transformer encoder. The output of the world model corresponds to the predicted future state. and future actions :

[0084] (4)

[0085] Furthermore, the world model will predict the future recursively. Step state Since the size of the bird's-eye view is relatively small, it does not introduce a large amount of computational overhead.

[0086] (5)

[0087] The reward value for candidate trajectories is calculated based on future states. Using a world model, the reward module considers the current bird's-eye view state and future... Bird's-eye view status prediction step Rewards for each candidate trajectory :

[0088] (6)

[0089] Based on the above steps, the reward module outputs reward values ​​for multiple second driving path point sequences.

[0090] In this embodiment of the invention, calling the reward module to calculate the predicted reward value of the corresponding second driving path point sequence based on the prediction result of driving perception data may include: encoding the prediction result of driving perception data to obtain global features of driving perception data; concatenating the action prediction sequence of the action prediction value at the corresponding time of the second driving path point sequence with the global features of driving perception data to obtain a first concatenated feature; and predicting the predicted reward value based on the first concatenated feature.

[0091] In this embodiment of the invention, the reward module can consist of three parts. First, a two-dimensional convolutional layer is used to process the feature map of the bird's-eye view. Encoding is used to aggregate information from different regions and time steps of the bird's-eye view state, followed by global average pooling to form the bird's-eye view pooled features. .

[0092] Then, pool the features of the bird's-eye view. With motion prediction spliced ​​together The concatenated features are then input into the encoding layer of a multilayer perceptron to predict the reward. The candidate trajectory with the highest reward is selected as the final trajectory.

[0093] In this embodiment of the invention, calling the reward module to calculate the predicted reward value of the corresponding second driving path point sequence based on the prediction result of driving perception data may include: calling the reward module to calculate the imitation reward value and the simulation reward value of the corresponding second driving path point sequence based on the prediction result of driving perception data; determining the predicted reward value based on the imitation reward value and the simulation reward value; wherein, the imitation reward value is the reward value of the degree to which the second driving path point sequence imitates the second sample path point sequence; and the simulation reward value is the reward value determined based on the driving safety conditions provided by the autonomous driving simulator.

[0094] To achieve efficient training of the joint reward module of the world model, the reward signal in this embodiment of the invention may include two parts: imitation reward and simulated reward.

[0095] The process of calculating the imitation reward value of the corresponding second driving path point sequence based on the prediction results of driving perception data may include: calculating the second path distance between the second driving path point sequence and the corresponding second sample path point sequence; and calculating the imitation reward value based on the second path distance.

[0096] That is to say, imitation reward The L2 distance between the trajectory anchor point and the expert trajectory can be used to measure the degree to which the predicted trajectory imitates the expert trajectory.

[0097] The simulated reward value for the corresponding second driving path point sequence is calculated based on the prediction results of driving perception data. This may include: calculating the no-fault collision reward value, drivable area compliance reward value, collision time reward value, vehicle progress reward value, and comfort reward value corresponding to the second driving path point sequence based on the prediction results of driving perception data; and weighting the no-fault collision reward value, drivable area compliance reward value, collision time reward value, vehicle progress reward value, and comfort reward value to obtain the simulated reward value.

[0098] Simulated rewards Trajectory quality can be evaluated based on standards defined by the simulator, including no-fault collision bonus values. Compliance bonus value for drivable areas Collision time bonus value Self-driving vehicle progress reward value Comfort bonus value These five dimensions employ a weighted strategy to achieve a dynamic balance between security and efficiency.

[0099] The "no-fault collision" rating is a binary indicator used to determine collision liability. It assesses whether the vehicle, despite adhering to all traffic rules and exercising reasonable avoidance, was still unable to avoid a collision. If so, the collision is marked as "no fault." The calculation steps for the no-fault collision reward value may include: Collision event detection: The simulation engine detects physical contact between the vehicle and any obstacle; Fault analysis: The system analyzes the state instantaneously before the collision, checking for any violations or inappropriate behavior by the vehicle, such as: Traffic rule violations: Whether the vehicle ran a red light, drove against traffic, or crossed lane lines; Right-of-way: Whether the vehicle's right-of-way was violated by other vehicles when it had it (e.g., a vehicle crossing a stop sign from the side); Reasonable avoidance: Even if the vehicle had the right-of-way, whether it took reasonable avoidance measures (e.g., slowing down in advance, honking the horn); If the vehicle had sufficient time and space to react but remained inactive, it may still be considered at fault; Dangerous behavior: Whether the vehicle engaged in sudden, unpredictable actions such as a cut-in that caused a rear-end collision.

[0100] Simulator judgment results: No_At-Fault_Collision=1 (True) indicates that if the vehicle did not commit any of the above-mentioned fault behaviors, the collision was mainly caused by the other party or other external factors; No_At-Fault_Collision=0 (False) indicates that if the vehicle committed any of the above-mentioned fault behaviors, it caused or contributed to the occurrence of the collision.

[0101] Drivable area compliance is used to measure whether the ego vehicle always travels within the permitted road area. Check whether the bounding rectangle (or multiple boundary points) of the ego vehicle exceeds the "drivable area" defined by the high-precision map (usually the area within the lane lines) at any time. The calculation steps of the drivable area compliance reward value can include: frame-by-frame check: at each time step (Δt) of the simulation: Is_Violation(t) = 1 if (Ego_Bounding_Box(t) ∩ Drivable_Area)!= Ego_Bounding_Box(t); if the intersection of the ego vehicle bounding box and the drivable area is not equal to the ego vehicle bounding box itself, it means that part of the vehicle body exceeds the range, which is recorded as a violation Is_Violation = 0 otherwise Is_Violation = 1.

[0102] Time to collision is a continuous and forward-looking metric for measuring safety. Assuming the ego vehicle and the obstacle maintain their current motion states (speed, acceleration), calculate the time required for them to collide. The calculation steps of the time to collision reward value can include: for each relevant obstacle, calculate its relative speed Δv and relative distance Δs with the ego vehicle: TTC_i(t)=Δs(t) / Δv(t); Time to Collision (TTC) threshold violation: if there exists TTC_i(t)<TTC_threshold, then it is 0, otherwise it is 1; the TTC threshold is the minimum value of TTC for all obstacles in the entire scenario.

[0103] Ego Progress is used to measure the efficiency of the ego vehicle moving forward along the planned path towards the target point, and calculate the distance traveled by the ego vehicle along the reference path within a specific time period. The calculation steps of the ego progress reward value can include: Progress_Ratio(t)=Distance_Traveled_Along_Path(t) / Total_Path_Length; Progress_Score=Progress_Ratio(t_end); that is, the progress ratio at the end of the scenario.

[0104] Comfort is quantified by changes in vehicle acceleration, primarily focusing on jerk. Rapid acceleration, sudden braking, and sharp turns can all lead to excessively high jerk values, causing discomfort. Jerk is the derivative of acceleration (a): Jerk(t) = da(t) / dt. The calculation steps for the comfort reward value can include: In discrete simulation time steps, Jerk(t) ≈ (a(t) - a(t - Δt)) / Δt; Jerk threshold violation: If there exists Jerk(t) > Jerk, then the value is 0; otherwise, it is 1. The Jerk threshold is the maximum value of the vehicle's jerk in the scenario; In summary, the reward value of the reward module can be expressed by the following formula:

[0105] (7)

[0106] in, This represents the reward value from the rewards module. Indicates reward for imitation. This indicates the weight of the imitation reward. This represents the no-fault collision reward value. This indicates the weight of the no-fault collision reward value. This indicates the reward value for compliance with regulations in drivable areas. The weighting of the drivable area compliance reward value. This represents the collision time bonus value. Indicates the comfort bonus value. This represents the vehicle's progress reward value. This represents the combined weight of the collision time bonus, comfort bonus, and vehicle progress bonus.

[0107] In this embodiment of the invention, calculating the second model learning loss value of the world model based on the optimal driving path point sequence and the second sample path point sequence at future times corresponding to the second driving perception data may include: obtaining the second sample driving perception data at future times corresponding to the second driving perception data; calculating the perception state supervision loss value of the second driving perception data compared to the second sample driving perception data; calculating the first imitation reward value of the multiple second driving path point sequences corresponding to the second driving perception data for the second sample path point sequences, wherein the optimal driving path point sequence corresponds to the second imitation reward value; and calculating the cross-entropy loss based on the first imitation reward value and the second imitation reward value to obtain the imitation reward. The process involves: supervising the loss value; simulating the third driving scenario data corresponding to multiple second driving path point sequences based on the second driving perception data using a simulator; calculating the first simulated reward value corresponding to the second driving path point sequence based on the third driving scenario data, where the optimal driving path point sequence corresponds to the second simulated reward value; calculating the simulated reward supervised loss value by performing cross-entropy loss based on the first and second simulated reward values; calculating the trajectory difference loss value between the optimal driving path point sequence and its corresponding second sample path point sequence; and calculating the second model learning loss value based on the perception state supervised loss value, the imitation reward supervised loss value, the simulated reward supervised loss value, and the trajectory difference loss value.

[0108] Based on this, in this embodiment of the invention, the loss function of the world model can be designed by combining the ground truth values ​​of the bird's-eye view features, the ground truth values ​​of the trajectory, and the reward supervision signal, and can be expressed as follows:

[0109] (8)

[0110] in, For the state supervision loss of the bird's-eye view, we first use upsampling and transposed convolutional layers to map the state features of the bird's-eye view into a semantic map of the bird's-eye view, including semantic information such as background, roads, sidewalks, center lines, static objects, vehicles, and pedestrians, to provide a comprehensive representation of the bird's-eye view scene:

[0111] (9)

[0112] Then, the focus loss between the future real bird's-eye view semantic map and the predicted bird's-eye view semantic map is calculated:

[0113] (10)

[0114] To simulate the reward-supervised loss, firstly, given the first... Candidate trajectories The system simulates the future positions of other vehicles and its own vehicle using a simulator, generating a future bird's-eye view semantic graph. A rule-based evaluator is then used to evaluate the future bird's-eye view scenario to generate corresponding simulation rewards. Using binary cross-entropy loss to supervise the predicted simulated reward :

[0115] (11)

[0116] To mimic the reward-monitoring loss, first calculate the... Second driving route point sequence L2 distance between the second sample path point sequence To calculate the target of the imitation reward, and then for the results from The negative distances of the second driving path point sequence are calculated using the softmax function to obtain... The second driving path sequence, which is closer to the second sample path sequence, will receive a higher reward:

[0117] (12)

[0118] By imitating rewards The similarity between each second driving path point sequence and the human trajectory is quantified, and the predicted imitation reward is calculated using cross-entropy loss. Supervision:

[0119] (13)

[0120] For trajectory supervision loss, the second driving path point sequence that is closest to the second sample path point sequence is selected. Corresponding candidate trajectory L1 loss is used to measure the second sample path point sequence. Second driving path point sequence Differences between them:

[0121] (14)

[0122] Based on the loss value calculated by formula (8), the world model network parameters are updated using the gradient optimization method. The above steps are repeated until the second iteration termination condition is met (such as reaching the maximum number of training iterations).

[0123] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0124] Embodiments of the present invention also provide a driving decision model optimization device, comprising: an acquisition module for acquiring driving data samples; a first training module for using the driving data samples to train a diffusion model to perform denoising processing on input first driving perception data and first noisy data to generate a first driving path point sequence for future times of the first driving perception data, thereby obtaining a first diffusion model; a first construction module for using the first diffusion model to generate multiple second driving path point sequences for future times of the second driving perception data based on the second driving perception data, and constructing evaluator training samples based on the second driving perception data and the corresponding second driving path point sequences; a second training module for using the evaluator training samples to train an evaluator model to select the optimal driving path point sequence from the multiple second driving path point sequences, thereby obtaining a first evaluator model; and a second construction module for using the first diffusion model and the first evaluator model to construct a driving decision model.

[0125] In this embodiment of the invention, the first training module uses driving data samples to train a diffusion model. Based on the input first driving perception data and first noisy data, a denoising process is performed to generate a first driving path point sequence for future moments of the first driving perception data, thus obtaining the first diffusion model. This process may include: extracting a first sample path point sequence corresponding to the future moments of the first driving perception data from the driving data samples; adding noise to the first sample path point sequence to obtain first noisy data; inputting the first driving perception data and the corresponding first noisy data into the diffusion model and outputting the first driving path point sequence; calculating a first model learning loss value for the diffusion model based on the first driving path point sequence and the corresponding first sample path point sequence; and updating the model parameters of the diffusion model using the first model learning loss value until the first iteration termination condition is met, thus obtaining the first diffusion model.

[0126] In this embodiment of the invention, the first training module inputs the first driving perception data and the corresponding first noisy data into the diffusion model and outputs the first driving path point sequence. This can include: inputting the first driving perception data and the corresponding first noisy data into the diffusion model and outputting multiple first driving path point sequences and their corresponding predicted classification scores; the first training module calculates the first model learning loss value of the diffusion model based on the first driving path point sequences and the corresponding first sample path point sequences. This can include: matching the multiple first driving path point sequences with the first sample path point sequences respectively, determining a first driving path point sequence that is closest to the first sample path point sequence as a positive sample, and treating the other first driving path point sequences as negative samples, using the positive and negative samples as sample labels for the first driving path point sequences; determining the first distance loss between the first driving path point sequences and the corresponding first sample path point sequences, and calculating the trajectory recovery loss value based on the sample labels of the first driving path point sequences and the first distance loss; calculating the cross-entropy based on the corresponding predicted classification scores of the first driving path point sequences and the sample labels of the first driving path point sequences to obtain the classification confidence; and calculating the first model learning loss value based on the trajectory recovery loss value and the classification confidence.

[0127] In this embodiment of the invention, the first training module inputs the first driving perception data and the corresponding first noisy data into a diffusion model and outputs a first driving path point sequence. This may include: inputting the first driving perception data and the corresponding first noisy data into the diffusion model for multi-level denoising calculation, and using the denoised driving path point sequence obtained from the last level of denoising calculation as the first driving path point sequence; wherein the first level of denoising calculation includes: performing cross-attention calculation on the input noisy data and the perception map features in the first driving perception data to obtain a first attention calculation result; wherein, corresponding to the first level of denoising calculation, the noisy data is the first noisy data; corresponding to the second level and subsequent denoising calculations, the noisy data is the denoised driving path point sequence output by the previous level of denoising network; the perception map features include at least one of bird's-eye view features and perspective view features; performing cross-attention calculation on the first attention calculation result and the driving data query information in the first driving perception data to output a second attention calculation result; the driving data query information includes at least one of vehicle query information and map query information; performing encoding calculation based on the second attention calculation result to obtain a first feature code; and predicting and generating a denoised driving path point sequence based on the first feature code.

[0128] In this embodiment of the invention, the second training module uses evaluator training samples to train the evaluator model to select the optimal driving path point sequence from multiple second driving path point sequences to obtain the first evaluator model. This can include: inputting second driving perception data and corresponding multiple second driving path point sequences into the world model to obtain driving perception data prediction results for each of the multiple second driving path point sequences; calling the reward module to calculate the predicted reward value of the corresponding second driving path point sequence based on the driving perception data prediction results; selecting the second driving path point sequence with the highest predicted reward value among the multiple second driving path point sequences as the optimal driving path point sequence; calculating the second model learning loss value of the world model based on the optimal driving path point sequence and the second sample path point sequence at future times corresponding to the second driving perception data; updating the model parameters of the world model using the second model learning loss value until the second iteration termination condition is met to obtain the first world model; and constructing the first evaluator model using the first world model and the reward module.

[0129] In this embodiment of the invention, the first construction module uses a first diffusion model to generate multiple second driving path point sequences at future moments of the second driving perception data based on the second driving perception data, and constructs evaluator training samples based on the second driving perception data and the corresponding second driving path point sequences. This can include: determining second driving state features from the second driving perception data; generating multiple second driving path point sequences based on the second driving state features using the first diffusion model; encoding the second driving path point sequences into action embedding features using a trajectory encoder; and constructing multiple state-action pairs from the second driving state features and the corresponding multiple action embedding features to obtain evaluator training samples.

[0130] In this embodiment of the invention, the second training module inputs the second driving perception data and the corresponding multiple second driving path point sequences into the world model to obtain the driving perception data prediction results corresponding to each of the multiple second driving path point sequences. This may include: obtaining a state-action pair, flattening the second driving state features and connecting them with the action embedding features to obtain a first feature vector; inputting the first feature vector into the world model so that the world model recursively outputs the driving perception data prediction results and the action prediction values ​​at the next moment until the driving perception data prediction results corresponding to the second driving path point sequences are obtained.

[0131] In this embodiment of the invention, the second training module calls the reward module to calculate the predicted reward value of the corresponding second driving path point sequence based on the prediction result of the driving perception data. This may include: calling the reward module to calculate the imitation reward value and the simulation reward value of the corresponding second driving path point sequence based on the prediction result of the driving perception data; determining the predicted reward value based on the imitation reward value and the simulation reward value; wherein, the imitation reward value is the reward value of the degree to which the second driving path point sequence imitates the second sample path point sequence; and the simulation reward value is the reward value determined based on the driving safety conditions provided by the autonomous driving simulator.

[0132] In this embodiment of the invention, the second training module calculates the second model learning loss value of the world model based on the optimal driving path point sequence and the second sample path point sequence at future times corresponding to the second driving perception data. This may include: obtaining the second sample driving perception data at future times corresponding to the second driving perception data; calculating the perception state supervision loss value of the second driving perception data compared to the second sample driving perception data; calculating the first imitation reward value of multiple second driving path point sequences corresponding to the second driving perception data for the second sample path point sequence, where the optimal driving path point sequence corresponds to the second imitation reward value; and calculating the cross-entropy loss based on the first and second imitation reward values ​​to obtain the model learning loss value. The simulation reward supervision loss is calculated as follows: The simulation reward supervision loss is calculated by simulating the third driving scenario data corresponding to multiple second driving path point sequences of the second driving perception data using a simulator; the first simulated reward value corresponding to the second driving path point sequence is calculated based on the third driving scenario data, where the optimal driving path point sequence corresponds to the second simulated reward value; cross-entropy loss is calculated based on the first and second simulated reward values ​​to obtain the simulation reward supervision loss value; the trajectory difference loss value between the optimal driving path point sequence and its corresponding second sample path point sequence is calculated; and the second model learning loss value is calculated based on the perception state supervision loss value, the simulation reward supervision loss value, the simulation reward supervision loss value, and the trajectory difference loss value.

[0133] For a description of the features in the embodiment corresponding to the driving decision model optimization device, please refer to the relevant description of the embodiment corresponding to the driving decision model optimization method, which will not be repeated here.

[0134] Embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-described embodiments of the driving decision model optimization method.

[0135] Embodiments of the present invention also provide a non-volatile storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described embodiments of the driving decision model optimization method when running.

[0136] In one exemplary embodiment, the aforementioned non-volatile storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0137] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described embodiments of the driving decision model optimization method.

[0138] Embodiments of the present invention also provide another computer program product, including a non-volatile storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described embodiments of the driving decision model optimization method.

[0139] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0140] The above provides a detailed description of the driving decision-making model optimization method and electronic device provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only intended to help understand the method and core ideas of this invention. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the protection scope of this invention.

Claims

1. A method for optimizing a driving decision-making model, characterized in that, include: Obtain driving data samples; Using the driving data samples, a diffusion model is trained to perform denoising processing on the input first driving perception data and first noisy data to generate a first driving path point sequence of the first driving perception data at future times, thus obtaining the first diffusion model; The first diffusion model is used to generate multiple sequences of second driving path points for future moments based on the second driving perception data. Evaluator training samples are constructed based on the second driving perception data and the corresponding sequences of second driving path points. Using the evaluator training samples, the evaluator model is trained to select the optimal driving path point sequence from multiple second driving path point sequences to obtain the first evaluator model. A driving decision model is constructed using the first diffusion model and the first evaluator model; Specifically, using the driving data samples, a diffusion model is trained to perform denoising processing on the input first driving perception data and first noisy data to generate a first driving path point sequence for future moments of the first driving perception data, thus obtaining a first diffusion model, including: Extract the first sample path point sequence corresponding to the future moment of the first driving perception data from the driving data sample, add noise to the first sample path point sequence, and obtain the first noisy data. The first driving perception data and the corresponding first noisy data are input into the diffusion model, and the first driving path point sequence is output. Calculate the first model learning loss value of the diffusion model based on the first driving path point sequence and the corresponding first sample path point sequence; The model parameters of the diffusion model are updated using the learning loss value of the first model until the first iteration termination condition is met, thus obtaining the first diffusion model. Using the training samples of the evaluator, the evaluator model is trained to select the optimal driving path point sequence from multiple second driving path point sequences to obtain the first evaluator model, including: The second driving perception data and the corresponding multiple second driving path point sequences are input into the world model to obtain the driving perception data prediction results corresponding to each of the multiple second driving path point sequences. The reward module is invoked to calculate the predicted reward value of the corresponding second driving path point sequence based on the prediction results of the driving perception data; Select the second driving path point sequence with the highest predicted reward value from multiple second driving path point sequences as the optimal driving path point sequence; The second model learning loss value of the world model is calculated based on the optimal driving path point sequence and the second sample path point sequence at future times corresponding to the second driving perception data. The model parameters of the world model are updated using the loss value learned from the second model until the second iteration termination condition is met, thus obtaining the first world model. The first evaluator model is constructed using the first world model and the reward module.

2. The driving decision model optimization method according to claim 1, characterized in that, The first driving perception data and the corresponding first noisy data are input into the diffusion model, and the first driving path point sequence is output, including: The first driving perception data and the corresponding first noisy data are input into the diffusion model, and multiple first driving path point sequences and their corresponding predicted classification scores are output. Based on the first driving path point sequence and the corresponding first sample path point sequence, the first model learning loss value of the diffusion model is calculated, including: Multiple first driving path point sequences are matched with the first sample path point sequence respectively. The first driving path point sequence that is closest to the first sample path point sequence is determined as a positive sample, and the other first driving path point sequences are determined as negative samples. The positive sample and the negative sample are used as the sample labels of the first driving path point sequence. Determine the first distance loss of the first driving path point sequence and the corresponding first sample path point sequence, and calculate the trajectory recovery loss value based on the sample label of the first driving path point sequence and the first distance loss; The classification confidence is obtained by calculating the cross-entropy based on the predicted classification score corresponding to the first driving path point sequence and the sample label of the first driving path point sequence. The first model learning loss value is calculated based on the trajectory recovery loss value and the classification confidence value.

3. The driving decision model optimization method according to claim 1, characterized in that, The first driving perception data and the corresponding first noisy data are input into the diffusion model, and the first driving path point sequence is output, including: The first driving perception data and the corresponding first noisy data are input into the diffusion model to perform multi-level denoising calculation, and the denoised driving path point sequence obtained by the last level of denoising calculation is the first driving path point sequence. The first-level denoising calculation includes: Cross-attention calculation is performed on the input noisy data and the perception map features in the first driving perception data to obtain the first attention calculation result; wherein, corresponding to the first level of denoising calculation, the noisy data is the first noisy data; corresponding to the second level and subsequent denoising calculations, the noisy data is the denoised driving path point sequence output by the previous level of the denoising network; the perception map features include at least one of bird's-eye view features and perspective view features. Cross-attention calculation is performed on the first attention calculation result and the driving data query information in the first driving perception data to output a second attention calculation result; the driving data query information includes at least one of vehicle query information and map query information. The first feature code is obtained by performing encoding calculation based on the second attention calculation result; The denoised driving path point sequence is generated based on the first feature encoding prediction.

4. The driving decision model optimization method according to claim 1, characterized in that, Using the first diffusion model, multiple sequences of second driving path points at future moments of the second driving perception data are generated based on the second driving perception data. Evaluator training samples are constructed based on the second driving perception data and the corresponding second driving path point sequences, including: The second driving state characteristics are determined from the second driving perception data; The first diffusion model is used to generate multiple sequences of second driving path points based on the second driving state features; The second driving path point sequence is encoded into action embedding features using a trajectory encoder. The second driving state feature and the corresponding multiple action embedding features are used to construct multiple state-action pairs to obtain the evaluator training samples.

5. The driving decision model optimization method according to claim 4, characterized in that, The second driving perception data and the corresponding multiple second driving path point sequences are input into the world model to obtain the driving perception data prediction results corresponding to each of the multiple second driving path point sequences, including: Obtain a state-action pair, flatten the second driving state feature and connect it with the action embedding feature to obtain a first feature vector; The first feature vector is input into the world model, so that the world model recursively outputs the driving perception data prediction result and the action prediction value of the next moment until the driving perception data prediction result corresponding to the second driving path point sequence is obtained.

6. The driving decision model optimization method according to claim 1, characterized in that, The reward module is invoked to calculate the predicted reward value of the corresponding second driving path point sequence based on the prediction results of the driving perception data, including: The reward module is invoked to calculate the imitation reward value and simulated reward value of the corresponding second driving path point sequence based on the prediction results of the driving perception data; The predicted reward value is determined based on the imitation reward value and the simulated reward value; Wherein, the imitation reward value is the reward value of the degree to which the second driving path point sequence imitates the second sample path point sequence; the simulation reward value is the reward value determined according to the driving safety conditions provided by the autonomous driving simulator.

7. The driving decision model optimization method according to claim 6, characterized in that, Based on the optimal driving path point sequence and the second sample path point sequence at future times corresponding to the second driving perception data, the second model learning loss value of the world model is calculated, including: Obtain the second sample driving perception data corresponding to the second driving perception data at a future time, and calculate the perception state supervision loss value of the second driving perception data compared to the second sample driving perception data; Calculate the first imitation reward value of the second sample path point sequence for the multiple second driving path point sequences corresponding to the second driving perception data, wherein the optimal driving path point sequence corresponds to the second imitation reward value. The imitation reward supervision loss value is obtained by calculating the cross-entropy loss based on the first imitation reward value and the second imitation reward value. The simulator is used to simulate the third driving scenario data corresponding to multiple second driving path point sequences corresponding to the second driving perception data, so as to calculate the first simulated reward value corresponding to the second driving path point sequence based on the third driving scenario data, wherein the optimal driving path point sequence corresponds to the second simulated reward value. The simulated reward supervision loss value is obtained by calculating the cross-entropy loss based on the first simulated reward value and the second simulated reward value. Calculate the trajectory difference loss value between the optimal driving path point sequence and the corresponding second sample path point sequence; The second model learning loss value is calculated based on the perceived state supervision loss value, the imitation reward supervision loss value, the simulated reward supervision loss value, and the trajectory difference loss value.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the driving decision model optimization method as described in any one of claims 1 to 7 when executing the computer program.

Images