Decision model optimization method and device based on world model, medium and product

Abstract

The invention discloses a decision-making model optimization method and device based on a world model, a medium and a product, relates to the technical field of automatic driving, and aims to improve the ability of the world model to understand complex traffic scenes by training the world model in the first stage to understand structured traffic conditions. A world model is trained to predict a future driving scene according to structured traffic conditions and driving actions, and environment prediction and generation quality is improved; and based on the trained world model and the decision model, cooperatively constructing a closed-loop optimization framework based on the world model, inputting the driving action output by the decision model and the corresponding structured traffic condition into the world model, calculating the reward value of the driving action according to the obtained future driving scene sequence, and updating the decision model according to the reward value. According to the method, efficient closed-loop optimization of the decision-making model is realized, the perception capability and decision-making capability of the model can be remarkably improved, the safety and reliability of automatic driving decision-making are improved, and the automatic driving performance of the vehicle in a complex traffic environment is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a world model-based decision model optimization method, equipment, medium, and product. Background Art

[0002] Based on the world model's ability to understand and interact with the environment, using world models to assist autonomous driving decision-making has become a key approach in the field. However, current applications of world models in autonomous driving primarily rely on using them to generate future driving scenarios for the autonomous driving decision-making model to assist in its decision-making. Furthermore, world models are not very effective at modeling the actual environment, resulting in limited performance improvements for autonomous driving decision-making models based on world models.

[0003] How to improve the decision-making performance of the autonomous driving decision-making model based on the world model is a technical problem that technicians in this field need to solve. Summary of the Invention

[0004] The present invention provides a world model-based decision model optimization method, device, medium and product to at least solve the problem of poor decision performance of the world model-based autonomous driving decision model in the related art.

[0005] The present invention provides a decision model optimization method based on a world model, comprising: Obtain driving video samples; Using the driving video sample to train the initial world model, generating image frames at corresponding moments according to input structured traffic conditions, thereby obtaining a first world model; The first world model is trained using the driving video samples to generate a future driving scene sequence based on an input image sequence and the structured traffic conditions and driving actions at corresponding moments, thereby obtaining a second world model; The decision model is iteratively trained using the second world model. During the training, the driving actions output by the decision model and their corresponding structured traffic conditions are input into the second world model. A reward value for the driving action is calculated based on the obtained sequence of future driving scenarios, and the decision model is updated based on the reward value.

[0006] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned world model-based decision model optimization methods when executing the computer program.

[0007] The present invention also provides a non-volatile storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned decision model optimization methods based on the world model are implemented.

[0008] The present invention also provides a computer program product, comprising a computer program, which implements the steps of any of the above-mentioned decision model optimization methods based on the world model when the computer program is executed by a processor.

[0009] The present invention adopts a two-stage world model training method. In the first stage, the world model is trained to understand structured traffic conditions, thereby improving the world model's ability to understand complex traffic scenarios. On this basis, the world model is retrained to predict future driving scenarios based on structured traffic conditions and driving actions, thereby improving the environmental prediction and generation quality of the world model. Based on the trained world model and the decision model, a closed-loop optimization framework based on the world model is collaboratively constructed. The driving actions and their corresponding structured traffic conditions output by the decision model are input into the pre-trained second world model. The reward value of the driving action is calculated based on the obtained future driving scenario sequence, and the decision model is updated based on the reward value. This realizes efficient closed-loop optimization of the decision model, can significantly improve the model's perception and decision-making capabilities, improve the safety and reliability of autonomous driving decisions, and improve the vehicle's autonomous driving performance in complex traffic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0011] Figure 1 A flowchart of a decision model optimization method based on a world model provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of an autonomous driving decision-making system based on a world model provided by an embodiment of the present invention; Figure 3 A schematic diagram of a driving data set collection process provided by an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a denoising network provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of a downsampling module of a denoising network provided by an embodiment of the present invention; Figure 6 A schematic diagram of the structure of an intermediate encoding module of a denoising network provided by an embodiment of the present invention; Figure 7 A schematic diagram of the structure of an upsampling module of a denoising network provided by an embodiment of the present invention; Figure 8A schematic diagram of the structure of an attention module of a denoising network provided by an embodiment of the present invention; Figure 9 A schematic diagram of a world model generation and training process provided by an embodiment of the present invention; Figure 10 A schematic diagram of a prediction training process of a world model provided by an embodiment of the present invention; Figure 11 A schematic diagram of a closed-loop optimization process of a decision model based on a world model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0013] It should be noted that, in the description of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. The terms "first," "second," etc., in the present invention are used to distinguish similar objects, and are not used to describe a particular order or precedence.

[0014] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0015] World models are attracting widespread attention in the field of intelligent decision-making due to their ability to understand the environment and its interactions. World models have great potential for generating high-quality driving videos and for end-to-end driving. In related technologies, the use of world model agents in autonomous driving technology is mainly to predict future driving environment information through world models and assist in autonomous driving decision-making. However, the modeling method of the world model is to model the environmental dynamics as a series of discrete latent variables. Although the discretization of the latent space helps to avoid compound errors within a multi-step time range, this encoding may lose information, resulting in a loss of versatility and reconstruction quality. Therefore, the world model has many problems, such as the inability to truly realize the prediction of future driving environment information and the inability to support closed-loop optimization of the autonomous driving decision-making model. As a result, the investment of the world model has limited effect on improving the performance of autonomous driving decision-making tasks.

[0016] In order to improve the performance of autonomous driving decision-making based on a world model, the present invention provides a decision model optimization method, device, medium and product based on a world model. By adopting a two-stage world model training method, the first stage trains the world model to understand structured traffic conditions to improve the world model's ability to understand complex traffic scenarios. On this basis, the world model is trained again to predict future driving scenarios based on structured traffic conditions and driving actions, thereby improving the environmental prediction and generation quality of the world model. Based on the trained world model and decision model, a closed-loop optimization framework based on the world model is collaboratively constructed. The driving actions output by the decision model and their corresponding structured traffic conditions are input into a pre-trained second world model. The reward value of the driving action is calculated based on the obtained future driving scene sequence, and the decision model is updated based on the reward value. This realizes efficient closed-loop optimization of the decision model, which can significantly improve the model's perception and decision-making capabilities, improve the safety and reliability of autonomous driving decisions, and improve the vehicle's autonomous driving performance in complex traffic environments.

[0017] An embodiment of the present invention provides a decision model optimization method based on a world model. The method is described in detail below in conjunction with the execution flow of the decision model optimization method based on a world model.

[0018] Figure 1 A flowchart of a decision model optimization method based on a world model provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of an autonomous driving decision-making system based on a world model provided by an embodiment of the present invention; Figure 3 A schematic diagram of a driving data set collection process provided by an embodiment of the present invention.

[0019] like Figure 1 As shown, a decision model optimization method based on a world model provided by an embodiment of the present invention may include: S101: obtaining a driving video sample.

[0020] S102: Using driving video samples to train an initial world model, an image frame at a corresponding moment is generated according to input structured traffic conditions to obtain a first world model.

[0021] S103: Using driving video samples to train the first-world model, a future driving scene sequence is generated based on the input image sequence and the structured traffic conditions and driving actions at the corresponding time, thereby obtaining a second-world model.

[0022] S104: The decision model is iteratively trained using the second-world model. During the training, the driving action output by the decision model and its corresponding structured traffic conditions are input into the second-world model. The reward value of the driving action is calculated based on the obtained future driving scenario sequence, and the decision model is updated based on the reward value.

[0023] like Figure 2 As shown, the model training steps in the world model-based decision model optimization method provided by the embodiment of the present invention can be applied to the model training device. The model training device can adopt one or more artificial intelligence servers, which may include artificial intelligence processors, storage, input and output devices, communication buses, communication interfaces and other basic settings, and are composed of software environments such as operating systems, databases, middleware and application software.

[0024] In an embodiment of the present invention, the environment is modeled as a standard partially observable Markov decision process. ,in is a collection of states, is a collection of actions. is the set of environmental observations. State transition function Describing environmental dynamics , the reward function Mapping state transitions to scalar rewards.

[0025] Since the intermediate process cannot directly access the state , can only be observed through environmental Perceiving the dynamics of the environment, these observations are based on the observation probability Emitted, described by the observation function The goal is to obtain a policy π that maps observations to actions to maximize the expected return ,in, is the discount factor.

[0026] The world model is a generative model of the environment, i.e. , using the model as a simulation environment to train autonomous decision-making systems in a sample-efficient manner.

[0027] Based on this, the decision model optimization process provided by the embodiment of the present invention mainly consists of three steps: S101 is to collect driving data to build a dataset; S102 and S103 are to train a world model on the driving dataset; S104 is to optimize the decision model in the environment of the world model.

[0028] For S101 , the driving video sample may be a driving video sample acquired based on a real driving scene. Figure 3 A method of obtaining driving video samples is shown.

[0029] like Figure 3As shown, the vehicle serves as the intelligent agent, controlled by a human driver. During driving, various onboard sensor modules collect driving data, which is then timestamped and preprocessed in the onboard edge server. Given the limited computing and storage resources of the onboard edge server, data is temporarily cached there. After preprocessing, it is synchronously uploaded to the model training device to construct a dataset for training the world model. Onboard sensor modules can include, but are not limited to, radar, cameras, vehicle control units, inertial measurement units, and navigation systems. Radar and cameras collect environmental perception data and obtain vehicle control signals from the onboard control unit. The inertial measurement unit and navigation system collect vehicle motion status. The environmental perception data, along with vehicle motion status, are input into the data preprocessing module of the onboard edge server. The multimodal data is timestamped and structured traffic condition data (such as objects and lane lines) is detected. Multi-view images, road topology images, traffic target data, and vehicle control information are then transmitted to the model training device for storage in the model training device to construct the driving dataset. The model training device receives manual annotations of the driving dataset, or automatically annotates it using the automatic annotation module to add scene description information. The model training device's world model update module batch samples the driving dataset from the storage module and performs conditional information encoding and diffusion generation training on the sampled multimodal data.

[0030] Based on Figure 3 The dataset collection process shown in the figure, each driving video sample in the dataset can include the following information: ① Multi-view images: The multi-view image information of the vehicle during driving can be obtained in real time by multiple on-board cameras. The collected images can be in red, green, blue (RGB) format. If 6 on-board cameras are used, they can correspond to the 6 perspectives of the vehicle respectively: front (F), left front (FL), right front (FR), back (B), left back (BL), right back (BR). Each image is in The image is stored in the form of , where H and W are the height and width of the image respectively. In addition, the corresponding view mark and time step mark are also included when the image is stored.

[0031] ② Road topology image: Extract the corresponding road topology information from multi-view images through manual annotation or related methods, including lane boundaries, lane dividers and crosswalks. The images are stored in the form of and aligned with the multi-view images according to the time step labels.

[0032] ③ Traffic target data: Extract road targets and category information from multi-view images through manual annotation or related methods, including pedestrians, vehicles, bicycles, animals, etc. Each target is annotated with a 3D box and projected onto the image plane. are stored in the form of is the number of target categories, and each image is aligned with the multi-view images according to the time step label.

[0033] ④ Vehicle control information: The vehicle control action at each time step is recorded by the onboard controller, including the steering angle and speed. The images are stored in the form of and aligned with the multi-view images according to the time step labels.

[0034] ⑤ Scene description information: Add scene description information based on multi-view images through manual annotation or related methods, including weather, time, etc., store it in text form, and align it with the multi-view images according to the time step mark.

[0035] During the data acquisition phase, images and other data can be collected and processed at a frequency of 12 Hz to obtain an image dataset. Multiple image data samples can be further aggregated into video data within a period of time according to a set number of frames, thereby dividing the image dataset into several video clip samples to obtain a driving video dataset. Each driving video sample contains a series of multi-view images aligned in time steps, road topology images, traffic target data, vehicle control, and scene description text data.

[0036] For S102 and S103, the present embodiment trains the world model in two stages. The first stage is generative training, focusing on learning single-frame structured traffic conditions. This training trains the world model to generate image frames of the driving scene at the corresponding moment based on the input structured traffic conditions. Building on this first-stage training, the second stage incorporates driving actions, training the world model to predict future driving scene information based on the input structured traffic conditions and driving actions. This gradually guides the world model to understand the impact of autonomous driving perception data and driving actions on the driving environment, improves the accuracy of the world model's generation of future driving scene information, and provides a foundation for closed-loop optimization of the decision-making model.

[0037] In an embodiment of the present invention, the generation unit of the world model can adopt a diffusion model, a vector quantized variational autoencoder (VQ-VAE), or an autoregressive of a Transformer-based generation model. If a variational autoencoder is used as the generation unit, the continuous environment state can be compressed into a vector in a discrete codebook, and the latent variables can be discretized through quantization. If an autoregressive of a Transformer-based generation model is used as the generation unit, the environment state (such as an image frame) can be segmented into a sequence of discrete tokens, and future tokens can be predicted through autoregression.

[0038] For S104, Figure 2 As shown, when training the decision model, the decision model interacts and explores with the second world model in the model training device for iterative training. The decision model makes decisions based on the prediction results output by the second world model and optimizes the decision strategy based on the cumulative return.

[0039] In the iterative training introduced in various embodiments of the present invention, the iteration end condition may be reaching a preset number of iterations or satisfying a preset convergence condition (the loss value is less than a preset loss).

[0040] like Figure 2 As shown in the figure, the trained decision model and the feature encoding module of the second-world model are deployed to the vehicle edge server. During the actual operation of the autonomous driving system, the vehicle interacts with the real environment in real time. The vehicle's sensor module collects environmental perception data from the real environment and transmits this perception data to the vehicle edge server via a data bus for processing. At the same time, the vehicle receives action commands from the vehicle edge server via the Controller Area Network (CAN) bus and transmits them to the vehicle control drive module to control the vehicle to execute the corresponding driving action.

[0041] After receiving the perception data sent by the vehicle, the on-board edge server first performs preprocessing on the original sensor data, such as format conversion, through the data preprocessing module. The obtained multimodal data is sent to the feature encoding module for processing into low-dimensional latent state features, and then input into the decision model for action reasoning and output driving actions.

[0042] The world model-based decision model optimization method provided by the present invention utilizes a two-stage world model training approach. In the first stage, the world model is trained to understand structured traffic conditions, thereby improving the world model's ability to understand complex traffic scenarios. Furthermore, the world model is trained to predict future driving scenarios based on structured traffic conditions and driving maneuvers, thereby improving the world model's environmental prediction and generation quality. A closed-loop optimization framework based on the trained world model and decision model is collaboratively constructed. The driving maneuvers and their corresponding structured traffic conditions output by the decision model are input into a pre-trained second world model. The reward value for the driving maneuvers is calculated based on the resulting future driving scenario sequence, and the decision model is updated based on the reward value. This achieves efficient closed-loop optimization of the decision model, significantly improving the model's perception and decision-making capabilities. Finally, an end-to-end autonomous decision-making system is constructed by combining the feature encoding module of the world model for vehicle-side deployment, promoting the practical application of autonomous decision-making technology and improving vehicle performance in complex traffic environments.

[0043] Based on the above embodiment, in an embodiment of the present invention, in S102, using the driving video sample to train the initial world model to generate an image frame at a corresponding moment according to the input structured traffic condition to obtain a first world model may include extracting a first image feature from the first image frame in the driving video sample; encoding the structured traffic condition corresponding to the first image frame to obtain a first conditional encoding result; using the initial world model to generate a first generated image based on the first image feature and the first conditional encoding result; calculating a first loss value of the first generated image compared to the first image frame, and using the first loss value to update the initial world model until a first iteration end condition is met to obtain the first world model.

[0044] In some optional implementations of the embodiments of the present invention, the generation unit of the initial world model can be a diffusion model. In this case, using the initial world model to generate the first generated image according to the first image feature and the first conditional encoding result can include: adding noise to the first image feature to obtain a first noisy feature; using the initial world model to perform denoising calculation according to the first image feature and the first conditional encoding result to obtain the first generated image.

[0045] In order to enable a world model to process multimodal data to accurately generate future driving scene information, an embodiment of the present invention provides a diffusion model structure.

[0046] Assume that the data distribution is , Represents the potential features of the image, let It is expressed as adding variance to the input data The data distribution after independent and identically distributed Gaussian noise, given a sufficiently large noise variance ,have Diffusion model from high variance noise Start by denoising the noisy data step by step to , and its iterative optimization process is expressed as the following probability flow differential equation: ; in, is the scoring function. The training of the diffusion model is simplified to the scoring function Training a model , the model is parameterized as follows: ; It is a method for predicting the original A learnable denoiser for , trained with the following denoising score matching objective: ; in, , represents the probability distribution over the noise level, is a weight function, is a conditional signal, including road topology characteristics , target condition , and text hint embedding .

[0047] The embodiment of the present invention adopts a conditional preprocessing framework to Parameterized as follows: ; in, is the network to be trained, and It is a pre-conditioner used to adjust the network input and output to any noise level. Under the condition of maintaining unit variance, it can be expressed as: , .

[0048] is the empirical transformation of the noise level and can be expressed as: .

[0049] is with The associated noise figure can be expressed as: .

[0050] The embodiment of the present invention provides a condition processing module for a driving video dataset. , each driving video sample can be expressed as , including continuous Frame image, through a variational autoencoder encoder (VAE) based image encoder to obtain the potential representation of each image frame in the video clip , given by the diffusion time step Parameterized noise schedule , the noisy samples are obtained through the forward diffusion process , diffusion model take over And optimize the trainable parameters in the model by minimizing the following denoising score matching objective .

[0051] The present invention uses the UNet framework as the backbone network of the diffusion model, and uses the structured traffic information projected onto the image plane as the conditional input to construct the diffusion model. The following describes the denoising network of the diffusion model used in the embodiments of the present invention.

[0052] Figure 4 A schematic diagram of the structure of a denoising network provided by an embodiment of the present invention. Figure 4 As shown, the embodiment of the present invention adopts a diffusion model backbone network based on the UNet framework, which is mainly composed of a downsampling module, an intermediate encoding module and an upsampling module.

[0053] Figure 5 The schematic diagram of the structure of a downsampling module of a denoising network provided by an embodiment of the present invention. The downsampling module is responsible for performing dimensionality reduction coding on the input data, such as Figure 5 As shown in the figure, after the noisy features and diffusion time steps are encoded separately, they are processed by a residual (Resnet) network block. The results are input into an attention module together with the position embedding and prompt embedding to obtain the latent features. On the one hand, the latent features are used as the input of the subsequent network module, and on the other hand, they are stored as intermediate results for feature calculation in the upsampling module. Figure 5 In the cascade, ×2 represents two identical network structures, and ×3 represents three identical network structures. The same network structure means that the computing operations performed are consistent, but the network parameters of the network structures at different positions may be different.

[0054] Figure 6 This is a schematic diagram of the structure of an intermediate encoding module of a denoising network provided by an embodiment of the present invention. The intermediate encoding module is responsible for further processing the data in the low-dimensional feature space. The input data of this part includes the latent features from the downsampling module, as well as the encoded time step embedding, position embedding and prompt embedding. Figure 6As shown in the figure, the potential features and time step embeddings are processed by a residual (Resnet) network block, and the results are input into an attention module together with the position embedding and prompt embedding. They are then processed again by a residual (Resnet) network block to obtain the potential features of this part of the output.

[0055] Figure 7 This is a schematic diagram of the structure of an upsampling module of a denoising network provided by an embodiment of the present invention. The upsampling module is responsible for restoring the encoded features from the low-dimensional space to the original input state space. The input data of this part includes the latent features from the intermediate encoding module, the encoded time step embedding, position embedding and prompt embedding, and the intermediate features from the downsampling module. Figure 6 As shown in the figure, the latent features are concatenated with the intermediate features. The concatenated latent features and time-step embeddings are processed through a series of residual (ResNet) network blocks. The resulting results, along with the position embedding and cue embedding, are input into an attention module to obtain the latent features. Finally, the latent features are further processed through a convolutional layer to obtain the final predicted features.

[0056] Figure 8 A schematic diagram of the structure of an attention module of a denoising network provided by an embodiment of the present invention. Figure 8 As shown, in the denoising network of the diffusion model of the embodiment of the present invention, the attention modules used by each network module all use the gated self-attention mechanism to integrate the potential features of each level. and positional embedding Putting it all together: ; in, is a learnable parameter, represents the self-attention operation, represents a label selection operation that only considers visual labels, Right now Figure 8 The activation function in .

[0057] Furthermore, the attention module uses cross-attention operations to perform feature interactions between multimodal input data, such as feature interactions between text input and visual signals, so that text descriptions can influence driving scene attributes such as weather and time, thereby solving the problem of world models processing multimodal input data.

[0058] Figure 9 A schematic diagram of a world model generation and training process provided by an embodiment of the present invention.

[0059] The world model based on the diffusion model, in the first stage of generative training, the model only considers single-frame input from structured traffic conditions. This stage focuses on learning traffic structure constraints, which helps to accelerate convergence.

[0060] In the embodiment of the present invention, Figure 9 As shown, encoding the structured traffic conditions corresponding to the first image frame to obtain a first conditional encoding result may include: extracting a first background feature from a first road topology map corresponding to the first image frame; concatenating the first noisy feature and the first background feature to obtain a second noisy feature; encoding first target information of a first target included in the first image frame to obtain a second conditional encoding result; spatially aligning the second conditional encoding result with the second noisy feature to obtain a first position embedding feature; and using the first background feature and the first position embedding feature as the first conditional encoding result. Using the initial world model to perform denoising calculations based on the first image features and the first conditional encoding result to obtain a first generated image may include: using the initial world model to perform denoising calculations based on the second noisy feature and the second conditional encoding result to obtain the first generated image.

[0061] That is, the structured traffic conditions used in the embodiment of the present invention may include road topology conditions. and target conditions . Among them, the road topology condition can be a road topology map (HDMaps). The target condition is the relevant information of the traffic behavior target (such as pedestrians and vehicles) in the driving scene, which can include target position conditions and target category conditions. The road topology condition and the target condition can be manually labeled information, or they can be obtained by using perception methods (such as the Latent Action Value (LAV) model, the bird's-eye view annotation model (BEVerse), the Unified Autonomous Driving (UniAD) model), etc. The road topology condition can be a three-channel (including lane boundaries, lane dividers and crosswalks), and the target position condition can be a three-dimensional box. The three-channel road topology map and the three-dimensional box are projected onto the image plane to generate the corresponding conditions.

[0062] In order to reduce the complexity of condition processing, the number of targets in each scene can be limited, and the target condition can be split into target (three-dimensional box) position conditions. and target category conditions ,in is the predefined maximum number of detection boxes with zero padding.

[0063] like Figure 9 As shown, a single frame of real image is obtained in the driving video sample, which is recorded as the first image frame. The first image feature (i.e., potential feature) is extracted from the first image frame by the variational autoencoder. Based on random noise, given by the diffusion time step Parameterized noise schedule , and obtain the noise added to the first image feature.

[0064] In the first stage of generative training of the world model, the road topology condition As the first road topology graph, the road topology condition After processing by a feature encoder based on a convolutional network, the first background feature and the second noisy feature are obtained. Splicing is performed to obtain the second noisy feature.

[0065] In an embodiment of the present invention, encoding first target information of a first target included in a first image frame to obtain a second conditional encoding result may include: encoding position information in the first target information to obtain a first position encoding result; encoding category information in the first target information to obtain a first category encoding result; and using the first position encoding result and the first category encoding result as a second conditional encoding result. Spatially aligning the second conditional encoding result with a second noisy feature to obtain a first position embedded feature may include: concatenating the first position encoding result and the first category encoding result, and aggregating the concatenated features using a linear layer to obtain the second conditional encoding result.

[0066] Due to the target position conditions With the first noisy feature In the absence of spatial alignment, the target position condition is firstly conditioned by Fourier encoding and contrastive language-image pre-training (CLIP) encoding. and target category conditions After processing and introducing learnable parameters, the two features are concatenated and the concatenated features are aggregated using a linear layer to obtain position embedding: ; in, Represents the MLP layer calculation operation, represents the Fourier encoding operation, To detect the category features of the frame, the CLIP network is used to process: .

[0067] In an embodiment of the present invention, the input driving data may also include a text description, such as a text description of the driving scene's weather, time, and location. Using the initial world model to perform a denoising calculation based on the first image features and the first conditional encoding result to obtain a first generated image may include: encoding the first text description information corresponding to the first image frame to obtain a first hint embedding feature; and using the initial world model to perform a denoising calculation based on the first image features, the first conditional encoding result, and the first hint embedding feature to obtain the first generated image.

[0068] The structured traffic conditions (first road topology map, three-dimensional bounding box, and object category) and text description are encoded separately and input into a denoising network along with noisy features obtained by adding noise to the first image frame. Denoising is performed to obtain first predicted features. Calculating a first loss value of the first generated image compared to the first image frame may include calculating a loss value of the first predicted features obtained by the denoising calculation compared to the first image features to obtain the first loss value.

[0069] Based on the first stage of training, the model has already understood structured traffic information. However, an ideal world model should be able to predict the future and interact with the environment. In an embodiment of the present invention, using a driving video sample to train a first world model to generate a future driving scene sequence based on the input image sequence and the structured traffic conditions and driving actions at the corresponding time to obtain a second world model can include: encoding the structured traffic conditions at the corresponding time of the first second image frame in the second image sequence of the driving video sample to obtain a third condition encoding result; encoding the first driving action sequence at the corresponding time of the second image sequence to obtain an action encoding result; generating future structured traffic conditions based on the third condition encoding result and the action encoding result; using the first world model to generate a future driving scene sequence based on the future structured traffic conditions; calculating a second loss value of the future driving scene sequence compared to the third image sequence at the corresponding time in the driving video sample, and using the second loss value to update the first world model until the second iteration end condition is met to obtain the second world model.

[0070] The first-world model obtained after the first stage of training can output the image frame at the corresponding moment according to the input structured traffic conditions. 、 , you can generate a driving video However, in practical applications, structured traffic conditions beyond the current time step cannot be obtained, while the task requirement of the embodiment of the present invention is to predict future driving scene information, such as generating future videos.

[0071] To solve this problem, the embodiment of the present invention introduces a traffic prediction module, which uses driving behavior To iteratively predict future structured traffic conditions. In an embodiment of the present invention, generating future structured traffic conditions based on the third condition encoding result and the action encoding result includes: using the third condition encoding result as the initial hidden state variable, performing a cross-attention calculation based on the hidden state variable and the action encoding result at the corresponding moment and converting it into a first latent variable of a Gaussian distribution; performing a gated loop calculation based on the first latent variable and the hidden state variable at the corresponding moment to obtain the hidden state variable at the next moment, until the hidden state variable at the next moment of the last second image frame of the second image sequence is obtained; and concatenating the hidden state variable with the action encoding result at the previous moment and performing a decoding calculation to obtain the future structured traffic conditions.

[0072] Among them, using the first-world model to generate a future driving scene sequence based on future structured traffic conditions can include: using the first-world model to output prediction features based on future structured traffic conditions; decoding the prediction features into the original image space corresponding to the driving video sample to obtain a future driving scene video including multiple future image frames; connecting the prediction features with the action coding results corresponding to the second image sequence and then performing decoding calculations to obtain a future driving action sequence at future moments of the second image sequence; and using the future driving scene video and the future driving action sequence as the future driving scene sequence.

[0073] That is to say, the first-world model can be used to predict future driving scene videos and future driving action sequences respectively, and based on the future driving scene videos and future driving action sequences, two types of losses can be calculated to speed up the convergence of the second-stage training.

[0074] That is, calculating the second loss value of the future driving scene sequence compared to the third image sequence at the corresponding moment in the driving video sample may include: calculating the third loss value of the future driving scene video compared to the third image sequence at the corresponding moment in the driving video sample; calculating the fourth loss value of the future driving action sequence compared to the second driving action sequence at the corresponding moment in the driving video sample; and calculating the second loss value based on the third loss value and the fourth loss value.

[0075] Similar to the above embodiment, text prompt conditions can also be introduced. In addition to the structured traffic conditions and text prompt conditions generated by the traffic prediction module, reference image conditions can also be introduced. , to enhance the background consistency in the generated future video. Then, using the first-world model to generate a future driving scene sequence based on future structured traffic conditions, the method may include: encoding the reference background image corresponding to the driving video sample to obtain a reference style embedding feature; and using the first-world model to generate a future driving scene sequence based on the reference style embedding feature and future structured traffic conditions. Reference image condition The first second image frame in the second image sequence may be used to constrain the first world model to generate the background of the future video. Alternatively, a background image of a specific traffic scene may be generated based on the current traffic scene as a reference background image.

[0076] Figure 10 A schematic diagram of a prediction training process of a world model provided by an embodiment of the present invention.

[0077] like Figure 10 As shown, in the second stage of training the world model, the video prediction task is used to build the world model. Provide the initial observation reference image condition corresponding to the first second image frame in the second image sequence , road topology conditions , target conditions and driving actions , the desired result is the future driving video , and future driving behavior .

[0078] Introducing traffic prediction module, using driving behavior , iteratively predict future structural conditions.

[0079] First, the initial structured traffic conditions 、 Encoded and expanded into a one-dimensional latent space (i.e. Figure 10 The encoded features are then concatenated and processed sequentially through self-attention and linear layers to obtain the hidden state .

[0080] Then, using the cross attention layer To build the association between hidden state and driving behavior.

[0081] Then the latent variable Parameterized as the following Gaussian distribution: ; in, and is the network layer used to learn the parameters of the Gaussian distribution, is the identity matrix.

[0082] In order to predict the future hidden state, the traffic prediction module uses the Gated Recurrent Unit (GRU) unit to iteratively update: .

[0083] Hide state and action characteristics The features are connected together and processed by the decoder, and the decoded features are finally used as the future traffic structure conditions. Because the traffic prediction module predicts future traffic conditions at the feature level, it can reduce pixel-level noise interference and produce more robust predictions.

[0084] In addition to the structured traffic conditions and text prompt conditions (prompt embedding) generated by the traffic prediction module, the reference image conditions are introduced (Ref. Style Embedding), enhancing background consistency in generated future videos.

[0085] Based on the above conditions, the first-world model trained in the first stage is expanded so that the model can jointly generate future driving videos. and driving behavior Using the denoising network trained in the first stage, the input conditions are fused and the features of the next moment are predicted through temporal attention calculation, gated self-attention calculation and cross-attention calculation. Among them, the video prediction features and action prediction features are respectively distributed with Gaussian distribution. and Laplace distribution To model, the mean square error and L1 loss are used to construct the loss function for video prediction training. 、 These are the learnable layers involved in the traffic prediction module, diffusion model, video decoder, and motion decoder.

[0086] For the video prediction part, the image decoder of the variational autoencoder is used to decode the predicted features into the original image space.

[0087] For the action prediction part, the predicted features are concatenated with the historical action features and then decoded through a linear layer to generate future driving actions.

[0088] To achieve closed-loop optimization of the decision-making agent within the world model, a reward predictor is required to complete the final construction of the world model. Since estimating rewards is a scalar prediction problem, and considering the difficulty of obtaining objectively evaluated reward labels in practical applications, this embodiment of the present invention utilizes an image-based reward function to achieve reward state prediction.

[0089] Figure 11 A schematic diagram of a closed-loop optimization process of a decision model based on a world model provided in an embodiment of the present invention.

[0090] In the embodiment of the present invention, Figure 11As shown, a reward module is constructed to calculate the driving safety factor of the vehicle at the future moment based on image frames in the future video generated by the second-world model. Specifically, it can identify the positional relationship between the future moment and lane lines and other traffic behavior targets to determine whether the vehicle at the future moment is driving in the specified lane and whether there is a risk of collision with other traffic behavior targets.

[0091] The above-described embodiment describes how, after training a world model using a collected driving video dataset, a virtual traffic environment is generated using the world model. A reinforcement learning interactive exploration training approach is then employed to iteratively optimize the autonomous decision-making system within this virtual environment, enabling learning within imagination. Given that the world model includes an encoding and processing module for driving perception data, only the decision-making module of the autonomous decision-making system needs to be trained at this stage. When the model is deployed, the image encoding module used to encode RGB video image data in the second world model can be connected to the decision-making module. The latent features encoded by the former are directly passed to the latter as state inputs, thus forming an end-to-end inference architecture.

[0092] like Figure 11 In the world model-based closed-loop optimization framework shown, during interactive training of the decision model, the potential prediction features output by the backbone network of the second world model are directly used as input to the decision model, avoiding redundant perceptual encoding operations during the decision model inference process and simplifying the computational process. During reward calculation, the world model relies on the decoder to generate future predictions for environmental feedback and action evaluation.

[0093] In an embodiment of the present invention, updating the decision model according to the reward value in S104 may include: calculating the cumulative reward value corresponding to the driving action according to the reward value and the environmental state information in the future driving scene sequence; calculating the value network loss value according to the value network, environmental state information and cumulative reward value of the decision model; determining the weighted entropy of the policy network of the decision model according to the driving action parameters; calculating the policy network loss value according to the driving action parameters, the value network, environmental state information, cumulative reward value and weighted entropy of the decision model; and updating the decision model according to the value network loss value and the policy network loss value.

[0094] Calculating the cumulative reward value corresponding to the driving action based on the reward value and the environmental state information in the future driving scene sequence can include: determining the imagined field of view of the vehicle; for the first time step in the future driving scene sequence that is within the imagined field of view of the vehicle, determining the cumulative reward value at the corresponding moment based on the sixth reward value at the corresponding moment, the environmental state information at the next moment, and the cumulative reward value at the next moment; and for the second time step in the future driving scene sequence that is equal to the imagined field of view, determining the cumulative reward value at the corresponding moment based on the environmental state information at the corresponding moment.

[0095] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0096] An embodiment of the present invention also provides a decision model optimization device based on a world model, including: an acquisition module for acquiring driving video samples; a first training module for using the driving video samples to train the initial world model to generate image frames at corresponding moments based on the input structured traffic conditions, thereby obtaining a first world model; a second training module for using the driving video samples to train the first world model to generate a future driving scene sequence based on the input image sequence and the structured traffic conditions and driving actions at the corresponding moments, thereby obtaining a second world model; and a third training module for iteratively training the decision model using the second world model. During the training, the driving actions output by the decision model and their corresponding structured traffic conditions are input into the second world model, the reward value of the driving action is calculated based on the obtained future driving scene sequence, and the decision model is updated based on the reward value.

[0097] In an embodiment of the present invention, the first training module uses the driving video sample to train the initial world model to generate an image frame at a corresponding moment according to the input structured traffic condition to obtain the first world model, which may include: extracting a first image feature from the first image frame in the driving video sample; encoding the structured traffic condition corresponding to the first image frame to obtain a first conditional encoding result; using the initial world model to generate a first generated image based on the first image feature and the first conditional encoding result; calculating a first loss value of the first generated image compared to the first image frame, and using the first loss value to update the initial world model until the first iteration end condition is met to obtain the first world model.

[0098] In an embodiment of the present invention, the generation unit of the initial world model can be a diffusion model; the first training module uses the initial world model to generate a first generated image based on the first image feature and the first conditional encoding result, which may include: adding noise to the first image feature to obtain a first noisy feature; using the initial world model to perform denoising calculation based on the first image feature and the first conditional encoding result to obtain the first generated image.

[0099] In an embodiment of the present invention, the first training module encodes the structured traffic conditions corresponding to the first image frame to obtain a first conditional encoding result, which may include: extracting a first background feature from a first road topology map corresponding to the first image frame; concatenating the first noisy feature and the first background feature to obtain a second noisy feature; encoding first target information of a first target included in the first image frame to obtain a second conditional encoding result; spatially aligning the second conditional encoding result with the second noisy feature to obtain a first position embedding feature; and using the first background feature and the first position embedding feature as the first conditional encoding result. The first training module uses an initial world model to perform denoising calculations based on the first image features and the first conditional encoding result to obtain a first generated image, which may include: using the initial world model to perform denoising calculations based on the second noisy feature and the second conditional encoding result to obtain the first generated image.

[0100] In an embodiment of the present invention, the first training module encodes first target information of a first target included in a first image frame to obtain a second conditional encoding result, which may include: encoding position information in the first target information to obtain a first position encoding result; encoding category information in the first target information to obtain a first category encoding result; and using the first position encoding result and the first category encoding result as the second conditional encoding result. The first training module spatially aligns the second conditional encoding result with the second noisy feature to obtain a first position embedded feature, which may include: feature concatenating the first position encoding result and the first category encoding result, and aggregating the concatenated features using a linear layer to obtain the second conditional encoding result.

[0101] In an embodiment of the present invention, the first training module calculates the first loss value of the first generated image compared to the first image frame, which may include: calculating the loss value of the first predicted feature obtained by denoising calculation compared to the first image feature to obtain the first loss value.

[0102] In an embodiment of the present invention, the first training module uses the initial world model to perform denoising calculations based on the first image features and the first conditional encoding results to obtain a first generated image, which may include: encoding the first text description information corresponding to the first image frame to obtain a first prompt embedding feature; using the initial world model to perform denoising calculations based on the first image features, the first conditional encoding results and the first prompt embedding features to obtain the first generated image.

[0103] In an embodiment of the present invention, the second training module uses the driving video sample to train the first-world model to generate a future driving scene sequence based on the input image sequence and the structured traffic conditions and driving actions at the corresponding time to obtain the second-world model, which may include: encoding the structured traffic conditions at the corresponding time of the first second image frame in the second image sequence in the driving video sample to obtain a third condition encoding result; encoding the first driving action sequence at the corresponding time of the second image sequence to obtain an action encoding result; generating future structured traffic conditions based on the third condition encoding result and the action encoding result; using the first-world model to generate a future driving scene sequence based on the future structured traffic conditions; calculating a second loss value of the future driving scene sequence compared to the third image sequence at the corresponding time in the driving video sample, and using the second loss value to update the first-world model until the second iteration end condition is met to obtain the second-world model.

[0104] In an embodiment of the present invention, the second training module generates future structured traffic conditions based on the third condition encoding result and the action encoding result, which may include: taking the third condition encoding result as the initial hidden state variable, performing cross-attention calculation based on the hidden state variable and the action encoding result at the corresponding moment and converting it into a first latent variable of a Gaussian distribution, performing a gated loop calculation based on the first latent variable and the hidden state variable at the corresponding moment to obtain the hidden state variable at the next moment, until the hidden state variable at the next moment of the last second image frame of the second image sequence is obtained; connecting the hidden state variable with the action encoding result at the previous moment and performing decoding calculation to obtain the future structured traffic conditions.

[0105] In an embodiment of the present invention, the second training module uses the first-world model to generate a future driving scene sequence based on future structured traffic conditions, which may include: using the first-world model to output prediction features based on future structured traffic conditions; decoding the prediction features into the original image space corresponding to the driving video sample to obtain a future driving scene video including multiple future image frames; connecting the prediction features with the action encoding results corresponding to the second image sequence and then performing decoding calculations to obtain a future driving action sequence at future moments of the second image sequence; and using the future driving scene video and the future driving action sequence as the future driving scene sequence.

[0106] In an embodiment of the present invention, the second training module calculates the second loss value of the future driving scene sequence compared to the third image sequence at the corresponding moment in the driving video sample, which may include: calculating the third loss value of the future driving scene video compared to the third image sequence at the corresponding moment in the driving video sample; calculating the fourth loss value of the future driving action sequence compared to the second driving action sequence at the corresponding moment in the driving video sample; and calculating the second loss value based on the third loss value and the fourth loss value.

[0107] In an embodiment of the present invention, the second training module uses the first-world model to generate a future driving scene sequence based on future structured traffic conditions, which may include: encoding a reference background image corresponding to a driving video sample to obtain a reference style embedding feature; and using the first-world model to generate a future driving scene sequence based on the reference style embedding feature and the future structured traffic conditions.

[0108] For the description of the features in the embodiment corresponding to the world model-based decision model optimization device, please refer to the relevant description of the embodiment corresponding to the world model-based decision model optimization method, which will not be repeated here.

[0109] An embodiment of the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned world model-based decision model optimization method embodiments.

[0110] An embodiment of the present invention further provides a non-volatile storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above-mentioned world model-based decision model optimization method embodiments when running.

[0111] In an exemplary embodiment, the non-volatile storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0112] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of any of the above-mentioned world model-based decision model optimization method embodiments.

[0113] An embodiment of the present invention also provides another computer program product, including a non-volatile storage medium, the non-volatile storage medium storing a computer program, and when the computer program is executed by a processor, implementing the steps of any of the above-mentioned world model-based decision model optimization method embodiments.

[0114] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0115] The above is a detailed introduction to the world model-based decision model optimization method, device, medium and product provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only intended to help understand the method of the present invention and its core ideas. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in several ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A decision model optimization method based on a world model, characterized in that: include: Obtain driving video samples; Using the driving video sample to train the initial world model, generating image frames at corresponding moments according to input structured traffic conditions, thereby obtaining a first world model; The first world model is trained using the driving video samples to generate a future driving scene sequence based on an input image sequence and the structured traffic conditions and driving actions at corresponding moments, thereby obtaining a second world model; The decision model is iteratively trained using the second world model. During the training, the driving actions output by the decision model and their corresponding structured traffic conditions are input into the second world model. A reward value for the driving action is calculated based on the obtained sequence of future driving scenarios, and the decision model is updated based on the reward value.

2. The decision model optimization method based on the world model according to claim 1, characterized in that: The driving video sample is used to train the initial world model to generate image frames at corresponding moments according to the input structured traffic conditions, thereby obtaining a first world model, including: extracting a first image feature from a first image frame in the driving video sample; Encoding the structured traffic condition corresponding to the first image frame to obtain a first condition encoding result; generating a first generated image using the initial world model according to the first image feature and the first conditional encoding result; Calculate a first loss value of the first generated image compared to the first image frame, and use the first loss value to update the initial world model until a first iteration end condition is met to obtain the first world model.

3. The decision model optimization method based on the world model according to claim 2, characterized in that: The generating unit of the initial world model is a diffusion model; Generating a first generated image using the initial world model according to the first image feature and the first conditional encoding result, comprising: adding noise to the first image feature to obtain a first noisy feature; The initial world model is used to perform denoising calculation according to the first image feature and the first conditional encoding result to obtain the first generated image.

4. The decision model optimization method based on the world model according to claim 3, characterized in that: Encoding the structured traffic condition corresponding to the first image frame to obtain a first condition encoding result includes: extracting a first background feature from a first road topology map corresponding to the first image frame; concatenating the first noisy feature and the first background feature to obtain a second noisy feature; encoding first target information of a first target included in the first image frame to obtain a second conditional encoding result; Performing spatial alignment processing on the second conditional encoding result and the second noisy feature to obtain a first position embedded feature; using the first background feature and the first position embedding feature as the first conditional encoding result; Performing a denoising calculation based on the first image feature and the first conditional encoding result using the initial world model to obtain the first generated image includes: The initial world model is used to perform denoising calculation according to the second noisy feature and the second conditional encoding result to obtain the first generated image.

5. The decision model optimization method based on the world model according to claim 4, characterized in that: Encoding first target information of a first target included in the first image frame to obtain a second conditional encoding result includes: Encoding the position information in the first target information to obtain a first position encoding result; Encoding the category information in the first target information to obtain a first category encoding result; Taking the first position encoding result and the first category encoding result as the second conditional encoding result; Performing spatial alignment processing on the second conditional encoding result and the second noisy feature to obtain a first position embedded feature, including: Perform feature splicing on the first position encoding result and the first category encoding result, aggregate the spliced ​​features using a linear layer, and obtain the second conditional encoding result.

6. The decision model optimization method based on the world model according to claim 3, characterized in that: Calculating a first loss value of the first generated image compared to the first image frame includes: Calculate the loss value of the first predicted feature obtained by the denoising calculation compared with the first image feature to obtain the first loss value.

7. The decision model optimization method based on the world model according to claim 3, characterized in that: Performing a denoising calculation based on the first image feature and the first conditional encoding result using the initial world model to obtain the first generated image includes: Encoding first text description information corresponding to the first image frame to obtain a first prompt embedding feature; The initial world model is used to perform denoising calculation according to the first image feature, the first conditional encoding result, and the first prompt embedding feature to obtain the first generated image.

8. The decision model optimization method based on the world model according to claim 2, characterized in that: The first world model is trained using the driving video samples to generate a future driving scene sequence based on the input image sequence and the structured traffic conditions and driving actions at the corresponding moments, thereby obtaining a second world model, including: encoding the structured traffic condition at a time corresponding to the first second image frame in the second image sequence in the driving video sample to obtain a third condition encoding result; encoding a first driving action sequence at a moment corresponding to the second image sequence to obtain an action encoding result; generating a future structured traffic condition according to the third condition encoding result and the action encoding result; generating the future driving scene sequence according to the future structured traffic conditions using the first world model; Calculate a second loss value of the future driving scene sequence compared to a third image sequence at a corresponding moment in the driving video sample, and use the second loss value to update the first world model until a second iteration end condition is met to obtain the second world model.

9. The decision model optimization method based on the world model according to claim 8, characterized in that: Generating a future structured traffic condition according to the third condition encoding result and the action encoding result includes: Using the third conditional encoding result as an initial hidden state variable, performing a cross-attention calculation based on the hidden state variable and the action encoding result at the corresponding moment and converting the result into a first latent variable of a Gaussian distribution, performing a gated loop calculation based on the first latent variable and the hidden state variable at the corresponding moment to obtain the hidden state variable at the next moment, until the hidden state variable at the next moment of the last second image frame of the second image sequence is obtained; The hidden state variable is connected with the action encoding result at the previous moment and then decoded and calculated to obtain the future structured traffic condition.

10. The decision model optimization method based on the world model according to claim 9, characterized in that: Generating the future driving scene sequence according to the future structured traffic conditions using the first world model includes: outputting prediction features based on the future structured traffic conditions using the first world model; Decoding the predicted features into the original image space corresponding to the driving video sample to obtain a future driving scene video including multiple future image frames; Connecting the predicted feature with the action coding result corresponding to the second image sequence and performing decoding calculation to obtain a future driving action sequence at a future moment of the second image sequence; The future driving scene video and the future driving action sequence are used as the future driving scene sequence.

11. The decision model optimization method based on the world model according to claim 10, characterized in that: Calculating a second loss value of the future driving scene sequence compared to a third image sequence at a corresponding moment in the driving video sample includes: calculating a third loss value of the future driving scene video compared to a third image sequence at a corresponding moment in the driving video sample; calculating a fourth loss value of the future driving action sequence compared to the second driving action sequence at a corresponding moment in the driving video sample; The second loss value is calculated based on the third loss value and the fourth loss value.

12. The decision model optimization method based on the world model according to claim 8, characterized in that: Generating the future driving scene sequence according to the future structured traffic conditions using the first world model includes: encoding a reference background image corresponding to the driving video sample to obtain a reference style embedding feature; The first world model is used to generate the future driving scene sequence according to the reference style embedding features and the future structured traffic conditions.

13. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the world model-based decision model optimization method according to any one of claims 1 to 12 when executing the computer program.

14. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the decision model optimization method based on the world model are implemented as described in any one of claims 1 to 12.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the decision model optimization method based on the world model are implemented.

Images