Training method and device for trajectory prediction model of autonomous vehicle

Abstract

The embodiment of the invention provides a training method and device for a trajectory prediction model of an automatic driving vehicle. The method comprises the following steps: acquiring a historical driving data set of a target vehicle; based on the historical driving data set, constructing a target unlabeled data set, a target labeled data set and a preference optimization data set; based on the target unlabeled data, pre-training the initial visual motion model to obtain a pre-trained model; training the pre-training model based on the target labeled data set to obtain a training model; and based on the preference optimization data set, performing direct preference optimization training on the training model to obtain a trajectory prediction model. The technical problem that the accuracy of automatic driving track prediction is low is solved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving, and more specifically, to a training method and apparatus for a trajectory prediction model of an autonomous vehicle. Background Technology

[0002] In the field of intelligent driving technology, trajectory prediction models for autonomous vehicles have become a crucial technology for ensuring safe driving. Traditional model training methods often result in limited dataset size, poor model generalization ability, and difficulty in making reasonable predictions. Furthermore, current models are prone to generating low-quality outputs during prediction, which severely impact the real-time control performance and driving safety of autonomous driving systems. Therefore, the traditional training methods for trajectory prediction models of autonomous vehicles lead to the technical problem of low accuracy in autonomous vehicle trajectory prediction.

[0003] There is currently no good solution to the above problems. Summary of the Invention

[0004] This application provides a training method for a trajectory prediction model of an autonomous vehicle, so as to at least solve the technical problem of low accuracy in autonomous vehicle trajectory prediction.

[0005] According to one aspect of the embodiments of this application, a training method for a trajectory prediction model of an autonomous vehicle is provided, comprising: acquiring a historical driving dataset of a target vehicle, wherein the historical driving dataset includes at least image data and driving trajectory data collected by the target vehicle during driving; constructing a target unlabeled dataset, a target labeled dataset, and a preference optimization dataset based on the historical driving dataset, wherein the target unlabeled dataset contains image data, the target labeled dataset contains image data and driving trajectory data, the data volume corresponding to the target labeled dataset is smaller than the data volume corresponding to the target unlabeled dataset, and the preference optimization dataset contains preference feedback information, which is used to represent the user's preference information for image data and driving trajectory data; pre-training an initial visual action model based on the target unlabeled data to obtain a pre-trained model; training the pre-trained model based on the target labeled dataset to obtain a trained model; and performing direct preference optimization training on the trained model based on the preference optimization dataset to obtain a trajectory prediction model, wherein the trajectory prediction model is used to predict the driving trajectory of the autonomous vehicle based on the input image data during the model inference stage.

[0006] Furthermore, based on the historical driving dataset, a target unlabeled dataset is constructed, including: acquiring a first image captured by the front camera of the target vehicle at a preset time in the historical driving dataset, and multiple second images captured within a first preset time window after the preset time; encoding the multiple second images based on a modulation vector quantization encoder to obtain multiple visual codes, wherein the modulation vector quantization encoder is used to capture the spatiotemporal dynamic relationship in the image by quantizing visual features; and constructing a target unlabeled dataset based on the first image and multiple visual codes.

[0007] Furthermore, based on the target unlabeled dataset, the initial visual action model is pre-trained to obtain a pre-trained model, including: based on the preset likelihood loss function and the target unlabeled dataset, the initial visual action model is pre-trained to obtain a pre-trained model, wherein the preset likelihood loss function is used to encourage the initial visual action model to autoregressively predict the visual token sequence of the next frame.

[0008] Furthermore, based on the historical driving dataset, a target labeled dataset is constructed, including: acquiring multiple third images of the target vehicle captured by cameras at various angles at preset times in the historical driving dataset, first driving trajectory data within a second preset time window before the preset time, second driving trajectory data within a second preset time window after the preset time, and multiple visual codes corresponding to the multiple second images; and constructing a target labeled dataset based on the multiple third images, the first driving trajectory data, the second driving trajectory data, and the multiple visual codes.

[0009] Furthermore, based on the target labeled dataset, the pre-trained model is trained to obtain the trained model, including:

[0010] Obtain the first driving command input by the user to the target vehicle within a second preset time window before a preset time in the historical driving dataset; train the pre-trained model based on the preset cross-entropy loss function, the first driving command, and the target labeled dataset to obtain the trained model, wherein the preset cross-entropy loss function is used to guide the alignment of the trained model with the expected trajectory corresponding to the first driving command.

[0011] Furthermore, the preference feedback information includes positive preference feedback information and negative preference feedback information; based on the historical driving dataset, a preference optimization dataset is constructed, including: constructing positive samples based on multiple third images, first driving trajectory data and second driving trajectory data in the historical driving dataset, wherein the positive samples are used to represent positive preference feedback information; constructing negative samples corresponding to the positive samples, wherein the negative samples include at least one of the following: all-zero trajectory, noisy trajectory and repeated trajectory, and the negative samples are used to represent negative preference feedback information; based on the positive samples and negative samples, a preference optimization dataset is constructed.

[0012] Furthermore, based on the preference optimization dataset, the training model is directly trained with preference optimization to obtain the trajectory prediction model, including: based on the preset low-rank adapter and the preference optimization dataset, the training model is directly trained with preference optimization to obtain the trajectory prediction model.

[0013] Furthermore, the method also includes: during the model inference stage, acquiring multiple fourth images captured by cameras at various angles of the autonomous vehicle, as well as a second driving command input by the user to the autonomous vehicle; inputting the multiple fourth images and the second driving command into the trajectory prediction model to obtain a first predicted trajectory; determining the displacement error and collision rate of the autonomous vehicle driving based on the first predicted trajectory; and performing open-loop feedback adjustment on the trajectory prediction model based on the displacement error and collision rate.

[0014] According to another aspect of the embodiments of this application, a training apparatus for a trajectory prediction model of an autonomous vehicle is also provided, comprising: an acquisition module for acquiring a historical driving dataset of a target vehicle, wherein the historical driving dataset includes at least image data collected by the target vehicle during driving and driving trajectory data; a construction module for constructing a target unlabeled dataset, a target labeled dataset, and a preference optimization dataset based on the historical driving dataset, wherein the target unlabeled dataset contains image data, the target labeled dataset contains image data and driving trajectory data, the data volume corresponding to the target labeled dataset is less than the data volume corresponding to the target unlabeled dataset, and the preference optimization dataset contains preference feedback information, which is used to represent the user's preference information for image data and driving trajectory data; a pre-training module for pre-training an initial visual action model based on the target unlabeled dataset to obtain a pre-trained model; a training module for training the pre-trained model based on the target labeled dataset to obtain a trained model; and a preference optimization training module for performing direct preference optimization training on the trained model based on the preference optimization dataset to obtain a trajectory prediction model, wherein the trajectory prediction model is used to predict the driving trajectory of the autonomous vehicle based on the input image data during the model inference stage.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0016] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0017] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0018] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0019] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0020] In this embodiment, firstly, the historical driving dataset of the target vehicle is acquired; then, based on the historical driving dataset, a target unlabeled dataset, a target labeled dataset, and a preference optimization dataset are constructed; next, based on the target unlabeled data, an initial visual action model is pre-trained to obtain a pre-trained model; then, based on the target labeled dataset, the pre-trained model is trained to obtain a trained model; finally, based on the preference optimization dataset, the trained model undergoes direct preference optimization training to obtain a trajectory prediction model. This application employs a three-stage training process. First, the visual action model is pre-trained on a large amount of unlabeled historical driving image data to enhance its ability to extract and understand visual information, resulting in a pre-trained model. Next, fine-tuning is performed using the labeled dataset to closely correlate the predicted behavior of the pre-trained model with the actual driving trajectory, improving the pre-trained model's adaptability and predictive ability for specific scenarios, ensuring that the model's output trajectory better matches actual driving needs. Finally, direct preference optimization training is employed to further optimize the model based on user feedback on safe driving preferences, reducing low-quality outputs such as zero coordinates, noisy trajectories, or repetitive trajectories, ensuring the quality and safety of the predicted trajectory, thereby improving the stability and reliability of autonomous driving decisions. Therefore, this application effectively solves the problems of insufficient model generalization and unstable output quality in the prior art, thereby enabling the trajectory prediction model to generate more accurate and safer driving trajectories in complex driving environments, and thus solving the technical problem of low accuracy in autonomous driving trajectory prediction. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 This is a schematic diagram of a training method for a trajectory prediction model of an autonomous vehicle based on existing technology;

[0023] Figure 2 This is a schematic diagram of the structure of an optional training method for a trajectory prediction model of an autonomous vehicle according to an embodiment of this application;

[0024] Figure 3 This is an optional end-to-end VLA model future trajectory prediction flowchart based on DPO-based spatiotemporal CoT inference according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the DPO training set construction for an optional training method of a trajectory prediction model for an autonomous vehicle according to an embodiment of this application.

[0026] Figure 5 This is a schematic diagram illustrating the application of an optional LoRA adapter in the DPO training phase according to an embodiment of this application.

[0027] Figure 6 This is an optional spatiotemporal CoT inference process logic diagram according to an embodiment of this application;

[0028] Figure 7 This is a schematic diagram of an optional autonomous driving device structure according to an embodiment of this application;

[0029] Figure 8 This is a schematic diagram of a training device for an optional trajectory prediction model of an autonomous vehicle according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] According to an embodiment of this application, a training method for a trajectory prediction model of an autonomous vehicle is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] This embodiment provides a method for training a trajectory prediction model for autonomous vehicles. Figure 1 This is a flowchart of a training method for a trajectory prediction model of an autonomous vehicle according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps:

[0034] Step S102: Obtain the historical driving dataset of the target vehicle.

[0035] The historical driving dataset includes at least image data and driving trajectory data collected during the driving process of the target vehicle.

[0036] The target vehicle mentioned above can refer to a specific vehicle of interest in this autonomous driving system training or testing. It is the object of data collection and analysis and can be any type of non-autonomous vehicle or any type of autonomous vehicle.

[0037] The aforementioned historical driving dataset refers to a collection of data records generated from the target vehicle's past driving experiences. Historical driving datasets contain a wealth of information about the vehicle's driving environment and its own dynamics. They provide training samples, allowing the system to learn from past experience and thus better predict future driving conditions.

[0038] The aforementioned image data can refer to data collected by the front-facing camera installed on the target vehicle and other cameras surrounding the vehicle. These cameras continuously capture dynamic changes in the surrounding environment, including but not limited to road traffic signs, obstacles, other road users, and weather and lighting conditions. Through this image data, embodiments of this application can provide the model with multi-angle and multi-directional visual environmental perception capabilities.

[0039] The aforementioned driving trajectory data can refer to information such as the position, speed, and direction of the target vehicle at different points in time, specifically represented by a series of coordinate points that accurately depict the actual path the vehicle travels.

[0040] As an optional embodiment, image data is captured using multiple cameras (including but not limited to a front-facing camera, front left, front right, rear left, rear right, and rear-facing cameras). These cameras continuously record the visual scene in front, to the sides, and behind the target vehicle during its movement, ensuring that the dataset covers multi-directional driving environment information. Furthermore, various sensors equipped on the target vehicle, such as GPS, radar, lidar, and inertial measurement units (IMUs), capture the vehicle's position, speed, direction, acceleration, and obstacle information in the environment. These sensors allow for precise tracking and recording of the target vehicle's dynamic trajectory during its movement, providing accurate driving path examples for model training.

[0041] An inertial measurement unit (IMU) is an electronic device commonly used for acquiring dynamic attitude and motion data. An IMU primarily contains two types of sensors: an accelerometer and a gyroscope, and sometimes also integrates a magnetometer.

[0042] An accelerometer is used to measure the acceleration of an object along three axes, namely linear acceleration along the X, Y, and Z axes. A gyroscope is used to measure the angular velocity about these three axes, i.e., rotational speed. A magnetometer is used to measure the strength of a magnetic field, helping to determine the orientation of a device.

[0043] In the above process, obtaining the historical driving dataset of the target vehicle provides rich, diverse, and realistic data support for the autonomous driving model, enabling the model to learn and understand the driving environment from both visual and motion control perspectives during training, thereby enhancing the model's scene understanding and decision-making capabilities.

[0044] Step S104: Based on the historical driving dataset, construct the target unlabeled dataset, the target labeled dataset, and the preference optimization dataset.

[0045] The target unlabeled dataset contains image data, the target labeled dataset contains both image data and driving trajectory data, the amount of data in the target labeled dataset is less than the amount of data in the target unlabeled dataset, and the preference optimization dataset contains preference feedback information, which is used to represent the user's preference information for image data and driving trajectory data.

[0046] The aforementioned target unlabeled dataset can refer to image data from historical driving datasets that does not contain any manually labeled information. By filtering out all labels associated with the actual driving paths, a dataset containing only visual information was constructed, providing a foundation for the model's pre-training phase.

[0047] The aforementioned labeled target dataset can refer to samples in a historical driving dataset that include both image data and driving trajectory data, and the data volume of the labeled target dataset is smaller than that of the unlabeled target dataset. The combination of driving trajectory data and image data provides the model with a direct supervisory signal, enabling it to learn how to predict reasonable future driving directions and positions from given visual input.

[0048] The aforementioned preference optimization dataset can refer to a dataset containing preference feedback information. This feedback information reflects users' preferences for image data and driving trajectory data, that is, users' expectations and evaluations of vehicle driving behavior in specific driving situations. Preference feedback information typically includes positive and negative samples. Positive samples represent driving behaviors that users consider reasonable and safe, while negative samples represent driving behaviors that users consider inappropriate or potentially dangerous.

[0049] The aforementioned preference feedback information can refer to the degree of preference of the user or autonomous driving system for the vehicle's driving trajectory and its corresponding guidance signals. In the embodiments of this application, preference feedback information is divided into two categories: positive preference feedback information and negative preference feedback information. Positive preference feedback information is usually associated with high-quality, safe, or efficient driving trajectories, indicating that the model should prioritize imitating or generating such trajectories; conversely, negative preference feedback information is associated with low-quality, dangerous, or ineffective driving behaviors, used to guide the model to identify and avoid such trajectories.

[0050] By constructing a preference optimization dataset, this embodiment of the application can utilize Direct Preference Optimization (DPO) technology to perform final training on the model, ensuring that the trajectory points generated by the model are not only technically feasible but also safe, comfortable, and as expected from the driver's perspective. The construction and training of the preference optimization dataset enables the model to learn behavioral patterns that conform to driver preferences, thereby exhibiting higher intelligence and adaptability in autonomous driving.

[0051] In one optional embodiment, image data is first selected to construct a target unlabeled dataset, which enhances the model's visual feature extraction capabilities. Then, samples containing image data and corresponding driving trajectory data are selected from historical data to form a target labeled dataset. Although this dataset is small in quantity, it provides direct supervision for model fine-tuning, ensuring the accuracy of trajectory generation. Finally, through user feedback or expert evaluation, preference information for specific images and driving trajectories is collected to construct a preference optimization dataset for model preference alignment, improving the safety and humanization of decision-making.

[0052] For example, a DPO training set is constructed, using real trajectory information as positive sample data, and negative sample data (such as all-zero trajectories, noisy trajectories, and repeated trajectories) is generated from this data.

[0053] This application provides customized data support for different stages of model training, thereby improving model performance. Unlabeled target datasets, through self-supervised learning, help the model establish a foundation for visual understanding; labeled target datasets, through supervised learning, further enhance the model's decision-making ability; and preference optimization datasets ensure that the model's output behavior meets the safety and comfort standards of human drivers, thus improving the overall safety and user experience of the autonomous driving system. The construction and utilization of this series of datasets effectively solves the problems of data type and data quality in autonomous driving model training, providing a data foundation for achieving safe, efficient, and human-centered autonomous driving technology.

[0054] Step S106: Based on the unlabeled target data, the initial visual action model is pre-trained to obtain a pre-trained model.

[0055] The initial visual action model mentioned above refers to a model that has not been trained or optimized before the pre-training process begins. This model typically has a suitable architectural design, such as including a multi-layered Transformer structure, for processing visual input and generating corresponding action outputs. The model's parameters are randomly initialized at this stage and have not yet been tuned or optimized by learning patterns from the dataset. The initial visual action model is designed to integrate visual perception and action decision-making, providing autonomous vehicles with the ability to directly map environmental images to driving trajectories.

[0056] The aforementioned pre-trained model refers to a model obtained by training on an unlabeled target dataset based on an initial visual action model. During the pre-training phase, the model can learn how to predict future image changes or reconstruct the input image from a given image through self-supervised learning. This process prompts the model parameters to be adjusted to better capture visual features and understand the dynamic changes in the driving environment.

[0057] In one alternative embodiment, target-unlabeled data is first extracted from historical driving records. This data consists of continuous images from the vehicle's front-facing camera, without any specific driving trajectory labels. Next, this image data is processed using an initial visual-action model. Through self-supervised learning mechanisms, such as contrastive prediction or image reconstruction tasks, the model learns the temporal variation patterns and spatial environmental characteristics from the image sequence.

[0058] During this process, the initial visual action model learns and optimizes itself through input of a large number of unlabeled images, gradually improving its ability to capture and understand visual information. The goal of model learning is to maximize predictive ability on unlabeled data, such as by predicting the content of subsequent image frames or reconstructing the input image, thereby enhancing its visual cognition of driving scenarios without direct trajectory guidance.

[0059] For example, given all unlabeled frame image data, at a given time t, firstly, the front-facing camera image of the vehicle at the next 0.5 seconds is visually encoded using a modulation vector quantization encoder. Then, the front-facing camera image at the given time t and the visual encoding of the front-facing camera image at the next 0.5 seconds are used to construct a pre-training dataset. Next, the initial visual action model is trained on this pre-training dataset to obtain a pre-trained model, and the model's parameters are saved.

[0060] The aforementioned initial visual-motor model is specifically designed for tasks in the field of autonomous driving that combine visual input with verbal commands, such as predicting the future trajectory of a vehicle from images and text instructions. Through training with a large amount of data, this model can learn rich visual and verbal representations, as well as the correlation between the two, thereby providing a more comprehensive and accurate information foundation for autonomous driving decisions.

[0061] This application utilizes a target unlabeled dataset for pre-training, significantly enhancing the environmental perception and visual understanding capabilities of the visual action model, especially in driving scenarios without human annotation. The pre-trained model can better adapt to changes in visual input and understand visual patterns in complex driving environments, providing a foundation for visual feature extraction for subsequent model fine-tuning and preference optimization. This lays a crucial foundation for realizing an autonomous driving system capable of understanding human driving preferences and generating safe and reliable driving trajectories.

[0062] Step S108: Train the pre-trained model based on the target labeled dataset to obtain the trained model.

[0063] The aforementioned trained model refers to a model obtained through a fine-tuning process, which is the result of a pre-trained model trained on a labeled dataset. This model not only possesses the ability to extract visual features learned from unlabeled datasets but can also predict future vehicle trajectories directly from image input through supervised learning. This enables the trained model to generate driving paths that conform to safety standards and driving habits in more specific and instructive scenarios, such as the actual driving environment of autonomous vehicles.

[0064] In one alternative embodiment, during the fine-tuning phase, images and trajectory labels from the target labeled dataset are used to guide the trained model in learning how to generate accurate future trajectory points from visual input. Typically, a supervised learning framework is employed, optimizing model parameters by minimizing the error between the predicted and actual trajectories (e.g., using cross-entropy loss), gradually transforming the pre-trained model into a trained model capable of accurately predicting future driving trajectories.

[0065] For example, given labeled keyframe image data, at a given time t, firstly, the front-facing camera image of the vehicle at the next 0.5 seconds is visually encoded using a modulation vector quantization encoder. Then, the visual encodings of the vehicle's 360-degree omnidirectional images at the given time t, the vehicle's historical trajectory over the past 3 seconds, and the front-facing camera image at the next 0.5 seconds, along with the vehicle's trajectory points over the next 3 seconds, are used to construct a fine-tuning training set and a corresponding validation set. Next, the pre-trained model is further trained on this fine-tuning training set to obtain the fine-tuned model and save its parameters.

[0066] The fine-tuning steps in this embodiment effectively enhance the model's ability to predict driving trajectories. By training with a target-labeled dataset, the pre-trained model learns a clear mapping from image input to trajectory output, enabling it to generate driving trajectories that closely match actual driving behavior. This process significantly improves the accuracy and safety of the trained model, providing strong support for the application of autonomous driving systems in real-world road environments. The trained model is thus able to make more accurate decisions in complex scenarios, reducing potential risks caused by prediction errors and achieving a safer autonomous driving experience.

[0067] Step S110: Based on the preference optimization dataset, perform direct preference optimization training on the training model to obtain the trajectory prediction model.

[0068] The trajectory prediction model is used to predict the driving trajectory of autonomous vehicles based on the input image data during the model inference stage.

[0069] The trajectory prediction model mentioned above refers to the DPO model trained using Direct Preference Optimization (DPO). Based on input image data, this model can predict the driving trajectory of autonomous vehicles. Its core function lies in combining visual understanding and human driving preferences to provide accurate and safe predictions of future trajectory points. Through DPO training, the DPO model not only retains its deep understanding of visual scenes but also generates prediction results that match the driving behavior of human drivers in similar scenarios, significantly improving the accuracy and reliability of predictions. This provides a more reliable and safer decision-making basis for autonomous driving decision-making systems.

[0070] In one alternative embodiment, a high-level training process is performed on the trained model based on a preference optimization dataset. Unlike traditional fine-tuning methods, Direct Preference Optimization (DPO) provides positive and negative feedback by comparing different trajectory samples generated by the model, prompting the model to learn and internalize human driving preferences. During training, using the DPO algorithm, the model receives signals about whether the generated trajectory is preferred, thereby optimizing its trajectory prediction decision-making mechanism while retaining its original visual understanding capabilities.

[0071] In another optional embodiment, the process of directly optimizing the training model in this application is as follows: First, a preference optimization dataset is created, which contains a series of preference pairs, each consisting of two samples: a positive sample considered to be favored by human drivers, and a negative sample not favored. To construct these preference pairs, trajectory samples with different driving behaviors or outcomes are selected from the dataset of the fine-tuning training phase. For example, a trajectory that follows traffic rules and avoids obstacles is used as a positive sample, while a trajectory that deviates from the lane or reacts slowly is used as a negative sample. This ensures that the dataset covers both normal driving behavior and potential inappropriate behavior.

[0072] Next, starting with the training model obtained in the fine-tuning stage, the parameters are adjusted using a pre-set low-rank adapter technique (LoRA) to reduce the consumption of computing resources and speed up the training process.

[0073] The LoRA technique described above achieves a balance between improving model capabilities and computational efficiency by performing a low-rank decomposition on the weight matrix of the pre-trained model and optimizing only the smaller matrices resulting from the decomposition. During DPO training, LoRA ensures that the model can quickly adapt to preferred optimization objectives without affecting its previous capabilities in visual encoding and trajectory prediction.

[0074] Then, the model is trained using the DPO training set with the goal of minimizing the preference loss. The preference loss is measured by comparing the model's predictive power for positive and negative samples, encouraging the model to produce results that more closely resemble human driver preferences.

[0075] For example, the loss function of DPO is designed to incentivize the model to prioritize positive sample trajectories in predictions, while avoiding or correcting prediction behaviors associated with negative sample trajectories, thereby driving the model toward higher-level decision-making.

[0076] Finally, after Direct Preference Optimization (DPO) training, a trajectory prediction model is obtained. This model not only possesses the visual understanding and trajectory prediction capabilities of the trained model, but also further enhances its performance in making decisions based on human driving preferences through DPO training. This means that when the DPO model is applied in autonomous vehicles, it can generate more reasonable and safer driving trajectories, reduce the probability of unintended behaviors, and improve the feasibility and acceptability of autonomous driving systems in real-world environments.

[0077] For example, using the LoRA technique, the fine-tuned model is trained one last time on the DPO training set to obtain the DPO model and save the model's parameters.

[0078] By employing a direct preference optimization training step based on a preference optimization dataset, this embodiment of the application optimizes the model. The resulting trajectory prediction model can more accurately predict the driving trajectory of autonomous vehicles, especially when dealing with complex traffic scenarios and rare driving situations, where its prediction results more closely match the driving decisions of human drivers. The DPO model significantly improves the driving safety and quality of autonomous vehicles on real roads.

[0079] In this embodiment, firstly, the historical driving dataset of the target vehicle is acquired; then, based on the historical driving dataset, a target unlabeled dataset, a target labeled dataset, and a preference optimization dataset are constructed; next, based on the target unlabeled data, an initial visual action model is pre-trained to obtain a pre-trained model; then, based on the target labeled dataset, the pre-trained model is trained to obtain a trained model; finally, based on the preference optimization dataset, the trained model undergoes direct preference optimization training to obtain a trajectory prediction model. This application employs a three-stage training process. First, the visual action model is pre-trained on a large amount of unlabeled historical driving image data to enhance its ability to extract and understand visual information, resulting in a pre-trained model. Next, fine-tuning is performed using the labeled dataset to closely correlate the predicted behavior of the pre-trained model with the actual driving trajectory, improving the pre-trained model's adaptability and predictive ability for specific scenarios, ensuring that the model's output trajectory better matches actual driving needs. Finally, direct preference optimization training is employed to further optimize the model based on user feedback on safe driving preferences, reducing low-quality outputs such as zero coordinates, noisy trajectories, or repetitive trajectories, ensuring the quality and safety of the predicted trajectory, thereby improving the stability and reliability of autonomous driving decisions. Therefore, this application effectively solves the problems of insufficient model generalization and unstable output quality in the prior art, thereby enabling the trajectory prediction model to generate more accurate and safer driving trajectories in complex driving environments, and thus solving the technical problem of low accuracy in autonomous driving trajectory prediction.

[0080] Optionally, based on the historical driving dataset, a target unlabeled dataset is constructed, including: acquiring a first image captured by the front camera of the target vehicle at a preset time in the historical driving dataset, and multiple second images captured within a first preset time window after the preset time; encoding the multiple second images based on a modulation vector quantization encoder to obtain multiple visual codes, wherein the modulation vector quantization encoder is used to capture the spatiotemporal dynamic relationship in the image by quantizing visual features; and constructing the target unlabeled dataset based on the first image and multiple visual codes.

[0081] The aforementioned front-facing camera can refer to a camera device installed at the front of an autonomous vehicle to capture images of the vehicle's forward field of view. This camera is responsible for acquiring a continuous video stream, and the image data acquired by the camera forms the basis for constructing the target unlabeled dataset, which is used for model learning and prediction.

[0082] The aforementioned preset time point can refer to a specific point in time selected by the system or researchers during the autonomous vehicle's operation, serving as a time reference for image acquisition and subsequent data processing. The aforementioned first image can refer to an image captured by the target vehicle's front-facing camera at a preset time selected from historical driving datasets.

[0083] The aforementioned first preset time window can refer to a period of time immediately following a preset moment, such as 0.5 seconds, used to collect subsequent image data. In this embodiment, the images collected within this time window will be used to construct positive and negative samples in the preference pair, and converted into visual encoding through a modulation vector quantization encoder to provide data for preference optimization training, ensuring that the model can learn the expected behavior and potential error patterns of the autonomous vehicle within this time window.

[0084] The aforementioned second image may refer to subsequent image frames continuously acquired within a first preset time window (e.g., 0.5 seconds) after a preset time, and these frames are defined as multiple second images.

[0085] The aforementioned modulation vector quantization encoder is used to capture spatiotemporal dynamic relationships in images by quantizing visual features. The encoder is responsible for converting multiple second images captured from a front-facing camera into compact visual codes rich in spatiotemporal dynamic information. First, feature extraction is performed on the images, converting pixel-level visual information into high-level visual features. Next, through quantization, the continuous visual feature space is mapped onto a set of discrete vectors. These discrete vectors can more efficiently represent image content while preserving key spatiotemporal relationships. Finally, each visual code output by the encoder corresponds to a specific time point and spatial location in the second image, providing a spatiotemporal dynamic visual representation for subsequent model training. The modulation vector quantization encoder not only compresses visual information but also enhances the model's ability to capture subtle dynamic changes in image sequences through the quantization process, effectively solving the problems of poor generalization and insufficient ability to handle complex scenes caused by a lack of deep understanding of spatiotemporal relationships in existing technologies.

[0086] The aforementioned visual encoding refers to a quantized representation obtained by processing image data using a modulation vector quantization encoder, which can efficiently store and convey image content and scene dynamics. This encoding not only includes the static features of the image, but more importantly, it expresses the spatiotemporal relationships and motion trends of objects in the image sequence, providing key visual information needed for DPO model training to understand scene dynamics and predict future trajectories.

[0087] In one optional embodiment, a first image captured by the front-facing camera of the target vehicle at a preset time is first acquired, followed by the acquisition of multiple subsequent second images within a first preset time window. The second images are encoded using a modulation vector quantization encoder to capture spatiotemporal dynamics and generate multiple visual codes. The first images and the visual codes are combined to construct a target unlabeled dataset for pre-training. This step enhances the model's spatiotemporal understanding of visual information, solves the problems of non-compliant model output formats and unstable trajectory prediction in existing technologies, and improves the safety and accuracy of autonomous driving decisions.

[0088] The embodiments of this application utilize a large amount of unlabeled driving data, enhancing the model's ability to understand the spatiotemporal dynamics of visual information. This construction strategy provides the model with rich learning data, enabling it to master the visual features and dynamic change patterns in complex driving scenarios during the pre-training stage, thereby significantly improving the model's generalization ability and safety in autonomous driving tasks.

[0089] Optionally, the initial visual action model is pre-trained based on the target unlabeled dataset to obtain a pre-trained model, including: pre-training the initial visual action model based on a preset likelihood loss function and the target unlabeled dataset to obtain a pre-trained model, wherein the preset likelihood loss function is used to encourage the initial visual action model to autoregressively predict the visual token sequence of the next frame.

[0090] The aforementioned pre-set likelihood loss function can be considered a mathematical expression used to quantify the difference between the initial visual action model's prediction and the actual visual sequence. The pre-set likelihood loss function guides the model's learning process by minimizing the negative log-likelihood ratio between the predicted visual token sequence and the true sequence, thus encouraging the model to more accurately predict the content of the next frame, especially focusing on its spatiotemporal dynamics. The pre-set likelihood loss function is central to the pre-training phase; it does not rely on specific trajectory point annotations but focuses on the model's ability to understand and generate visual information, thereby overcoming the limitations of existing models in handling complex visual scenes.

[0091] In one optional embodiment, a pre-training process is performed on an initial visual action model based on a constructed target unlabeled dataset, which includes visual information of the target vehicle at different time points. The core of pre-training is the use of a pre-defined likelihood loss function. This function guides the model to adjust parameters by calculating the difference between the model's predicted next visual token sequence and the actual sequence, thereby improving the accuracy of sequence prediction. Specifically, the model receives a first image of the target vehicle at a certain moment, and subsequent second images at multiple time points. These images are encoded by a modulation vector quantization encoder and used as input for training. The goal of pre-training is to enable the model to predict the visual features of future frames from the current image without trajectory annotations. This is achieved by maximizing the pre-defined likelihood loss function. This process allows the model to understand and predict the changing trends of the visual scene even without specific action or trajectory guidance, enhancing the model's generalization ability when facing unseen scenes and providing more robust and flexible decision-making basis for autonomous driving systems in complex environments.

[0092] This application embodiment significantly improves the sensitivity and prediction accuracy of the initial visual action model to spatiotemporal dynamic relationships in image sequences through pre-training on the target unlabeled dataset. The pre-set likelihood loss function effectively guides the model's learning, enabling it to gain a deep understanding of visual information on unlabeled data, thereby reducing reliance on manual annotation and ensuring the model's stability and safety in complex and changing driving environments.

[0093] Optionally, a target labeled dataset is constructed based on the historical driving dataset, including: acquiring multiple third images of the target vehicle captured by cameras at various angles at preset times in the historical driving dataset, first driving trajectory data within a second preset time window before the preset time, second driving trajectory data within a second preset time window after the preset time, and multiple visual codes corresponding to the multiple second images; and constructing a target labeled dataset based on the multiple third images, the first driving trajectory data, the second driving trajectory data, and the multiple visual codes.

[0094] The aforementioned cameras at various angles can refer to multiple cameras installed on the target vehicle to capture comprehensive image information of the vehicle's surrounding environment. These cameras cover different perspectives such as front, rear, left, right, and even top and bottom views, ensuring the multi-dimensionality and completeness of visual information.

[0095] The aforementioned multiple third images may refer to images captured by cameras at various angles of the target vehicle at preset times, including but not limited to images from the front, rear, left, and right perspectives, comprehensively covering the environment around the vehicle and providing multi-angle visual information.

[0096] The aforementioned second preset time window can refer to a time interval before or after a preset time, such as within 3 seconds before or after the preset time. The trajectory data (first driving trajectory data and second driving trajectory data) collected within this time window, combined with the corresponding image information (multiple third images and multiple second images), constitute the main components of the target labeled dataset. By limiting the time window, this embodiment ensures that the information in the dataset is closely related to the preset time, avoiding interference from irrelevant information and improving training efficiency and model performance.

[0097] The aforementioned first driving trajectory data can refer to the target vehicle's driving trajectory recorded within a second preset time window before a preset time, including information such as the vehicle's speed, direction, and acceleration. This provides the model with historical driving behavior, helping to understand the vehicle's dynamic characteristics and predict its future driving state. The aforementioned second driving trajectory data can refer to the target vehicle's driving trajectory recorded within a second preset time window after a preset time.

[0098] The aforementioned multiple visual codes can refer to the transformation of multiple second images by a modulation vector quantization encoder, used to represent the spatiotemporal dynamic relationships in the images. The model learns the changing patterns of the visual scene through these codes, thereby enabling it to more accurately predict visual information and corresponding driving behaviors.

[0099] In one optional embodiment, multiple third images of the target vehicle acquired from cameras at various angles at preset times are collected, along with first and second driving trajectory data within specific time windows before and after the preset times. For the multiple second images of the target vehicle after the preset times, a modulation vector quantization encoder is used for feature encoding to obtain multiple visual codes, ensuring that the encoded information is rich in the spatiotemporal dynamic attributes of the images. The multiple third images, the first driving trajectory data, the second driving trajectory data, and the corresponding multiple visual codes are combined to form a target-labeled dataset, providing comprehensive training material for the model, covering the time series of visual information and driving behavior.

[0100] The embodiments of this application can guide the model to learn the close relationship between visual information and driving trajectory, especially with advantages in processing multi-angle camera images and time-series data. This dataset construction method not only enriches the model's learning content but also enables it to extract effective driving strategies and preferences from past driving experiences, thereby generating driving trajectory predictions that are more consistent with real-world scenarios. During the fine-tuning training phase, the model can learn using a target-labeled dataset, which can significantly improve its performance in autonomous driving tasks, such as increasing prediction accuracy, enhancing the ability to handle long-tail events, and reducing collision risk.

[0101] Optionally, a pre-trained model is trained based on a target labeled dataset to obtain a trained model, including: obtaining the first driving instruction input by the user to the target vehicle within a second preset time window before a preset time in the historical driving dataset; and training the pre-trained model based on a preset cross-entropy loss function, the first driving instruction, and the target labeled dataset to obtain a trained model, wherein the preset cross-entropy loss function is used to guide the alignment of the trained model with the expected trajectory corresponding to the first driving instruction.

[0102] The users mentioned above refer to the drivers or operators of autonomous vehicles; they are individuals who provide input commands to the target vehicle in actual driving or simulated environments. Users can be drivers sitting in the driver's seat, responsible for operating the vehicle on the road, or autonomous driving testers, inputting commands to the vehicle in a simulated environment to evaluate its behavior. In some cases, users can also be remote operators, sending commands to the vehicle via a communication system.

[0103] The aforementioned first driving instruction can refer to a driving action command issued by the user to the target vehicle at a specific point in time, which directly reflects the user's driving intentions and preferences. In the embodiments of this application, the first driving instruction can be a specific operation such as acceleration, deceleration, steering, lane changing, braking, or it can be "turn left ahead" or "keep driving in the lane". These instructions not only include changes in the motion state of the target vehicle, but also include responses and predictions to the surrounding environment.

[0104] The aforementioned preset cross-entropy loss function can refer to a commonly used machine learning loss function, used in the fine-tuning training phase to measure the difference between the model output and the actual labeled data. In this embodiment, the preset cross-entropy loss function is designed to guide the model output to align with the expected trajectory corresponding to the first driving instruction.

[0105] In one optional embodiment, a first driving instruction input by the user to the target vehicle within a second preset time window before a preset time is obtained from a historical driving dataset. Based on a preset cross-entropy loss function, the first driving instruction, and the target labeled dataset, the pre-trained model is further fine-tuned. The preset cross-entropy loss function serves as a guide, aligning the model's predicted trajectory with the driver's expected trajectory. By incorporating real driving instructions and desired trajectories, the model can more accurately understand and simulate optimal path selection in complex driving situations.

[0106] This application's embodiments achieve further optimization of the pre-trained model during the fine-tuning stage. Its core effect lies in the model's ability to learn trajectory prediction capabilities that better align with human driving habits and scenario understanding, based on the spatiotemporal dynamic information from the first driving instruction and the target labeled dataset. A pre-defined cross-entropy loss function serves as a guide, ensuring that the model's output trajectory highly matches the decisions of human drivers in similar situations, thereby significantly improving the accuracy and safety of the predicted trajectory. This technical solution independently brings a significant improvement in model output quality, especially reducing the generation of format violations and low-quality trajectories. This provides a more reliable and safer decision-making basis for autonomous driving systems, reduces potential risks caused by model output deviations, and promotes the robust development of autonomous driving technology.

[0107] Optionally, the preference feedback information includes positive preference feedback information and negative preference feedback information; based on the historical driving dataset, a preference optimization dataset is constructed, including: constructing positive samples based on multiple third images, first driving trajectory data and second driving trajectory data in the historical driving dataset, wherein the positive samples are used to represent positive preference feedback information; constructing negative samples corresponding to the positive samples, wherein the negative samples include at least one of the following: all-zero trajectory, noisy trajectory and repeated trajectory, and the negative samples are used to represent negative preference feedback information; based on the positive samples and negative samples, a preference optimization dataset is constructed.

[0108] The aforementioned positive samples can consist of multiple third images, first driving trajectory data, and second driving trajectory data from the historical driving dataset. Positive samples represent high-quality driving cases related to positive preference feedback information. These positive samples are used to train the model, enabling it to understand and replicate safe and reasonable driving behavior.

[0109] The negative samples mentioned above are the opposite of the positive samples. They intentionally include driving trajectories that do not conform to safe driving standards, such as all-zero trajectories, noisy trajectories, or repetitive trajectories, to represent negative preference feedback information. The construction of negative samples enables the model to clearly distinguish and avoid inefficient or dangerous driving patterns, ensuring that the generated trajectories always tend towards safer and better solutions.

[0110] In one alternative embodiment, firstly, based on a historical driving dataset, multiple third images, first driving trajectory data, and second driving trajectory data that meet quality criteria are selected to construct a positive sample set. This process filters out high-quality driving instances that match positive preference feedback information. Secondly, a negative sample set corresponding to the positive samples is constructed by artificially introducing all-zero trajectories, noisy trajectories, or repeated trajectories into the second driving trajectory data to form examples of negative preference feedback information. Finally, the positive and negative sample sets are integrated to construct a complete preference optimization dataset.

[0111] This application's embodiments introduce the concept of preference feedback information into the training of autonomous driving models. This innovative method significantly enhances the model's ability to identify and generate high-quality and safe driving trajectories. Comparative training with positive and negative samples enables the model not only to learn to imitate excellent driving behaviors from historical data but also to clearly distinguish and exclude trajectories that do not meet safety standards, greatly reducing the risk of collisions and other safety issues caused by inappropriate model output. This technical effect is independent of other training steps, providing a solid foundation for improving the safety and user experience of autonomous driving. The use of preference optimization datasets ensures that the model can consistently output driving trajectories that conform to both human driving preferences and safety requirements in various driving scenarios, promoting the development of autonomous driving technology towards greater intelligence and reliability.

[0112] Optionally, based on the preference optimization dataset, the training model is directly trained with preference optimization to obtain the trajectory prediction model, including: based on a preset low-rank adapter and the preference optimization dataset, the training model is directly trained with preference optimization to obtain the trajectory prediction model.

[0113] The aforementioned preset low-rank adapter can refer to a tool used for efficient and accurate parameter tuning of existing models to achieve the goal of direct preference optimization. LoRA adjusts only the low-rank part of the model through low-rank decomposition, rather than changing the parameters of the entire model. This method reduces training time and computational resource consumption. The use of the preset low-rank adapter ensures that the embodiments of this application can maintain computational efficiency while optimizing model performance, adapting to resource-constrained deployment environments.

[0114] In one alternative embodiment, firstly, a preference optimization dataset is collected. This dataset includes not only ideal driving behavior as positive samples but also various non-standard or low-quality driving trajectories as negative samples, aiming to clearly distinguish between safe and effective driving preferences and unsafe and ineffective ones. Next, LoRA is used to adjust only the low-rank matrix in the model parameters. In direct preference optimization training, the preference optimization dataset is input into the model after pre-training and fine-tuning phases. Through comparative training with positive and negative samples, the model learns human driving preferences, reducing format violations and low-quality trajectory generation. Ultimately, the trained trajectory prediction model can more accurately predict safe and reasonable driving routes, improving the accuracy and safety of trajectory prediction and avoiding driving safety hazards caused by model output bias.

[0115] This application's embodiments implement model preference learning and optimization for safe and high-quality driving trajectories. This process improves the accuracy and safety of model-generated trajectories, reduces format-incorrect outputs and the generation of low-quality trajectories, and makes the model's predicted future trajectories more closely match human drivers' preferences and actual driving needs. The efficient adaptation mechanism of the pre-set low-rank adapter ensures that the model can quickly learn preferences under limited computing resources, providing autonomous vehicles with more intelligent, safe, and reliable trajectory prediction capabilities, and promoting the robust development and practical application of autonomous driving technology.

[0116] Optionally, the method further includes: during the model inference stage, acquiring multiple fourth images captured by cameras at various angles of the autonomous vehicle, as well as a second driving command input by the user to the autonomous vehicle; inputting the multiple fourth images and the second driving command into the trajectory prediction model to obtain a first predicted trajectory; determining the displacement error and collision rate of the autonomous vehicle driving based on the first predicted trajectory; and performing open-loop feedback adjustment on the trajectory prediction model based on the displacement error and collision rate.

[0117] The aforementioned model inference stage predicts the future trajectory of the autonomous vehicle based on real-time input visual information and user commands. The goal of the model inference stage is to generate a trajectory that meets both safety standards and the requirements of the driving task.

[0118] The aforementioned multiple fourth images refer to images captured in real-time by cameras at various angles of the autonomous vehicle, encompassing panoramic information about the vehicle's surrounding environment. Compared to the images used during the training phase, the fourth images reflect real-time visual data and serve as direct input for the model to predict trajectory, enabling the model to understand the current driving environment and make real-time decisions.

[0119] The aforementioned second driving command can refer to the specific operation command issued by the user to the autonomous vehicle in a real-time driving scenario, which can be input via voice, touch, or other interactive methods.

[0120] The aforementioned first predicted trajectory can refer to the future driving path prediction generated by the trajectory prediction model during the model inference stage, based on the fourth image and the second driving instruction.

[0121] The aforementioned displacement error refers to the degree of difference between the first predicted trajectory and the actual driving trajectory. This indicator is obtained by calculating the distance deviation between trajectory points and is used to quantify the accuracy of the model's prediction.

[0122] The collision rate described above assesses the probability of an autonomous vehicle colliding with surrounding objects or vehicles while traveling based on a first predicted trajectory. This metric quantifies the safety of the model's predicted trajectory by analyzing the distance relationship between the predicted trajectory and obstacles.

[0123] The aforementioned open-loop feedback adjustment refers to the evaluation results based on displacement error and collision rate. By directly adjusting model parameters, open-loop feedback adjustment aims to improve the model's prediction accuracy and safety, ensuring more precise future trajectory predictions and reducing potential collision risks. This adjustment mechanism enables the model to optimize based on feedback from actual driving scenarios, without relying on dynamic adjustments in closed-loop control, thus enhancing the model's adaptability and robustness.

[0124] In one optional embodiment, firstly, during the model inference phase, multiple fourth images from cameras at various angles of the autonomous vehicle are acquired in real time, while simultaneously receiving second driving commands provided by the user as input data. Subsequently, this data is fed into a trajectory prediction model that has been pre-trained, fine-tuned, and optimized using DPO training. The model employs a Chain of Thought (CoT) mechanism to analyze the correlation between visual information and commands, generating a first predicted trajectory. Next, the displacement error and collision rate of the autonomous vehicle following this trajectory are evaluated, i.e., the distance deviation between the predicted trajectory and the ideal safe trajectory, as well as the collision probability, are calculated. Finally, based on these feedback metrics, the model parameters are directly corrected through open-loop feedback adjustment to enhance the accuracy and safety of the model's output trajectory, resolving format violations and low-quality trajectory issues caused by model prediction bias.

[0125] This application's embodiments can generate a first predicted trajectory based on a trajectory prediction model and achieve open-loop feedback adjustment of model performance through quantitative evaluation of displacement error and collision rate. This mechanism aims to directly optimize the model's predictive ability based on feedback from actual driving scenarios, improving the accuracy and safety of predicted trajectories, thereby reducing collision risks and driving deviations encountered by autonomous vehicles during operation. The application of open-loop feedback adjustment technology enhances the model's adaptability to changes in the real-time driving environment, providing autonomous vehicles with real-time, safe, and efficient driving path planning, and promoting a smooth transition of autonomous driving technology from the laboratory to real roads.

[0126] Figure 2 This is a schematic diagram illustrating the structure of an optional training method for a trajectory prediction model of an autonomous vehicle according to an embodiment of this application, as shown below. Figure 2 As shown, the method provided in this embodiment of the invention includes the following steps S1 to S4. Step S1 is pre-training, step S2 is fine-tuning, step S3 is direct preference optimization, and step S4 is inference and evaluation. Steps S1 to S3 are trained using a vehicle-related training set, and the future trajectory is obtained after inference and evaluation using a vehicle-related validation set.

[0127] Figure 3 This is an optional end-to-end VLA model future trajectory prediction flowchart based on DPO-based spatiotemporal CoT inference according to an embodiment of this application, such as... Figure 3 As shown, images from the vehicle's front-facing camera are modulated and quantized to construct a pre-training dataset for pre-training the initial visual-action model. Six 360-degree images of the vehicle and its actual trajectory information are also modulated and quantized to construct a fine-tuning training set and a validation set. The pre-trained model is then fine-tuned, and a DPO training set is constructed. The fine-tuned model is then trained using the DPO-trained model for inference and subsequent open-loop and closed-loop evaluations to determine the future trajectory.

[0128] S1, Pre-training phase. Given all unlabeled frame image data, at a given preset time... First, a modulation vector quantization encoder is used to visually encode the image from the vehicle's front-facing camera 0.5 seconds after a preset time. This encoder captures the spatiotemporal dynamics in the image by quantizing visual features, avoiding the information loss problem of traditional encoders. Then, the image at the given preset time... A pre-training dataset is constructed using the visual encodings of images taken from the front-facing camera and images taken 0.5 seconds after a preset time. Next, the initial visual-action model is trained on this pre-training dataset. The pre-training objective is to maximize the likelihood, and the loss function is... Designed as follows:

[0129] ;

[0130] in, To obtain the expected value, For the current preset time The camera image entered below, The visual word sequence generated in the previous moment. These are the model parameters. This loss function encourages the model to autoregressively predict the visual token sequence for the next frame, facilitating the initial formation of the spatiotemporal CoT. The training process employs a suitable optimizer and learning strategy, progressively optimizing the parameters through an appropriate number of iterations to obtain a pre-trained model and save its parameters.

[0131] S2, Fine-tuning training phase. Using labeled keyframe image data, at a given preset time... First, the same modulation vector quantization encoder is used to visually encode the image from the vehicle's front-facing camera 0.5 seconds after a preset time point. This encoding process remains consistent with the pre-training phase. Then, the image from the vehicle's front-facing camera at the given preset time point is... The following data is used to construct a fine-tuning training set and a corresponding validation set: six 360-degree images of the vehicle (including front left, front, front right, rear left, rear, and rear right views); the vehicle's historical trajectory 3 seconds before a preset time; and the visual encoding of the front-facing camera image 0.5 seconds after the preset time, along with the trajectory points of the vehicle 3 seconds after the preset time. Next, the pre-trained model obtained in stage S1 is loaded, and fine-tuning training is performed on this training set. The fine-tuning objective is to minimize the cross-entropy loss, and the loss function is... for:

[0132] ;

[0133] in, The position of the actual output sequence The word corresponding to the location, The total length of the output sequence. The prompt words are the dichotomized versions of the input words. For model parameters, In the model with parameters And the input word suggestions after dichotomy are Based on the actual output sequence position Location-related lexical units The probability of future trajectories is calculated. This loss function combines text description and trajectory coordinates, encouraging the model to first output 300-500 visual terms before generating accurate future trajectory points. The training process uses a suitable optimizer, refining parameters through an appropriate number of iterations to obtain a fine-tuned model and save its parameters. However, due to a lack of preference optimization, low-quality trajectories can still occur.

[0134] Figure 4 This is a schematic diagram illustrating the construction of the DPO training set for an optional training method of a trajectory prediction model for an autonomous vehicle according to an embodiment of this application. Figure 4 As shown, six 360-degree images of the bicycle and its actual trajectory information are used to convert the bicycle's actual trajectory information into a positive sample data format and construct negative sample data, including: zero trajectory, noisy trajectory and repeated trajectory, forming DPO data pairs.

[0135] S3, in the DPO training phase, this step first constructs preference pairs from the fine-tuned training set, forming DPO data pairs, including positive samples (good outputs) and negative samples (bad outputs). For each sample, positive sample data comes from six 360-degree images of the vehicle and its actual trajectory information; positive samples are correct outputs, which consist of two parts: visual encoding and trajectory coordinates. Negative samples are constructed in three ways: all-zero trajectories, simulating common static errors in the model; noisy trajectories, adding appropriate noise to simulate inaccurate predictions; and repetitive trajectories, simulating deviations due to a lack of dynamism. This construction process ensures that the dataset covers common error patterns, promoting preference alignment.

[0136] Figure 5 This is a schematic diagram illustrating the application of an optional LoRA adapter in the DPO training phase according to an embodiment of this application, as shown below. Figure 5 As shown, in the fine-tuned model, low-rank matrices A1 and B1 are applied to the first-layer weight parameters, and low-rank matrices A2 and B2 are applied to the second-layer weight parameters. Low-rank matrices An and Bn are applied to the weight parameters of the nth layer, and DPO improves training.

[0137] Load the fine-tuned model and train it on the DPO dataset using the LoRA adapter. LoRA adapts the weight matrix through low-rank decomposition. :

[0138] ;

[0139] in, This represents a low-rank update of the weight matrix, used for efficient adaptation of pre-trained models without modifying the original weights; assuming the original weight matrix... The shape is ,in, It is the input dimension. If it is the output dimension, then The shape is , The shape is , much smaller Given a shape (batch size, sequence length, ... Input tensor Then we have:

[0140] ;

[0141] in, This is the scaling factor. This mechanism only updates the low-rank matrix. and This improves computational efficiency and reduces the consumption of computing resources.

[0142] The reward function obtained after reparameterization in DPO is:

[0143] ;

[0144] in, This represents the word sequence output by the model. This indicates the prompt word or context after word dichotomy. Indicates the current policy model. Represents the reference model. Let be the temperature parameter. Based on this, the loss function for DPO is:

[0145] ;

[0146] This loss function encourages the model to favor high-quality outputs and avoid low-quality trajectories. The training process employs an appropriate optimization strategy, optimizing parameters through a suitable number of iterations to obtain the DPO model and save its parameters. The working principle of this stage is to reinforce human feedback through preference alignment, enabling the model to learn without format violations and noise interference, while LoRA ensures improved computational efficiency and reduced computational resource consumption.

[0147] Figure 6 This is an optional spatiotemporal CoT inference process logic diagram according to an embodiment of this application, such as... Figure 6 As shown, during inference, the input includes six images of the vehicle from all 360 degrees (including front left, front, front right, rear left, rear, and rear right views), the vehicle's actual trajectory information, and the second driving command. The model trained by DPO is used to make predictions, and the trajectory information in the prediction results is matched out.

[0148] In S4, during the inference and evaluation phase, real-time images from six cameras and instructions are input during inference. The model generates future frames and then outputs the visual encoding and trajectory points for the predicted future moments through Visual Spatiotemporal CoT. Specifically, open-loop evaluation uses L2 displacement error and collision rate as metrics; closed-loop evaluation uses the Predictive Driver Model Score (PDMS) as the metric. The working principle of this phase is to optimize the model through evaluation feedback, ensuring that the trajectory meets safety standards. Technical effect: This phase confirms the robustness of the model in dynamic scenarios and facilitates the transition from simulation to real-world roads.

[0149] Figure 7 This is a schematic diagram of an optional autonomous driving device structure according to an embodiment of this application, such as... Figure 7 As shown, the device includes a perception module (six cameras: front left camera, front camera, front right camera, rear left camera, rear camera, and rear right camera), a computing module (involving a modulation vector quantization encoder, an initial visual action model, DPO, and LoRA), and a prediction module (inference framework vLLM, open-loop evaluation, and closed-loop evaluation). This device is integrated into the vehicle system to ensure real-time processing.

[0150] In summary, this invention achieves efficient and reliable autonomous driving through a three-stage training process of pre-training, fine-tuning, and DPO, followed by inference and evaluation.

[0151] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0152] According to an embodiment of this application, an apparatus embodiment for training a trajectory prediction model for an autonomous vehicle is provided. It should be noted that this apparatus can be used to execute the aforementioned method for training the trajectory prediction model for an autonomous vehicle. Figure 8 This is a schematic diagram of a training device for an optional trajectory prediction model of an autonomous vehicle according to an embodiment of this application, as shown below. Figure 8 As shown, the device includes: an acquisition module 802, a construction module 804, a pre-training module 806, a training module 808, and a preference optimization training module 810.

[0153] The system comprises the following modules: an acquisition module for acquiring the historical driving dataset of the target vehicle, which includes at least image data and trajectory data collected during the vehicle's operation; a construction module for constructing a target unlabeled dataset, a target labeled dataset, and a preference optimization dataset based on the historical driving dataset, wherein the target unlabeled dataset contains image data, the target labeled dataset contains both image data and trajectory data (the labeled dataset has a smaller data size than the unlabeled dataset), and the preference optimization dataset contains preference feedback information representing user preferences for the image data and trajectory data; a pre-training module for pre-training an initial visual action model based on the target unlabeled dataset; a training module for training the pre-trained model based on the target labeled dataset; and a preference optimization training module for performing direct preference optimization training on the trained model based on the preference optimization dataset to obtain a trajectory prediction model, which is used during the model inference phase to predict the autonomous vehicle's trajectory based on the input image data.

[0154] The acquisition module is also used to acquire the first image captured by the front camera of the target vehicle at a preset time in the historical driving dataset, as well as multiple second images captured within a first preset time window after the preset time; based on the modulation vector quantization encoder, the multiple second images are encoded to obtain multiple visual codes, wherein the modulation vector quantization encoder is used to capture the spatiotemporal dynamic relationship in the image by quantizing visual features; based on the first image and multiple visual codes, a target unlabeled dataset is constructed.

[0155] The pre-training module is also used to pre-train the initial visual action model based on the preset likelihood loss function and the target unlabeled dataset to obtain the pre-trained model. The preset likelihood loss function is used to encourage the initial visual action model to autoregressively predict the visual token sequence of the next frame.

[0156] The construction module is also used to acquire multiple third images of the target vehicle captured by cameras at various angles at preset times in the historical driving dataset, first driving trajectory data within a second preset time window before the preset time, second driving trajectory data within a second preset time window after the preset time, and multiple visual codes corresponding to the multiple second images; and to construct a target labeled dataset based on the multiple third images, the first driving trajectory data, the second driving trajectory data, and the multiple visual codes.

[0157] The training module is also used to obtain the first driving instruction input by the user to the target vehicle within a second preset time window before a preset time in the historical driving dataset; based on the preset cross-entropy loss function, the first driving instruction and the target labeled dataset, the pre-trained model is trained to obtain the training model, wherein the preset cross-entropy loss function is used to guide the alignment of the training model with the expected trajectory corresponding to the first driving instruction.

[0158] The construction module is also used to construct positive samples based on multiple third images, first driving trajectory data and second driving trajectory data in the historical driving dataset, wherein the positive samples are used to represent positive preference feedback information; construct negative samples corresponding to the positive samples, wherein the negative samples include at least one of the following: all-zero trajectory, noisy trajectory and repeated trajectory, wherein the negative samples are used to represent negative preference feedback information; and construct a preference optimization dataset based on the positive samples and negative samples.

[0159] The preference optimization training module is also used to perform direct preference optimization training on the training model based on a preset low-rank adapter and preference optimization dataset to obtain a trajectory prediction model.

[0160] The device is further configured to, during the model inference phase, acquire multiple fourth images captured by cameras at various angles of the autonomous vehicle, as well as a second driving command input by the user to the autonomous vehicle; input the multiple fourth images and the second driving command into the trajectory prediction model to obtain a first predicted trajectory; determine the displacement error and collision rate of the autonomous vehicle driving based on the first predicted trajectory; and perform open-loop feedback adjustment on the trajectory prediction model based on the displacement error and collision rate. Embodiments of this application also provide a vehicle, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods described in the various embodiments of this application during runtime.

[0161] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0162] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0163] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0164] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of this application.

[0165] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0166] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0167] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0169] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0170] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A training method for a trajectory prediction model of an autonomous vehicle, characterized in that, include: Obtain the historical driving dataset of the target vehicle, wherein the historical driving dataset includes at least image data and driving trajectory data collected by the target vehicle during driving; Based on the historical driving dataset, a target unlabeled dataset, a target labeled dataset, and a preference optimization dataset are constructed. The target unlabeled dataset contains the image data, the target labeled dataset contains the image data and the driving trajectory data, and the data volume of the target labeled dataset is smaller than that of the target unlabeled dataset. The preference optimization dataset contains preference feedback information, which is used to represent the user's preference information for the image data and the driving trajectory data. Based on the target unlabeled dataset, the initial visual action model is pre-trained to obtain a pre-trained model; Based on the target labeled dataset, the pre-trained model is trained to obtain the trained model; Based on the preference optimization dataset, the training model is trained by direct preference optimization to obtain a trajectory prediction model, wherein the trajectory prediction model is used to predict the driving trajectory of the autonomous vehicle based on the input image data during the model inference stage.

2. The method according to claim 1, characterized in that, Based on the aforementioned historical driving dataset, a target unlabeled dataset is constructed, including: The first image captured by the front camera of the target vehicle at a preset time and multiple second images captured within a first preset time window after the preset time are obtained from the historical driving dataset. Based on the modulation vector quantization encoder, the plurality of second images are encoded to obtain a plurality of visual codes, wherein the modulation vector quantization encoder is used to capture the spatiotemporal dynamic relationship in the image by quantizing visual features; Based on the first image and the multiple visual codes, the target unlabeled dataset is constructed.

3. The method according to claim 1 or 2, characterized in that, Based on the aforementioned unlabeled target dataset, the initial visual action model is pre-trained to obtain a pre-trained model, including: Based on a preset likelihood loss function and the target unlabeled dataset, the initial visual action model is pre-trained to obtain the pre-trained model, wherein the preset likelihood loss function is used to encourage the initial visual action model to autoregressively predict the visual token sequence of the next frame.

4. The method according to claim 1, characterized in that, Based on the aforementioned historical driving dataset, a target labeled dataset is constructed, including: The historical driving dataset includes multiple third images captured by cameras at various angles of the target vehicle at preset times, first driving trajectory data within a second preset time window before the preset time, second driving trajectory data within the second preset time window after the preset time, and multiple visual codes corresponding to the multiple second images. The target labeled dataset is constructed based on the multiple third images, the first driving trajectory data, the second driving trajectory data, and the multiple visual codes.

5. The method according to claim 1 or 4, characterized in that, Based on the target labeled dataset, the pre-trained model is trained to obtain a trained model, including: The user inputs a first driving command to the target vehicle within a second preset time window before a preset time in the historical driving data set. The pre-trained model is trained based on a preset cross-entropy loss function, the first driving instruction, and the target labeled dataset to obtain the trained model. The preset cross-entropy loss function is used to guide the alignment of the trained model with the expected trajectory corresponding to the first driving instruction.

6. The method according to claim 1, characterized in that, The preference feedback information includes positive preference feedback information and negative preference feedback information; Based on the aforementioned historical driving dataset, a preference optimization dataset is constructed, including: Based on multiple third images, first driving trajectory data and second driving trajectory data in the historical driving dataset, positive samples are constructed, wherein the positive samples are used to characterize the positive preference feedback information; Construct negative samples corresponding to the positive samples, wherein the negative samples include at least one of the following: all-zero trajectories, noisy trajectories, and repeated trajectories, and the negative samples are used to characterize the negative preference feedback information; Based on the positive samples and the negative samples, the preference optimization dataset is constructed.

7. The method according to claim 1 or 6, characterized in that, Based on the aforementioned preference optimization dataset, the training model is directly trained using preference optimization to obtain a trajectory prediction model, including: Based on the preset low-rank adapter and the preference optimization dataset, the training model is directly trained with preference optimization to obtain the trajectory prediction model.

8. The method according to claim 1, characterized in that, The method further includes: During the model inference phase, multiple fourth images captured by cameras at various angles of the autonomous vehicle are acquired, as well as the second driving command input by the user to the autonomous vehicle. The multiple fourth images and the second driving command are input into the trajectory prediction model to obtain the first predicted trajectory; Determine the displacement error and collision rate of the autonomous vehicle based on the first predicted trajectory; Based on the displacement error and the collision rate, the trajectory prediction model is adjusted using open-loop feedback.

9. A training device for a trajectory prediction model of an autonomous vehicle, characterized in that, include: The acquisition module is used to acquire the historical driving dataset of the target vehicle, wherein the historical driving dataset includes at least image data and driving trajectory data collected by the target vehicle during driving. The construction module is used to construct a target unlabeled dataset, a target labeled dataset, and a preference optimization dataset based on the historical driving dataset. The target unlabeled dataset contains the image data, the target labeled dataset contains the image data and the driving trajectory data, the data volume of the target labeled dataset is less than the data volume of the target unlabeled dataset, and the preference optimization dataset contains preference feedback information, which is used to represent the user's preference information for the image data and the driving trajectory data. The pre-training module is used to pre-train the initial visual action model based on the target unlabeled dataset to obtain a pre-trained model. The training module is used to train the pre-trained model based on the target labeled dataset to obtain the trained model; The preference optimization training module is used to perform direct preference optimization training on the training model based on the preference optimization dataset to obtain a trajectory prediction model. The trajectory prediction model is used to predict the driving trajectory of the autonomous vehicle based on the input image data during the model inference stage.

10. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 8.

Images