Trajectory prediction planning and model training method and device, medium and equipment

Abstract

The application provides a trajectory prediction planning and a training method, device, medium and equipment thereof. The trajectory prediction planning model training method comprises: acquiring multiple groups of training data grouped in time sequence; when the current training data is the first group of training data in the multiple groups of training data, inputting the current training data into a target model to obtain a first output result, and calculating a first loss based on the first output result; when the current training data is not the first group of training data, generating target training data according to the last group of input data and the last group of output results of the target model, inputting the target training data into the target model to obtain a second output result, and calculating a second loss based on the second output result; determining a target loss according to the first loss and the second loss, determining model parameters of the target model based on the target loss, and obtaining a trajectory prediction and planning model. The application can alleviate the error accumulation problem and improve the stability and reliability of the trajectory prediction and planning model.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method, apparatus, medium, and device for training trajectory prediction planning and its model. Background Technology

[0002] In recent years, autonomous driving technology has been widely researched and applied in the field of intelligent transportation. A typical autonomous driving system usually includes functional modules such as environmental perception, behavior prediction, path planning, and vehicle control. Among these, the path planning module plays a core role in specific driving decisions, and its output quality directly affects the safety and driving experience of the entire vehicle. Existing autonomous driving path planning methods mainly rely on a combination of end-to-end neural networks based on deep learning and traditional phased architectures (perception, planning, and control). For neural network-based planning models, the industry typically adopts an open-loop training approach: by using a large amount of feature data collected from real or simulated scenarios and corresponding manually labeled trajectories, the model is trained to generate planned paths under given environmental inputs. However, while this type of open-loop training can achieve high accuracy with similarly distributed data, its decision safety and robustness need improvement in complex traffic environments, especially in situations with perceptual noise, atypical behaviors of dynamic traffic participants, and long-tail scenarios such as road construction and extreme weather. Summary of the Invention

[0003] This application provides a method, apparatus, medium, and device for trajectory prediction planning and its model training to solve one or more of the above-mentioned technical problems.

[0004] In a first aspect, embodiments of this application provide a trajectory prediction and planning model training method, comprising: acquiring multiple sets of training data grouped in chronological order; the training data including vehicle state data; when the current training data is the first set of training data in the multiple sets of training data, inputting the current training data into a target model to obtain a first output result, and calculating a first loss based on the first output result; when the current training data is not the first set of training data, generating target training data according to the previous set of input data and the previous set of output results of the target model, inputting the target training data into the target model to obtain a second output result, and calculating a second loss based on the second output result; determining a target loss according to the first loss and the second loss, and determining the model parameters of the target model based on the target loss to obtain a trajectory prediction and planning model.

[0005] Secondly, embodiments of this application provide a trajectory prediction and planning method, comprising: acquiring vehicle driving data; the driving data including vehicle state data and environmental feature data; inputting the driving data into a trajectory prediction and planning model to obtain trajectory prediction results and trajectory planning results; wherein the trajectory prediction and planning model is trained according to the trajectory prediction and planning model training method described in any of the above claims.

[0006] Thirdly, embodiments of this application provide a trajectory prediction and planning model training device, comprising: an acquisition module, configured to acquire multiple sets of training data grouped in chronological order; the training data including vehicle state data; a first calculation module, configured to input the current training data into a target model to obtain a first output result when the current training data is the first set of training data in the multiple sets of training data, and calculate a first loss based on the first output result; a second calculation module, configured to generate target training data based on the previous set of input data and the previous set of output results of the target model when the current training data is not the first set of training data, input the target training data into the target model to obtain a second output result, and calculate a second loss based on the second output result; and a loss module, configured to determine a target loss based on the first loss and the second loss, determine the model parameters of the target model based on the target loss, and obtain a trajectory prediction and planning model.

[0007] Fourthly, embodiments of this application provide a trajectory prediction and planning device, comprising: a data module for acquiring vehicle driving data; the driving data including vehicle state data and environmental feature data; and a prediction and planning module for inputting the driving data into a trajectory prediction and planning model to obtain trajectory prediction results and trajectory planning results; wherein the trajectory prediction and planning model is trained according to the trajectory prediction and planning model training method described in any of the above claims.

[0008] Fifthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method described in any of the above-mentioned embodiments.

[0009] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any of the preceding claims.

[0010] In a seventh aspect, embodiments of this application provide a computer program product, wherein the computer program product includes a computer program that, when executed by a processor, implements the method described in any of the preceding claims.

[0011] Compared with related technologies, this application has the following advantages:

[0012] This application provides a method, apparatus, medium, and device for training a trajectory prediction and planning model. The trajectory prediction and planning model training method includes: acquiring multiple sets of training data grouped in chronological order; the training data includes vehicle state data; when the current training data is the first set of training data in the multiple sets of training data, inputting the current training data into a target model to obtain a first output result, and calculating a first loss based on the first output result; when the current training data is not the first set of training data, generating target training data based on the previous set of input data and the previous set of output results of the target model, inputting the target training data into the target model to obtain a second output result, and calculating a second loss based on the second output result; determining a target loss based on the first loss and the second loss, and determining the model parameters of the target model based on the target loss to obtain a trajectory prediction and planning model. This application employs a two-stage training approach. During the training phase using the first set of training data, open-loop learning is performed based on the vehicle's real-state data to obtain a stable initial model. Subsequently, a fine-tuning phase is initiated, using the vehicle's real-state data (i.e., the previous set of input data) and the self-predicted trajectory of the target model (i.e., the previous set of output results) to generate target training data. Based on the target training data, a second output result and a second loss are obtained. Then, the target loss is determined based on the first loss and the second loss. By introducing a feedback loop of the model's self-generated trajectory, the problem of error accumulation is mitigated, and the stability and reliability of the trajectory prediction and planning model are improved.

[0013] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0014] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0015] Figure 1 A flowchart of a trajectory prediction and planning model training method provided in an embodiment of this application is shown;

[0016] Figure 2 This illustration shows a schematic diagram of an overall framework for hybrid training of open-loop and closed-loop models provided in an embodiment of this application.

[0017] Figure 3This illustration shows a schematic diagram of a training strategy provided in an embodiment of this application;

[0018] Figure 4 A flowchart of a trajectory prediction and planning method provided in an embodiment of this application is shown;

[0019] Figure 5 This paper shows a structural block diagram of a trajectory prediction and planning model training device provided in an embodiment of this application;

[0020] Figure 6 This paper shows a structural block diagram of a trajectory prediction and planning device provided in an embodiment of this application;

[0021] Figure 7 A block diagram of an electronic device used to implement embodiments of this application is shown. Detailed Implementation

[0022] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0023] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0024] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0025] This application provides a method for training a trajectory prediction and planning model. For example... Figure 1 The diagram shown is a flowchart of a trajectory prediction and planning model training method according to an embodiment of this application. The method may include:

[0026] Step S101: Obtain multiple sets of training data grouped in chronological order; the training data includes vehicle status data.

[0027] In this possible implementation, the training set data used in a single training session includes multiple sets of training data. Each set of training data can correspond to a time period, specifically including a start time and an end time. The multiple sets of training data can be grouped and sorted according to the order of their start times. For example, the time periods corresponding to the multiple sets of training data are 0 seconds to 1 second, 1 second to 2 seconds, 2 seconds to 3 seconds, and 3 seconds to 4 seconds, respectively. After sorting these four sets of data according to their start times, we obtain the first set of training data, the second set of training data, the third set of training data, and the fourth set of training data in sequence.

[0028] Each set of training data may include vehicle state data, which can be used to describe the vehicle's trajectory. This state data may include, for example, the vehicle's historical state, including but not limited to, data such as position, speed, acceleration, and heading angle. The state data may also include multimodal perception information of surrounding traffic participants, such as their trajectories.

[0029] In one possible implementation, the vehicle's state data includes the trajectory data of the current vehicle and the trajectory data of a designated vehicle. In this embodiment, the trained trajectory prediction and planning model is used to plan the trajectory of the current vehicle in the future and predict the trajectory of a designated vehicle in the future. The designated vehicle refers to vehicles surrounding the current vehicle, and the distance between them can be less than a specified distance. For example, it may include, but is not limited to: vehicles within n meters in front of the current vehicle, vehicles within m meters behind the current vehicle, and vehicles in one or more adjacent lanes.

[0030] It should be noted that the label data in each training data set is a reference trajectory, which can be a real trajectory obtained by a human driver or generated by a high-confidence planner. The target model can be a neural network model. This application does not specifically limit the specific structure of the target model, and the structure of the model can be determined according to actual needs.

[0031] In this step, multiple sets of training data are acquired, grouped chronologically, with each set including vehicle state data. This allows the training process to preserve the time-series characteristics of the vehicle's operation. Grouping and sorting the training data according to the order of its start time not only clearly reflects the dynamic changes of the vehicle over continuous time periods but also facilitates the model's learning of the evolution of vehicle state over time during training, thereby improving the model's understanding of continuous scenarios and prediction accuracy. Furthermore, the chronologically organized data structure helps reduce random interference in the training samples over time, lowers the complexity of feature extraction, and enables the network to more stably capture temporal correlation information during driving, thus effectively improving the model's dynamic response performance and robustness in subsequent inference and planning.

[0032] Step S102: When the current training data is the first set of training data among the multiple sets of training data, input the current training data into the target model to obtain the first output result, and calculate the first loss based on the first output result.

[0033] In this possible implementation, the current training data refers to the set of data selected by the current training loop from multiple sets of training data grouped in chronological order. According to the grouping and sorting results of step S101, the first set of training data typically corresponds to the initial time period of the training set, such as vehicle state data within 0 to 1 second. The target model refers to a neural network model used to perform planning and prediction tasks, such as an end-to-end path planning model or behavior prediction model. This model receives input feature data during the training phase and outputs output results related to the task objective. The output result refers to the output information generated by the target model based on the input data, specifically including trajectory prediction results for vehicles surrounding the target vehicle and trajectory planning results for the target vehicle. For example, the trajectory prediction result or trajectory planning result includes a set of future trajectory points of the vehicle, control commands (steering angle, acceleration, etc.), or other parameters characterizing vehicle behavior, respectively. In this embodiment, when the current training data is the first set of training data among the multiple sets of training data, the current training data is input into the target model, and the resulting output result is used as the first output result. The loss calculated based on the first output result is called the first loss. The first loss includes one or more of the following types: supervised loss, collision loss, smoothing loss, and comfort loss. Supervised loss is a numerical metric used during deep learning training to measure the difference between the model's output and the true label; it is also called "cost" or "error." Collision loss can be calculated using the planned trajectory of the vehicle and the trajectories and dimensions of surrounding vehicles. Smoothing loss can be calculated using the model's output trajectory. In this embodiment, the first loss can be calculated by a loss function, such as mean squared error (MSE), cross-entropy loss, or other evaluation functions suitable for planning tasks, to guide the updating of model parameters.

[0034] In this step, the neural network planning model is trained end-to-end in an open-loop manner using a large-scale real-world driving dataset. By inputting the first set of training data into the target model and calculating the first loss, an initial assessment of model performance and parameter tuning can be achieved from the beginning of the training sequence. This design ensures that the model establishes a stable prediction baseline in the initial stage of the time series, helping to reduce the accumulation of biases in subsequent stages of the sequence. Since the first set of training data typically reflects the initial state of the vehicle and the initial environmental conditions, its feature distribution is relatively concentrated and controllable for the model. The first loss calculated using this set of data can quickly identify the model's convergence trend and discover potential problems early in the training process. Furthermore, this step provides a reference benchmark when introducing closed-loop feedback or subsequent sets of data, thereby further enhancing the model's stability and robustness in dynamic sequence tasks.

[0035] Step S103: When the current training data is not the first set of training data, generate target training data based on the previous set of input data and the previous set of output results of the target model, input the target training data into the target model to obtain the second output result, and calculate the second loss based on the second output result.

[0036] In this possible implementation, the previous set of input data refers to a set of feature data immediately preceding the current training data in the time series of the training process, such as the vehicle's state information and environmental perception data collected in the previous time period (e.g., 1 to 2 seconds). The previous set of output results refers to the output generated by the target model when processing the previous set of input data, such as the predicted next trajectory of the vehicle, control commands, or other parameters related to the driving task. The target training data, in this embodiment, is the training input composed of the previous set of input data and the previous set of output results, used to simulate the actual inference process under closed-loop operating conditions during the training phase. This data includes not only the actually collected state data but also the predicted output generated by the model itself, thus reflecting the model's ability to utilize and correct its historical outputs during continuous operation.

[0037] When the current training data is not the first set of training data, target training data is generated based on the previous set of input data and output results of the target model. This target training data is then input into the target model, and the resulting output is used as the second output result. The loss calculated based on the second output result is called the second loss. The second loss includes one or more of the following types: supervised loss, collision loss, smoothing loss, and comfort loss. The supervised loss is a measure of the difference between the output result of the target model under closed-loop conditions and the corresponding true label on the target training data. This loss can use the same or different loss function as the first loss, such as mean squared error (MSE) or weighted error, to measure the model's performance under closed-loop inference conditions. The collision loss can be calculated using the vehicle's planned trajectory and the trajectories and dimensions of surrounding vehicles. The smoothing loss can be calculated using the model's output trajectory.

[0038] When the current training data is not the first set of training data, introducing the output results and input data from the previous set to generate target training data and inputting it into the model can effectively simulate the closed-loop characteristics of the online inference process of autonomous driving systems. This allows the model to handle state changes caused by its own output during the training phase. This design not only significantly reduces the error accumulation problem in a purely open-loop training mode, but also effectively alleviates the distribution drift caused by the inconsistency between the training data distribution and the actual operational data distribution, improving the robustness and generalization ability of the model in long-term sequence tasks and complex traffic scenarios. Furthermore, by introducing closed-loop feedback, the model can continuously learn how to correct its own deviations during the training cycle, thereby improving the stability and safety of planning and control, and providing a more reliable decision-making basis for the actual deployment of autonomous driving systems.

[0039] Step S104: Determine the target loss based on the first loss and the second loss, and determine the model parameters of the target model based on the target loss to obtain the trajectory prediction and planning model.

[0040] In this possible implementation, model parameters refer to the adjustable values ​​that constitute the target model, including the weights and biases of the neural network. These parameters are continuously updated during training using optimization algorithms (such as stochastic gradient descent (SGD) and the Adam optimizer) to improve the accuracy of model prediction and planning. The trajectory prediction and planning model is the target model formed after training and optimization. It can predict future driving trajectories based on the vehicle's current state data and environmental information, and generate planned paths that comply with traffic rules and safety constraints, providing a decision-making basis for the vehicle control system.

[0041] In this step, by jointly determining the target loss based on the first and second losses, and updating the model parameters of the target model based on this target loss, the training process can simultaneously consider both accuracy under open-loop conditions and stability under closed-loop conditions, enabling the model to maintain high performance under different operating modes. Furthermore, the comprehensive calculation of the target loss effectively integrates the model's prediction baseline in the initial time period with its robustness performance in the middle and subsequent stages of the sequence, thereby mitigating performance degradation caused by error accumulation and data distribution drift. This training method results in a trajectory prediction and planning model with stronger generalization ability and adaptability to abnormal scenarios, enabling it to stably output safe, smooth, and executable trajectory planning results in long-tail and complex traffic environments, significantly improving the decision-making reliability and safety of autonomous driving systems.

[0042] This application provides a method for training a trajectory prediction and planning model. The method includes: acquiring multiple sets of training data grouped in chronological order; the training data includes vehicle state data; when the current training data is the first set of training data in the multiple sets of training data, inputting the current training data into a target model to obtain a first output result, and calculating a first loss based on the first output result; when the current training data is not the first set of training data, generating target training data based on the previous set of input data and the previous set of output results of the target model, inputting the target training data into the target model to obtain a second output result, and calculating a second loss based on the second output result; determining a target loss based on the first loss and the second loss, and determining the model parameters of the target model based on the target loss to obtain a trajectory prediction and planning model. This application employs a two-stage training approach. During the initial training phase using the first set of training data, open-loop learning is performed based on the vehicle's real-state data to obtain a stable initial model. Subsequently, a fine-tuning phase is initiated, where the vehicle's real-state data (i.e., the previous set of input data) and the target model's self-predicted trajectory (i.e., the previous set of output results) are used to generate target training data. Based on the target training data, a second output result and a second loss are obtained. Then, the target loss is determined based on the first loss and the second loss. By introducing a feedback loop of the model's self-generated trajectory, the problem of error accumulation is mitigated, thereby improving the stability and reliability of the trajectory prediction and planning model.

[0043] In one possible implementation, the training data further includes environmental feature data of the vehicle; inputting the current training data into the target model can be performed by the following steps: adjusting the environmental feature data and the state data, and inputting the adjusted data into the target model; the adjustment includes coordinate system transformation and time alignment processing.

[0044] In this possible implementation, vehicle state data refers to a set of parameters that describe the vehicle's own operating state, such as speed, acceleration, heading angle, steering angle, and position coordinates. This data is collected by vehicle sensors, control systems, or positioning modules, reflecting the vehicle's dynamic state at a specific moment. Environmental feature data refers to data information reflecting the environment surrounding the vehicle, such as the location and shape of objects collected by perception systems (cameras, radar, lidar), road geometry, traffic sign locations, and obstacle distribution. This data is used to assist the model in perceiving and understanding the external environment. Coordinate system transformation refers to converting spatial position data collected from different sensors or data sources to a unified reference coordinate system. For example, converting camera imaging coordinates to the vehicle's body coordinate system, or further to the world coordinate system, to ensure the comparability of position information from different sources. Time alignment processing refers to calibrating timestamps from different data sources, mapping vehicle state data and environmental feature data to the same time point, thereby avoiding temporal misalignment caused by acquisition delays or frequency differences, and achieving data consistency in the time dimension.

[0045] In this possible implementation, by performing coordinate system transformation and time alignment on vehicle environmental feature data and state data before model input, the problems of spatial reference differences and temporal asynchrony in multi-sensor data can be effectively solved, thereby ensuring the consistency and accuracy of input data and helping the target model learn a stable and realistic relationship between the environment and vehicle state. Coordinate system transformation enables the model to accurately fuse perception information from different sensors under a unified spatial reference; time alignment ensures the precise temporal correspondence between state information and environmental features, avoiding prediction bias caused by information misalignment. This data adjustment strategy significantly improves the input quality of the model, reduces the interference of feature noise on training and inference, thereby improving the accuracy and robustness of trajectory prediction and planning results, especially in long-tail scenarios and complex traffic environments, enhancing the model's generalization ability and safety.

[0046] In one possible implementation, generating target training data based on the previous set of input data and the previous set of output results of the target model can be performed according to the following steps: concatenating the first data in the previous set of input data with the second data in the previous set of output results according to timestamps; transforming the concatenated data to obtain the target training data; the transformation includes one or more of the following operations: coordinate system transformation, data deletion, and data addition.

[0047] In this possible implementation, the first data refers to a selected portion of key data from the previous set of input data. This could be environmental feature information corresponding to a timestamp or vehicle dynamic parameters, used to combine with the previous set of output results to form a new input. The second data refers to a portion of information from the previous set of output results, such as predicted position coordinates and velocity direction vectors, which have a temporal correlation with the first data. Timestamp concatenation refers to aligning and concatenating the first and second data in time series, using timestamps as a reference, to ensure accurate correspondence between data from different sources in the time dimension.

[0048] Transformation refers to further processing of the concatenated data to form inputs that can be directly used for closed-loop training of the model. This includes, but is not limited to: coordinate system transformation: converting spatial representations from different data sources to a unified reference system to reduce fusion errors caused by differences in spatial benchmarks; data deletion: removing abnormal, incomplete, or irrelevant data points to reduce noise interference with model training; and data addition: supplementing missing features as needed, such as inserting predicted or interpolated data points to improve the input structure when perceptual blank areas exist. Target training data: The training samples obtained after the above concatenation and transformation are used as inputs to the target model during the closed-loop training phase to simulate the next environmental state formed by its own output during actual inference.

[0049] In this possible implementation, by precisely concatenating the first data point from the previous set of input data with the second data point from the previous set of output results along the timestamp dimension, and then performing coordinate system transformation, data deletion, and data addition on the concatenated data, the quality and consistency of input data in the closed-loop training phase can be effectively improved. Timestamp concatenation ensures the accurate temporal correspondence between the historical states acquired by the model during training and its own output results, helping to maintain the integrity of temporal causal relationships; coordinate system transformation unifies the spatial reference system of data from different sources, reducing spatial errors in information fusion; data deletion and data addition are used to remove abnormal noise and supplement missing information, respectively, thereby improving the effectiveness and representativeness of the input data. The target training data generated through this step is closer to the input distribution under actual closed-loop conditions, which can significantly reduce the impact of distribution drift and error accumulation on model performance, and enhance the robustness and generalization ability of trajectory prediction and planning models in complex dynamic scenarios.

[0050] In one possible implementation, determining the target loss based on the first loss and the second loss can be performed by taking any one or more combinations of the first loss and the second loss as the target loss.

[0051] In this possible implementation, the target loss refers to the comprehensive error metric used to guide the optimization of model parameters. In some possible implementations, the target loss can be either the first loss or the second loss, or a combination thereof (such as weighted summation, weighted average, etc.), in order to optimize model performance in a targeted manner according to different training stages or task requirements.

[0052] In this possible implementation, by flexibly selecting a first loss, a second loss, or a combination of both during the determination of the target loss, open-loop performance and closed-loop robustness can be dynamically balanced for different training objectives. On the one hand, when the first loss is chosen as the target loss, the model can quickly improve the prediction accuracy on standard input data; on the other hand, when the second loss is chosen as the target loss, the model will focus more on its ability to suppress error accumulation and distribution drift in closed-loop operation; and choosing a combination of both as the target loss balances open-loop baseline performance and closed-loop stability, achieving the best balance between overall performance and adaptability to actual operating conditions during training. This method provides a highly flexible optimization strategy for training trajectory prediction and planning models, significantly improving the model's generalization ability and safety in complex traffic environments and long-tail scenarios.

[0053] In one possible implementation, the target loss can be obtained by taking any one or more of the first loss and the second loss as a combination of the two losses. This can be done by summing the first loss and the multiple second losses to obtain the target loss.

[0054] In this possible implementation, by summing the first loss with multiple second losses to obtain the target loss, the model's performance under both open-loop and closed-loop operating conditions can be considered simultaneously during the training and optimization process. Furthermore, the performance of closed-loop training at different time periods is fully incorporated into the optimization objective. This comprehensive calculation method guides the model to continuously improve its robustness and error correction capabilities in long-term sequence inference and complex dynamic traffic environments while maintaining initial prediction accuracy. The integration of multi-stage second losses into the target loss effectively suppresses the accumulation of errors in the time series during training, mitigating performance degradation caused by data distribution drift. The resulting trajectory prediction and planning model achieves higher stability and security in practical deployments, especially exhibiting stronger generalization capabilities in long-tail and abnormal scenarios.

[0055] In one possible implementation, the status data includes: trajectory data of the current vehicle and trajectory data of a specified vehicle; the environmental feature data includes one or more of map data, vehicle category data, and obstacle data.

[0056] In this possible implementation, the current vehicle's trajectory data is collected by the vehicle's own positioning system (such as GNSS, IMU, wheel odometer) or a fusion sensing module, recording the vehicle's continuous position points, velocity vectors, heading angles, and other information over a certain time span, used to represent the vehicle's actual driving trajectory. The trajectory data of designated vehicles refers to the historical or real-time trajectory information of other selected target vehicles around the current vehicle. This data can be obtained by a sensing system (radar, lidar, cameras, etc.) through tracking algorithms, reflecting the target vehicles' motion state and behavior patterns, used by the model to predict their future paths during planning.

[0057] Map data, sourced from high-precision maps or online map services, includes information such as road geometry, lane layout, traffic sign locations, and speed limits, providing geometric and rule constraints for the model during trajectory planning. Vehicle category data, identified and classified by the perception module, identifies the types of traffic participants, such as cars, trucks, buses, motorcycles, and bicycles, enabling the model to make differentiated predictions based on the behavioral characteristics of different vehicle categories. Obstacle data identifies the location, size, and type of static or dynamic obstacles on the road (such as parked vehicles, construction barriers, fallen objects, and pedestrians), used by the planning algorithm for obstacle avoidance and safety strategy decisions.

[0058] In this possible implementation, by simultaneously including current vehicle trajectory data and specified vehicle trajectory data in the state data, the target model can consider its own motion state and the dynamic behavior of key traffic participants during the planning process, improving the decision-making accuracy in interactive scenarios (such as merging, lane changing, and obstacle avoidance). Introducing environmental feature information such as map data, vehicle category data, and obstacle data into the model input provides multi-dimensional information such as road geometric constraints, traffic rules, and environmental risk factors, making trajectory prediction and planning results more consistent with actual road conditions and safety requirements. This input structure achieves comprehensive modeling of the vehicle's own state and the external environment, significantly improving the model's generalization ability and robustness in complex traffic environments, long-tail scenarios, and abnormal events, reducing collision risks, and improving the smoothness and executability of driving paths.

[0059] This application provides a method for training trajectory prediction and planning models, such as Figure 2As shown, this method can include acquiring training data from an offline dataset, and sequentially performing simulation learning and prediction training, establishing a basic planning model, and closed-loop training fine-tuning to finally obtain a planning model. Specifically, the offline dataset module provides raw data containing vehicle operating status and environmental characteristics; the simulation learning and prediction training module generates preliminary output results based on this data and forms a basic planning model; this basic planning model is optimized and adjusted in the closed-loop training fine-tuning module by incorporating supplementary data from the offline dataset; after the above steps, the output planning model can be used to implement trajectory prediction and path planning tasks. This graphical flowchart illustrates the sequential relationship between data acquisition, model training, and optimization, as well as the data transfer paths between modules, where arrows indicate the data flow and processing dependencies between different modules.

[0060] This application embodiment trains the target model by constructing a two-stage hybrid training framework of "imitation learning pre-training and closed-loop training fine-tuning". The specific training methods for each stage are described below:

[0061] The first stage utilizes a large-scale real-world driving dataset to perform end-to-end open-loop training on the neural network planning model. This dataset includes the vehicle's historical state (e.g., position, speed, acceleration, heading angle), multimodal perception information of surrounding traffic participants (e.g., trajectory, category, intent), high-precision map features (e.g., lane topology, traffic rules), and corresponding expert reference trajectories (generated by human drivers or high-confidence planners). In this stage, the model's input at each time step is the actual observed state, without using its own output results, thus ensuring that the model learns robust prior knowledge of basic driving behaviors.

[0062] See Figure 3 , Figure 3 This demonstrates the time-series data processing flow for training or inference. Vertically, it shows five processing layers: offline data, vehicle state, environmental features, model, and action / planning trajectory. Horizontally, it is arranged chronologically, corresponding to time 0 through 4 in the time series from left to right. Specifically, at each time point, the topmost log node log[n] stores or records the raw input information for that time. This log data is processed to obtain state data s[n], which describes the vehicle's operating state and environmental conditions at that time. The state data is further converted into preprocessed vehicle state parameters and environmental feature parameters, i.e., target training data, which is input to the model. In the model layer, the target training data x[n] for the corresponding time point is received and calculated based on a predetermined trajectory prediction or path planning algorithm, outputting the action data a[n] for that time point. The action data characterizes the decision results generated by the model at that time, such as trajectory points and control commands. Figure 3The paper also illustrates data dependencies between different time steps, such as the connection between log data and subsequent time state data, as well as the continuous transmission relationship between state data, feature data, model data, and action data in adjacent time steps. This time-series processing structure maintains the temporal correlation of data during training and inference, and enables model computation and output across time points.

[0063] The model obtained from the first stage of imitation learning pre-training is used as the initial parameters to enter the closed-loop fine-tuning stage. For example... Figure 3 As shown, in this stage, the model no longer relies solely on the real historical state as input during training. Instead, it uses the trajectory state predicted in the previous time step as part of the input for the current time step, constructing a temporal closed-loop inference chain of "prediction → execution → re-prediction". Crucially, this embodiment introduces a cross-time-step error feedback mechanism in the closed-loop fine-tuning: after generating a complete predicted trajectory, the model compares this trajectory with the expert reference trajectory to identify subsequent trajectory divergence or safety issues caused by early decision-making biases. Subsequently, this global bias information is used to retrospectively adjust the model's output strategy at the beginning and middle time steps of the trajectory. In other words, the prediction errors exposed in later moments serve as a supervisory signal, guiding the model to correct its decision-making behavior in previous moments, thereby breaking the vicious cycle of unidirectional error accumulation in traditional open-loop models. Through this mechanism, the model not only learns "how to predict the next step" but also learns "how to make long-term decisions that remain stable and safe in the future," significantly improving its robustness, self-correction ability, and trajectory consistency in closed-loop deployment scenarios.

[0064] To improve the decision-making safety and robustness of autonomous driving systems in complex traffic environments, this application provides a trajectory prediction planning and its training method, device, medium, and equipment. The trajectory prediction planning model training method designs a neural network planning model based on a hybrid open-loop and closed-loop training mechanism. By introducing a feedback loop for the model's self-generated trajectory, it effectively suppresses error accumulation and distribution drift during the inference process, and significantly enhances the model's generalization and error correction capabilities for abnormal and long-tailed scenarios.

[0065] In this embodiment, a two-stage strategy of "imitation learning pre-training and closed-loop temporal fine-tuning" is adopted. The pre-training stage uses open-loop learning based on real-world conditions to obtain a robust initial policy. The fine-tuning stage uses the model's self-predicted trajectory as input to construct a closed-loop inference chain. This architecture effectively bridges the distributional differences between open-loop training and closed-loop deployment, significantly alleviates error accumulation problems, and improves the stability and reliability of the model in actual operation. In closed-loop fine-tuning, the complete predicted trajectory generated by the model is compared with the expert trajectory to identify subsequent problems caused by early decision-making biases. This bias information is then used to backtrack and optimize the output policy of previous time steps, achieving "correcting past decisions with future errors." This embodiment provides a training framework independent of specific model structures, widely adaptable to various end-to-end neural network planning models, supporting different input modalities and output formats, and possessing good scalability and compatibility.

[0066] This application also provides a trajectory prediction and planning method, such as... Figure 4 The diagram shown is a flowchart of a trajectory prediction and planning method according to an embodiment of this application. The method may include:

[0067] Step S401: Obtain vehicle driving data; the driving data includes vehicle status data and environmental feature data.

[0068] In this application embodiment, driving data refers to the raw or processed data set describing the vehicle's operation in a road environment. State data characterizes the vehicle's dynamic operation, such as speed, acceleration, heading angle, steering angle, and position coordinates, and is collected by onboard sensors, positioning systems, or control units. Environmental feature data describes the traffic and road environment surrounding the vehicle, such as road geometry information (lane position, curvature, etc.), the position and category of traffic participants (other vehicles, pedestrians), obstacle information, traffic signs and signal light status, etc., and can be acquired by sensing devices such as cameras, radar, and lidar, or provided by high-precision maps.

[0069] In this step, by acquiring driving data containing vehicle state data and environmental feature data, and inputting it into the trajectory prediction and planning model trained using the above method, accurate prediction of the vehicle's future motion trend and planning of driving paths that comply with road and safety constraints can be achieved. The introduction of state data ensures that the model fully captures the vehicle's own dynamic characteristics, while the introduction of environmental feature data enables the model to identify external traffic and road conditions. The combination of the two can significantly improve the accuracy and practicality of prediction and planning.

[0070] Step S402: Input the driving data into the trajectory prediction and planning model to obtain trajectory prediction results and trajectory planning results; wherein, the trajectory prediction and planning model is trained according to the method described in any of the above-mentioned methods.

[0071] In this embodiment, the trajectory prediction and planning model is a neural network or other machine learning model trained according to the aforementioned method. It can predict a series of future trajectory points of the vehicle based on input driving data (trajectory prediction result), and on this basis, combine road constraints and safety rules to generate an executable driving path (trajectory planning result). The trajectory prediction result refers to the model's predicted output of the vehicle's position and motion state at future moments based on the current vehicle state and environmental characteristics. The trajectory planning result refers to the executable driving path designed by the model based on the predicted trajectory, combined with road geometry, traffic rules, and safety constraints, used to guide actual vehicle control.

[0072] The model trained using the above method exhibits higher robustness and generalization ability in complex traffic environments and long-tail scenarios. It can effectively reduce trajectory deviation, suppress error accumulation, and cope with distribution drift during the inference stage, thereby improving the reliability and safety of autonomous driving system decision-making.

[0073] The trajectory prediction and planning model in this application adopts a two-stage training phase. When training with the first set of training data, open-loop learning is performed based on the vehicle's real state data to obtain a stable initial model. Then, in the fine-tuning phase, the vehicle's real state data, i.e., the previous set of input data, and the self-predicted trajectory of the target model, i.e., the previous set of output results, are used to generate target training data. Based on the target training data, the second output result and the second loss are obtained. Then, the target loss is determined based on the first loss and the second loss. By introducing a feedback loop of the model's self-generated trajectory, the problem of error accumulation is alleviated, and the stability and reliability of the trajectory prediction and planning model are improved.

[0074] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide a trajectory prediction and planning model training device. For example... Figure 5 The diagram shown is a structural block diagram of a trajectory prediction and planning model training device according to an embodiment of this application. The device may include:

[0075] The acquisition module 501 is used to acquire multiple sets of training data grouped in chronological order; the training data includes vehicle state data; the first calculation module 502 is used to input the current training data into the target model when the current training data is the first set of training data in the multiple sets of training data, obtain a first output result, and calculate a first loss based on the first output result; the second calculation module 503 is used to generate target training data based on the previous set of input data and the previous set of output results of the target model when the current training data is not the first set of training data, input the target training data into the target model, obtain a second output result, and calculate a second loss based on the second output result; the loss module 504 is used to determine the target loss based on the first loss and the second loss, determine the model parameters of the target model based on the target loss, and obtain a trajectory prediction and planning model.

[0076] This application provides a method, apparatus, medium, and device for training trajectory prediction and planning models. The trajectory prediction and planning model training method includes: acquiring multiple sets of training data grouped chronologically; the training data includes vehicle state data; when the current training data is the first set of training data in the multiple sets, inputting the current training data into a target model to obtain a first output result, and calculating a first loss based on the first output result; when the current training data is not the first set of training data, generating target training data based on the previous set of input data and the previous set of output results of the target model, inputting the target training data into the target model to obtain a second output result, and calculating a second loss based on the second output result; determining a target loss based on the first loss and the second loss, and determining the model parameters of the target model based on the target loss to obtain a trajectory prediction and planning model. This application employs a two-stage training approach. During the training phase using the first set of training data, open-loop learning is performed based on the vehicle's real-state data to obtain a stable initial model. Subsequently, a fine-tuning phase is initiated, using the vehicle's real-state data (i.e., the previous set of input data) and the self-predicted trajectory of the target model (i.e., the previous set of output results) to generate target training data. Based on the target training data, a second output result and a second loss are obtained. Then, the target loss is determined based on the first loss and the second loss. By introducing a feedback loop of the model's self-generated trajectory, the problem of error accumulation is mitigated, and the stability and reliability of the trajectory prediction and planning model are improved.

[0077] In one possible implementation, the training data further includes environmental feature data of the vehicle; inputting the current training data into the target model includes: adjusting the environmental feature data and the state data, and inputting the adjusted data into the target model; the adjustment includes coordinate system transformation and time alignment processing.

[0078] In one possible implementation, generating target training data based on the previous set of input data and the previous set of output results of the target model includes: concatenating the first data in the previous set of input data with the second data in the previous set of output results according to timestamps; transforming the concatenated data to obtain the target training data; the transformation includes one or more of the following operations: coordinate system transformation, data deletion, and data addition.

[0079] In one possible implementation, determining the target loss based on the first loss and the second loss includes: taking any one or more combinations of the first loss and the second loss as the target loss.

[0080] In one possible implementation, the target loss is a combination of any one or more of the first loss and the second loss, including: summing the first loss and multiple second losses to obtain the target loss.

[0081] In one possible implementation, the state data includes: trajectory data of the current vehicle and trajectory data of a specified vehicle; the environmental feature data includes one or more of map data, vehicle category data, and obstacle data.

[0082] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide a trajectory prediction and planning device. For example... Figure 6 The diagram shown is a structural block diagram of a trajectory prediction and planning device according to an embodiment of this application. The device may include:

[0083] Data module 601 is used to acquire vehicle driving data; the driving data includes vehicle status data and environmental feature data; prediction and planning module 602 is used to input the driving data into a trajectory prediction and planning model to obtain trajectory prediction results and trajectory planning results; wherein, the trajectory prediction and planning model is trained according to the above method.

[0084] The trajectory prediction and planning model in this application adopts a two-stage training phase. When training with the first set of training data, open-loop learning is performed based on the vehicle's real state data to obtain a stable initial model. Then, in the fine-tuning phase, the vehicle's real state data, i.e., the previous set of input data, and the self-predicted trajectory of the target model, i.e., the previous set of output results, are used to generate target training data. Based on the target training data, the second output result and the second loss are obtained. Then, the target loss is determined based on the first loss and the second loss. By introducing a feedback loop of the model's self-generated trajectory, the problem of error accumulation is alleviated, and the stability and reliability of the trajectory prediction and planning model are improved.

[0085] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0086] Figure 7 This is a block diagram of an electronic device used to implement embodiments of this application. Figure 7 As shown, the electronic device includes a memory 701 and a processor 702. The memory 701 stores a computer program that can run on the processor 702. When the processor 702 executes the computer program, it implements the method described in the above embodiments. The number of memories 701 and processors 702 can be one or more.

[0087] The electronic device also includes:

[0088] The communication interface 703 is used to communicate with external devices and perform data exchange and transmission.

[0089] If the memory 701, processor 702, and communication interface 703 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0090] Optionally, in a specific implementation, if the memory 701, processor 702, and communication interface 703 are integrated on a single chip, then the memory 701, processor 702, and communication interface 703 can communicate with each other through an internal interface.

[0091] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0092] This application provides a computer program product, wherein the computer program product includes a computer program, which, when executed by a processor, implements the method provided in this application embodiment.

[0093] This application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0094] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0095] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0096] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0097] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0098] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0099] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0100] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0101] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0102] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0103] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0104] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0105] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training a trajectory prediction and planning model, characterized in that, include: Obtain multiple sets of training data grouped in chronological order; The training data includes vehicle status data and vehicle environmental characteristic data; The status data includes: the trajectory data of the current vehicle and the trajectory data of the specified vehicle; the environmental feature data includes: one or more of the following: map data, vehicle category data, and obstacle data; When the current training data is the first set of training data among the multiple sets of training data, the current training data is input into the target model to obtain the first output result, and the first loss is calculated based on the first output result; When the current training data is not the first set of training data, target training data is generated based on the previous set of input data and the previous set of output results of the target model. The target training data is then input into the target model to obtain a second output result. A second loss is calculated based on the second output result. The previous set of input data refers to a set of feature data that is immediately before the current training data in the time series of the training process. The target loss is determined based on the first loss and the second loss, and the model parameters of the target model are determined based on the target loss to obtain the trajectory prediction and planning model.

2. The method according to claim 1, characterized in that, Inputting the current training data into the target model includes: The environmental feature data and the state data are adjusted, and the adjusted data is input into the target model; the adjustment includes coordinate system transformation and time alignment processing.

3. The method according to claim 1, characterized in that, Target training data is generated based on the previous set of input data and the previous set of output results of the target model, including: The first data in the previous set of input data is concatenated with the second data in the previous set of output results according to the timestamp. The concatenated data is transformed to obtain the target training data; the transformation includes one or more of the following operations: coordinate system transformation, data deletion, and data addition.

4. The method according to claim 1, characterized in that, Determining the target loss based on the first loss and the second loss includes: The target loss is a combination of one or more of the first loss and the second loss.

5. The method according to claim 4, characterized in that, The target loss is a combination of one or more of the first loss and the second loss, including: The target loss is obtained by summing the first loss and multiple second losses.

6. A trajectory prediction and planning method, characterized in that, include: Acquire vehicle driving data; the driving data includes vehicle status data and environmental characteristic data; The driving data is input into the trajectory prediction and planning model to obtain trajectory prediction results and trajectory planning results; wherein the trajectory prediction and planning model is trained by the method according to any one of claims 1-5.

7. A trajectory prediction and planning model training device, characterized in that, include: The acquisition module is used to acquire multiple sets of training data grouped in chronological order. The training data includes vehicle status data and vehicle environmental characteristic data; The status data includes: the trajectory data of the current vehicle and the trajectory data of the specified vehicle; the environmental feature data includes: one or more of the following: map data, vehicle category data, and obstacle data; The first calculation module is used to input the current training data into the target model when the current training data is the first set of training data in the plurality of training data, to obtain a first output result, and to calculate a first loss based on the first output result; The second calculation module is used to generate target training data based on the previous set of input data and the previous set of output results of the target model when the current training data is not the first set of training data, input the target training data into the target model to obtain the second output result, and calculate the second loss based on the second output result; the previous set of input data refers to a set of feature data that is immediately before the current training data in the time series of the training process. The loss module is used to determine the target loss based on the first loss and the second loss, and to determine the model parameters of the target model based on the target loss, so as to obtain the trajectory prediction and planning model.

8. A trajectory prediction and planning device, characterized in that, include: The data module is used to acquire vehicle driving data; the driving data includes vehicle status data and environmental characteristic data. The prediction and planning module is used to input the driving data into the trajectory prediction and planning model to obtain trajectory prediction results and trajectory planning results; wherein the trajectory prediction and planning model is trained by the method according to any one of claims 1-5.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor implements the method of any one of claims 1-6 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-6.

11. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images