System and method for supporting rapid iteration and quantitative deployment of trajectory prediction algorithm

Abstract

The invention discloses a system and method for supporting rapid iteration and quantitative deployment of a trajectory prediction algorithm, and the system comprises a data sample processing module which is used for integrating multi-source heterogeneous data, generating a dynamic sample library in a standard format, and generating a normalized preprocessing trajectory prediction sample set; the model training and quantification module is used for carrying out multi-stage training on the trajectory prediction model in the model library by using the preprocessed trajectory prediction sample set, and the multi-stage training comprises the steps of firstly carrying out floating point model training and then carrying out quantification perception training; and the model deployment and test module is used for carrying out hardware-in-the-loop test and automatic driving real vehicle test on the trajectory prediction model after multi-stage training so as to further iteratively optimize the trajectory prediction model according to a test result. The trajectory prediction model of the system and method disclosed by the invention shortens the iteration period of the automatic driving trajectory prediction algorithm, reduces the quantitative precision loss of the model, improves the real vehicle verification efficiency, and improves the algorithm development efficiency and deployment reliability.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a system and method for supporting rapid iteration and quantitative deployment of a trajectory prediction algorithm. Background Art

[0002] Trajectory prediction is a key technology in autonomous driving and is attracting increasing attention from both academia and industry. Currently, trajectory prediction relies primarily on deep learning techniques. These models analyze historical trajectory data, environmental information, and traffic flow data to predict the future behavior of traffic participants, such as vehicles and pedestrians. Improving trajectory prediction accuracy directly impacts key aspects of autonomous driving systems, such as decision-making and path planning, and is fundamental to ensuring safe and stable operation.

[0003] However, the development and iteration of existing trajectory prediction algorithms and their deployment on real vehicles face challenges such as difficulty integrating heterogeneous data from multiple sources, lengthy algorithm iteration cycles, severe precision loss in model quantization and deployment, a single evaluation dimension, and an imperfect verification system for real vehicle deployment models. Specifically, there are significant differences in format, coordinate system, and annotation standards between public datasets and real vehicle data, making them difficult to use together for training and the dynamic expansion of sample libraries. Traditional training processes use a single model of "supervised training-real vehicle tuning," failing to design training and testing methods specifically for real vehicle deployment goals. Performance testing of deployed models relies heavily on offline simulation testing and real vehicle road testing, lacking a standardized testing mechanism that combines hardware-in-the-loop (HIL) testing with real vehicle closed-loop testing, resulting in high testing time and hardware costs. These issues severely restrict the rapid iteration, high-precision deployment, and all-round performance testing of trajectory prediction algorithms, necessitating an urgent need for a full-link solution covering data processing, model training and verification, quantization and compression, and deployment testing. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides a system and method that supports rapid iteration and quantitative deployment of trajectory prediction algorithms, aiming to improve the efficiency of constructing, training, evaluating, and deploying trajectory prediction models in autonomous driving systems.

[0005] The technical solution adopted in the present invention is: Provides a system that supports rapid iteration and quantitative deployment of trajectory prediction algorithms, including: The data sample processing module is used to integrate multi-source heterogeneous data, build a dynamically expandable sample library through a standardized format conversion mechanism, and generate a standardized pre-processed trajectory prediction sample set. The sample library includes public datasets and real-vehicle datasets collected in stages. The model training and quantization module is used to perform multi-stage training on the pre-built autonomous vehicle trajectory prediction model using the pre-processed trajectory prediction sample set. This includes first performing floating-point model training and then performing quantization-aware training. The quantization-aware training includes quantizing and calibrating the parameters in the trajectory prediction model after floating-point model training, inserting pseudo-quantization nodes into the trajectory prediction model to generate a pseudo-quantized model, fine-tuning the parameters of the pseudo-quantized model, and finally converting it to a fixed-point model. The model deployment and testing module is used to perform hardware-in-the-loop testing and autonomous driving real-vehicle testing on the trajectory prediction model after multi-stage training, so as to further optimize the iterative trajectory prediction model based on the test results.

[0006] Following the above technical solution, floating-point model training is specifically as follows: the vehicle trajectory encoder and map encoder in the trajectory prediction model are pre-trained through a self-supervised learning mechanism, followed by supervised training on a public dataset, and finally transfer learning using a real vehicle dataset to train the floating-point parameters in the trajectory prediction model that are adapted to the real vehicle data domain.

[0007] Following the above technical solution, the hardware-in-the-loop test specifically uses the embedded twin environment to pre-verify the model inference latency and resource occupancy; the autonomous driving real-vehicle test specifically uses the on-board computing platform and the wire-controlled actuator to realize the closed-loop scenario adaptability test of the trajectory prediction model. The road scene data collected during the real-vehicle test will be fed back to the data sample processing module to expand the sample library.

[0008] Following the above technical solution, the multi-source heterogeneous data includes public road data and self-collected real vehicle data. The data sample processing module generates a public data set by summarizing and integrating the public road data, and integrates the real vehicle data collected in stages in real scenes by on-board sensor equipment into a real vehicle data set.

[0009] Following the above technical solution, the on-board sensor equipment is used to perceive the surrounding environment information and publish the original observation data in real time, mainly including cameras, lidar, inertial measurement unit and wheel speed meter.

[0010] Following the above technical solution, the model training and quantization module also includes performance evaluation, which is used to perform performance evaluation in stages during floating-point model training and quantization-aware training, monitor the model's training effect and convergence during the process, and provide intuitive evaluation and optimization direction of the model's prediction ability in complex scenarios.

[0011] Following the above technical solution, during the hardware-in-the-loop test, a data re-injection program is run to obtain and replay newly collected perception result data from the real vehicle data set, including motor vehicle and non-motor vehicle perception results, vehicle status information, and road structure information; the perception result data is pre-processed on the pre-built twin computing platform, and the trajectory of the pre-processed data is predicted through the multi-stage trained trajectory prediction model, and the trajectory prediction results are post-processed and the trajectory prediction result data is published.

[0012] Following the above technical solution, during the hardware-in-the-loop test, a model performance evaluation program is also run to evaluate the model prediction performance of the prediction results and perception result data. This includes quantitative performance evaluation and qualitative performance evaluation. Among them, the quantitative performance evaluation is calculated based on three indicators: trajectory prediction accuracy, trajectory prediction time domain, and trajectory prediction delay. The qualitative performance evaluation uses specific rendering tools to visualize the real-time perception results and prediction results.

[0013] The present invention also provides a method for rapid iteration and quantitative deployment of an autonomous vehicle trajectory prediction algorithm, comprising the following steps: Integrate multi-source heterogeneous data, build a dynamically expandable sample library through a standardized format conversion mechanism, and generate a standardized pre-processed trajectory prediction sample set; the sample library includes public datasets and real-vehicle datasets collected in stages; A pre-built autonomous vehicle trajectory prediction model is trained in multiple stages using a pre-processed trajectory prediction sample set. This includes first performing floating-point model training, followed by quantization-aware training. Quantization-aware training involves quantizing and calibrating the parameters in the trajectory prediction model after floating-point model training, inserting pseudo-quantization nodes into the trajectory prediction model to generate a pseudo-quantized model, fine-tuning the parameters of the pseudo-quantized model, and finally converting it to a fixed-point model. The trajectory prediction model after multi-stage training is subjected to hardware-in-the-loop testing and autonomous driving real vehicle testing to further optimize the iterative trajectory prediction model based on the test results.

[0014] The present invention also provides a computer storage medium storing a computer program executable by a processor, wherein the computer program implements the method for supporting rapid iteration and quantitative deployment of trajectory prediction algorithms as described in the above technical solution.

[0015] The beneficial effects of the present invention are as follows: the present invention performs floating-point model training and quantization perception training on the trajectory prediction model of an autonomous driving vehicle, and converts the trajectory prediction model with floating-point weight accuracy into a fixed-point model through multi-stage training, wherein the quantization perception training includes three steps: quantization parameter calibration, pseudo-quantization fine-tuning and fixed-point model conversion, ensuring efficient conversion of the floating-point precision model to a low-ratio fixed-point model, while minimizing the accuracy loss in the quantization process, improving the efficiency of actual vehicle verification, greatly shortening the iteration cycle of the autonomous driving trajectory prediction algorithm, and significantly improving the algorithm development efficiency and deployment reliability.

[0016] Furthermore, self-supervised pre-training is performed through floating-point model training to provide initial weights for downstream public dataset supervised training and real-vehicle data transfer learning; and public dataset supervised training is performed to provide a trajectory prediction model with strong generalization ability for downstream real-vehicle data transfer learning; finally, real-vehicle data transfer learning is performed to make full use of the diverse driving scene data collected by the real vehicle. The performance offset caused by the distribution difference between the public dataset and the real-vehicle data can be reduced based on weight regularization technology, and the data resampling strategy based on sample balancing and diversified processing can be used to increase the sampling probability of long-tail scene samples, further improving the performance of the trajectory prediction model on the real-vehicle dataset.

[0017] Furthermore, the autonomous driving real-vehicle test specifically implements closed-loop scenario adaptability testing of the trajectory prediction model through the on-board computing platform and the wire-controlled actuator. The road scene data collected during the real-vehicle test will be fed back to the data sample processing module, which can be used to dynamically expand the sample library.

[0018] Furthermore, by adding performance evaluation to the model training and quantization modules, the model's training effect and convergence can be monitored during floating-point model training and quantization-aware training, providing intuitive evaluation and optimization direction of the model's prediction capabilities in complex scenarios, and ensuring the comparability and versatility of the model under standard benchmarks (unified data sets and evaluation indicators).

[0019] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 This is a system block diagram of an embodiment of the present invention that supports rapid iteration and quantitative deployment of trajectory prediction models; Figure 2 is a flow chart of a data sample processing method according to an embodiment of the present invention; Figure 3 is a flow chart of model training and quantization according to an embodiment of the present invention; Figure 4 This is a flow chart of an offline test based on data re-injection according to an embodiment of the present invention; Figure 5 This is a flowchart of an online test based on a real vehicle platform according to an embodiment of the present invention; Figure 6 This is a flow chart of a method for supporting rapid iteration and quantitative deployment of trajectory prediction algorithms according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0023] It should be noted that the illustrations provided in the embodiments of the present invention are only schematic illustrations of the basic concept of the present invention. Therefore, the drawings only show components related to the present invention and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0024] In the present invention, it should also be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer" and the like are used to indicate positions or locations based on those shown in the accompanying drawings. These terms are intended solely to facilitate the description of the present application and to simplify the description. They are not intended to indicate or imply that the devices or components referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present application. Furthermore, the terms "first" and "second" are used solely for descriptive and distinguishing purposes and should not be construed as indicating or implying relative importance.

[0025] In addition, it should be noted that the features of the various embodiments of the present invention may be combined or coupled in part or in whole, and, as will be appreciated by those skilled in the art, may interact and operate in different ways. Each embodiment may be implemented independently of one another or in an associated relationship.

[0026] Example 1 like Figure 1 As shown, the present invention supports a system for rapid iteration and quantitative deployment of trajectory prediction algorithms, including three main modules: a data sample processing module, a model training and quantization module, and a model deployment and testing module.

[0027] The data sample processing module is used to integrate multi-source heterogeneous data, build a dynamically expandable sample library through a standardized format conversion mechanism, and generate a standardized pre-processed trajectory prediction sample set. The sample library includes public datasets and real-vehicle datasets collected in stages. The model training and quantization module is used to perform multi-stage training on the pre-built autonomous vehicle trajectory prediction model using the pre-processed trajectory prediction sample set. This includes first performing floating-point model training and then performing quantization-aware training. The quantization-aware training includes quantizing and calibrating the parameters in the trajectory prediction model after floating-point model training, inserting pseudo-quantization nodes into the trajectory prediction model to generate a pseudo-quantized model, fine-tuning the parameters of the pseudo-quantized model, and finally converting it to a fixed-point model. The model deployment and testing module is used to perform hardware-in-the-loop testing and autonomous driving real-vehicle testing on the trajectory prediction model after multi-stage training, and further optimize the iterative trajectory prediction model based on the test results.

[0028] The data sample processing module integrates heterogeneous data from multiple sources and builds a dynamically scalable sample library through a standardized format conversion mechanism. This library is based on multiple public road datasets and combines them with real-world vehicle data collected in phases, ensuring dynamic expansion. Furthermore, the data sample processing module supports the generation of standardized pre-processed trajectory prediction samples for specific model input requirements, providing data support for subsequent training and evaluation.

[0029] Specifically, the data sample processing module includes a public road dataset, a self-collected real vehicle database, a standardized traffic flow data format conversion unit, a specific model sample generation unit and a training and testing sample database. The public road dataset is used to summarize and integrate multiple large-scale trajectory prediction open source datasets to provide a variety of publicly available road and traffic flow data for model training and testing. The self-collected real vehicle database is used to store the real vehicle data collected by the on-board sensor equipment of the autonomous driving real vehicle test platform in real scenes, including information such as the speed, position and lane lines of the vehicle itself and surrounding vehicles, and can be dynamically expanded according to the incremental data collected during the subsequent real vehicle test operation. The standardized traffic flow data format conversion unit is used to convert the raw data in the public road dataset and the self-collected real vehicle database into a unified format, so as to facilitate the model to call data from different sources for training and testing. The specific model sample generation unit is used to generate the sample data required for specific model training based on the standardized unified format data to meet the input requirements of different algorithms. The training and testing sample database is used to store the dataset processed by the specific model sample generation unit, and supports providing the required data to the model in the form of standard input or streaming input during the model training phase.

[0030] Among them, the model training and quantization module uses the training and verification samples from the data sample processing module to execute a multi-stage training strategy including self-supervised pre-training, supervised training, transfer learning and quantization-aware training on the pre-built trajectory prediction model. It also includes a comprehensive model performance evaluation unit combining qualitative and quantitative methods, and finally outputs a low-bit specific point model after parameter quantization for deployment and testing.

[0031] Specifically, the model training and quantization module consists of three parts: a floating-point model training unit, a quantization-aware training unit, and a performance evaluation unit. The floating-point model training unit is used to train floating-point models based on public data sets and real-vehicle data sets (the parameters in the pre-built trajectory prediction model are floating-point type, which can be called a floating-point model), and provide the downstream quantization-aware training unit with a model that has strong generalization performance in multiple scenarios. The quantization-aware training unit includes quantization parameter calibration, quantization-aware training, and fixed-point model conversion. Quantization parameter calibration is used to calibrate the floating-point model parameters and determine the quantization parameters through training on a small number of data sets. Quantization-aware training inserts pseudo-quantization nodes into the model, so that the model can perceive the impact of quantization during the training process, fine-tune the model parameters, and minimize the loss of accuracy when converting to a fixed-point model downstream. Fixed-point model conversion is used to convert a pseudo-quantization model into a fixed-point model. The performance evaluation unit includes a validation set indicator self-test program, a key scenario visualization program, and an official indicator test program for public datasets. It is used to monitor the training effect and convergence of the trajectory prediction model during floating-point model training and quantization-aware training, provide intuitive evaluation and optimization direction of the model's prediction ability in complex scenarios, and ensure the comparability and versatility of the model under standard benchmarks (unified datasets and evaluation indicators).

[0032] Furthermore, based on the floating-point model training unit provided by the present invention, a multi-stage training strategy is used to gradually improve the model's representational capabilities and generalization performance. The floating-point model training unit includes self-supervised pre-training, public dataset supervised training, and real-vehicle data transfer learning. Self-supervised pre-training uses multiple masking mechanisms and reconstruction strategies to pre-train the trajectory encoder and map encoder, effectively extracting the latent semantic features of the trajectory and map, and providing initial weights for downstream public dataset supervised training and real-vehicle data transfer learning. Public dataset supervised training uses a label-based supervised learning strategy on the public dataset, combined with data augmentation techniques (such as adding noise, random data loss, and multi-scene feature combination training strategies), to provide a trajectory prediction model with strong generalization capabilities for downstream real-vehicle data transfer learning. Real-vehicle data transfer learning, based on public dataset supervised training, fully utilizes the diverse driving scene data collected from real vehicles. It uses weight regularization techniques to reduce performance offsets caused by distribution differences between public datasets and real-vehicle data. A data resampling strategy based on sample balancing and diversified processing increases the sampling probability of long-tail scene samples, further improving the performance of the trajectory prediction model on real-vehicle datasets.

[0033] The model deployment and testing module, connected to the model training and quantization module, comprises a hardware-in-the-loop (HIL) test platform and an autonomous vehicle test platform. The HIL test platform uses an embedded twin environment to pre-verify model inference latency and resource usage, while the vehicle test platform utilizes an onboard computing platform and drive-by-wire actuators to conduct closed-loop scenario adaptability testing of the prediction model. Furthermore, road scene data collected during autonomous vehicle testing can be fed back to the data sample processing module to expand the trajectory prediction training sample library.

[0034] Specifically, the model deployment and testing module includes a hardware-in-the-loop (HIL) test platform and an autonomous driving real-vehicle test platform. The HIL test platform includes a data re-injection program, a trajectory prediction program, and a model performance evaluation program for offline testing based on the re-injected data. The data re-injection program runs on the host computer platform and utilizes specialized software tools to retrieve and replay newly collected perception data from a self-collected real-vehicle database. This includes perception data from motor vehicles and non-motor vehicles, vehicle status information, and road structure information. The trajectory prediction program runs on the embedded platform, forming a twin computing platform with the on-board computing platform. The trajectory prediction program subscribes to the perception data, pre-processes it, performs trajectory prediction model inference, post-processes the model inference results, and publishes the model prediction data. The model performance evaluation program runs on the host computer platform and receives the prediction results sent by the trajectory prediction program. It then combines the perception data with the model prediction performance evaluation, including both quantitative and qualitative performance evaluation. The quantitative performance evaluation evaluates the prediction model performance by calculating the results of three metrics: prediction accuracy, prediction time domain, and prediction latency. Qualitative performance evaluation uses specific rendering tools to visualize the real-time perception results and prediction results to evaluate the performance of the prediction model.

[0035] Specifically, the autonomous driving test platform is equipped with onboard sensing equipment and runs upstream perception programs, trajectory prediction programs, and downstream regulatory control programs. It also includes a drive-by-wire actuator for online testing based on the real-vehicle platform. The onboard sensing equipment, including cameras, lidar, inertial measurement units, and wheel speedometers, is used to perceive the surrounding environment and publish raw observation data in real time. The upstream perception program, running on the onboard computing platform, subscribes to the raw observation data published by the onboard sensing equipment in real time, performs perception model processing, and publishes model perception result data. The trajectory prediction program (which uses a trajectory prediction model to predict the trajectories of surrounding traffic participants in the driving scene of the autonomous vehicle) runs on the onboard computing platform, receives the perception result data published by the upstream perception program, performs data preprocessing, model inference, post-processes the inference results, and publishes the model prediction result data. The downstream regulatory control program, running on the onboard computing platform, receives the prediction result data published by the trajectory prediction program, processes the data with regulatory control algorithms, and publishes control result data. The drive-by-wire actuator receives the control results sent by the downstream regulatory control program and drives the vehicle.

[0036] It can be seen that this embodiment performs floating-point model training and quantization perception training on the trajectory prediction model of the autonomous driving vehicle, and converts the trajectory prediction model with floating-point weight accuracy into a fixed-point model through multi-stage training. The quantization perception training includes three steps: quantization parameter calibration, pseudo-quantization fine-tuning and fixed-point model conversion. It ensures the efficient conversion of the floating-point precision model to the low-resolution fixed-point model, while minimizing the accuracy loss in the quantization process, improving the efficiency of actual vehicle verification, greatly shortening the iteration cycle of the autonomous driving trajectory prediction algorithm, and significantly improving the algorithm development efficiency and deployment reliability.

[0037] Example 2 The method for rapid iteration and quantitative deployment of the autonomous vehicle trajectory prediction algorithm in this embodiment is based on the above system, such as Figure 6 As shown, the method mainly includes the following steps: Step 1. Data Collection and Processing: By integrating multi-source heterogeneous data, a dynamically expandable sample library is constructed through a standardized format conversion mechanism, and a standardized pre-processed trajectory prediction sample set is generated; the sample library includes public datasets and real-vehicle datasets collected in stages.

[0038] Step 2: Multi-stage training of the trajectory prediction model: The pre-built autonomous vehicle trajectory prediction model is trained in multiple stages using the pre-processed trajectory prediction sample set. This involves first training the model with a floating-point model, followed by quantization-aware training. Quantization-aware training involves quantizing and calibrating the parameters in the trajectory prediction model after floating-point model training. Pseudo-quantization nodes are inserted into the trajectory prediction model to generate a pseudo-quantized model. The parameters of the pseudo-quantized model are then fine-tuned, and finally converted to a fixed-point model. This step ensures efficient conversion from a floating-point precision model to a low-resolution fixed-point model while minimizing accuracy loss during the quantization process.

[0039] Step 3. Model deployment and testing: Conduct hardware-in-the-loop testing and autonomous driving vehicle testing on the trajectory prediction model after multi-stage training, and further optimize the iterative trajectory prediction model based on the test results.

[0040] In Step 1, a dynamic and scalable traffic segment standard database is constructed by collecting heterogeneous data from multiple sources in real time, including public road datasets and in-house real-world vehicle data. This data is converted to a unified format using a standardized format conversion mechanism. Sample preprocessing and dynamic updating involve performing model-specific training and validation sample preprocessing on the traffic segment standard database, including scene information extraction, trajectory information extraction, and map feature extraction. Furthermore, a data loading sampler specific to the training method is developed.

[0041] Step 2 can also include using the validation set self-test program, key scenario visualization program and public data set official indicator test program to conduct a comprehensive qualitative and quantitative evaluation of the model, monitor the model's training effect in real time, evaluate its performance in complex scenarios, and ensure that the model has strong generalization.

[0042] In Step 3, an offline re-injection test is performed on a hardware-in-the-loop test platform to verify the model's reasoning capabilities on the embedded platform. This is followed by an online reasoning test on an autonomous driving vehicle test platform to verify the model's accuracy and real-time performance in actual driving scenarios.

[0043] The test results can be used to further optimize and iterate the trajectory prediction model algorithm. Specifically, the model structure, training strategy, quantization parameters, etc. can be optimized based on the feedback from the model test results. It is also ensured that the trajectory prediction model runs stably on different hardware platforms, thereby accelerating the iteration and deployment of the algorithm.

[0044] Example 3 This embodiment is based on Example 1. By building a complete closed-loop algorithm development loop, this embodiment achieves full-process optimization of the autonomous driving trajectory prediction model from data collection to deployment. First, in the data collection and processing stage, the system obtains multi-source heterogeneous data from public road datasets (such as Argoverse and nuScenes) and self-collected real-vehicle databases. The system converts the raw data into a standardized representation containing timestamps, coordinate systems, traffic participant attributes, and lane topology relationships through a standardized traffic flow data format conversion unit, thereby constructing a standard database of traffic segments that supports dynamic expansion. In the sample preprocessing and dynamic update stage, scene information extraction, trajectory information extraction, and map feature extraction operations are performed based on the standardized database to generate standardized samples containing historical trajectory sequences, high-precision map features, and future trajectory labels. At the same time, a sampler that supports streaming data loading is developed to achieve dynamic fusion of real-time incremental data during training.

[0045] The floating-point model training phase employs a three-stage progressive strategy: First, a trajectory encoder and a map encoder (i.e., the encoding portion of the trajectory prediction model) are constructed through self-supervised pre-training. Common feature representations are extracted using trajectory mask reconstruction and map contrast learning tasks. Subsequently, supervised training is performed on public datasets, optimizing prediction accuracy using a multimodal loss function. Finally, the pre-trained model is adapted to the real-world vehicle data domain through transfer learning, employing domain adaptation techniques to mitigate data distribution discrepancies. During the fixed-point model quantization phase, a quantization parameter calibration module analyzes the model weight distribution characteristics and determines the optimal quantization scaling factors for each layer. During quantization-aware training, pseudo-quantization operators are inserted to simulate low-precision computations. A progressive fine-tuning strategy is employed to balance model accuracy and inference efficiency, ultimately generating an 8-bit fixed-point model suitable for embedded platforms. During the training evaluation phase, a validation set self-test program is run to calculate the ADE / FDE metrics. A key scene visualization program is used to perform 3D trajectory rendering analysis in complex scenes such as intersections and dense traffic. Regular official benchmarks are conducted on public dataset benchmarks to ensure that model performance meets industry benchmarks.

[0046] During the model deployment and testing phase, a hardware-in-the-loop test platform was used to build an embedded twin environment. A data re-injection program replicated the perception data streams collected by the real vehicle to verify the fixed-point model's latency and memory usage on the target hardware. During the real-vehicle testing phase, an onboard computing platform performed online inference, feeding the prediction results into the regulatory control algorithm to generate control instructions. A closed-loop iterative mechanism was established, culminating in scenario data collection, model optimization, and deployment verification. Finally, based on the quantitative evaluation report output by the test platform, targeted optimizations were made to the model structure, training hyperparameters, and quantization strategies. An automated pipeline was then used to rapidly verify and deploy the new model version, forming a sustainable and evolving algorithm iteration system.

[0047] Figure 2The specific process of the data sample processing module of the system used in the present invention is demonstrated. Specifically, the data to be processed consists of two parts, namely, a public road dataset and a self-collected real vehicle database. The public road dataset includes large-scale autonomous driving datasets such as nuScenes, nuPlan, and Argoverse, which are used to provide a large amount of sample data for model training; the data in the self-collected real vehicle dataset is divided into existing real vehicle collected data and newly added real vehicle collected data, wherein the existing real vehicle collected data is the real driving scene data collected and stored in historical real vehicle experiments; the newly added real vehicle collected data is the incremental data obtained during the latest real vehicle test operation. The original scene data is obtained in real time by the on-board sensor equipment such as cameras and lidars carried by the autonomous driving real vehicle test platform during operation, and is preliminarily processed and optimized through data preprocessing, and organized into map data and intelligent body data. The map data mainly includes attributes and information related to map elements such as lane lines and crosswalks; the intelligent body data includes information such as the physical properties, trajectory, speed and direction of the vehicle and surrounding traffic participants.

[0048] Based on the self-collected real-world vehicle database and the integrated public road dataset, both are processed through a standardized traffic flow data format conversion unit. The format conversion process is related to the dataset type, and a specific dataset format conversion program processes various large-scale autonomous driving datasets in the public road dataset and the self-collected real-world vehicle database separately. Specifically, the format conversion program first reads the scene data from the specified original dataset file path and returns a scene list and scene data instances based on the set number of working threads. Subsequently, during the scene data conversion phase, scene information extraction, trajectory information extraction, and map feature extraction are performed in sequence. Scene information extraction primarily extracts all relevant frame data from a given scene, including the identifiers, location information, type, and speed of all objects in each frame, and returns this frame data along with its corresponding scene information. Trajectory information extraction further processes the trajectory data of all objects based on the extracted scene frame data, removing invalid data, filling in missing frames, and interpolating position, speed, and orientation to obtain complete trajectory data. Map feature extraction primarily extracts specific map element information from the corresponding map data based on the scene information, such as drivable lane boundaries, crosswalk boundaries, lane boundaries, and lane identifiers. After scene data conversion, the resulting unified format data is saved as a binary file and stored in a specified path. Furthermore, the system aggregates all processed scene data and generates an index file, recording metadata such as each scene's identifier and timestamp, as well as the storage path of the processed scene data, to facilitate rapid retrieval and access during the model training phase.

[0049] After converting the raw data into a unified format, the model-specific sample generation unit executes different data processing scripts based on the model type. Based on the formatted data, it generates cache files for different model training and validation. First, the scene data is loaded from a specified path. Subsequently, during the specific model-specific data processing phase, three operations are performed: map data processing, agent data processing, and sample feature extraction. Map data processing performs equidistant interpolation and segmentation on scene map elements, such as lane lines and crosswalks, and transforms their coordinates to a coordinate system centered on the target agent. Agent data processing selects the target agent based on a conditional mask and performs coordinate transformations on its historical and future trajectories. The processed agent data also includes information such as a trajectory validity mask and agent index. Sample feature extraction evaluates individual sample data by applying a Kalman filter to the historical trajectory and comparing it with the actual future trajectory. The accuracy of the Kalman filter prediction is then evaluated as the Kalman difficulty of the sample. Furthermore, this step classifies the agent trajectory based on information such as velocity, position, and heading angle at the start and end points of the trajectory, labeling the sample's trajectory type.

[0050] Data processed by a specific model sample generation unit is stored in a training and testing sample database and divided into training, validation, or test sets based on pre-set functional requirements. Data file names contain the scene name, frame index, and agent name, so the database supports both standard input formats and streaming input. Specifically, the model loads relevant data for the target agent in the same scene in chronological order as input, enabling stability constraints to be applied using historical prediction results, ensuring that predictions remain largely consistent across the same scene.

[0051] Figure 3 The details of the model training and quantization modules of the proposed system are shown in this paper. First, in the self-supervised pre-training stage, a masking strategy for trajectory data is required to randomly mask the data of certain time steps, so that the trajectory sequence forms an incomplete historical input in the time dimension. The trajectory encoder completes these missing time steps through time inference. Its loss function is actually

[0052] in, represents the masked trajectory sequence, Completion trajectories generated for the model.

[0053] For the map masking strategy, in order to enable the map encoder to better capture local structural characteristics, some key features in the map (such as lane centerlines, intersections, etc.) are randomly masked and the encoder is required to reconstruct these masked features. The actual loss function is

[0054] in, Represents the local features of the obscured map points, The reconstructed map point features of the model.

[0055] By masking the trajectory points and map point features, the model learns to infer missing information from the context and overall features, thereby obtaining an encoder that has a holistic understanding of historical trajectories and maps. The overall loss function of its self-supervised pre-training is

[0056] Next, a supervised training phase is conducted on a large public dataset. First, the weights of the trajectory encoder and map encoder generated during the supervised pre-training phase are loaded. These weights have been pre-trained using various masking mechanisms and reconstruction strategies, effectively extracting the latent semantic features of the trajectory and map. On this basis, a trajectory prediction network is constructed, in which the outputs of the trajectory encoder and map encoder are passed through a feature fusion module to generate global context features. Subsequently, a prediction head based on the Laplace distribution is used to model the probability distribution of future trajectories. Supervised training is performed using the following Laplace distribution negative log-likelihood (NLL) loss function:

[0057] in, is the true trajectory point, and are the mean and variance of the model predictions, is the sample size.

[0058] During supervised training, to improve the model's generalization capabilities, we first standardized public datasets such as nuScenes, nuPlan, and Argoverse into the data format required by the model. Gaussian noise was added to the trajectory points to simulate actual sensor errors. The formula is as follows:

[0059] in, The mean is 0 and the variance is A Gaussian distribution is used. Furthermore, some time steps in the trajectory data are randomly deleted to enhance the model's robustness to sensor frame loss. Finally, we adjust the sampling probability for rare long-tail scenarios in driving data (such as turns and overtaking) to ensure diversity and balance in the training data.

[0060] The real-world data transfer learning module fine-tunes the weights of the trajectory encoder and map encoder generated by the supervised training module using a public dataset. The weights of the trajectory encoder and map encoder generated by the supervised training module using a public dataset are first loaded as the initial parameters for fine-tuning. The trajectory points in the real-world data are filtered, including noise filtering (such as Kalman filtering). Based on the processed real-world trajectory data, the trajectory encoder and map encoder are fine-tuned. To prevent catastrophic forgetting during transfer learning, a weight regularization strategy is introduced. This strategy constrains changes in model weights during fine-tuning, ensuring that previously learned useful features are retained during updates while guiding the model to better adapt to new data.

[0061] The quantization parameter calibration step determines the quantization range (including scaling factors and zero points) of the parameters that need to be quantized in the model through training on a small number of data sets. By training with a small number of samples, the quantization parameter range is adapted to the data characteristics of the current task, ensuring that the quantized model can still maintain a high level of accuracy. The goal of calibration is to minimize the error before and after quantization.

[0062] Quantization-aware training (QAT) inserts pseudo-quantization nodes during training, enabling the model to "perceive" the impact of quantization on model accuracy. Inserting pseudo-quantization nodes: Pseudo-quantization nodes are inserted into various layers of the model (such as convolutional layers and fully connected layers). Training is performed using the model with pseudo-quantization nodes. During training, the backpropagation algorithm adjusts model weights and biases to ensure that the model maintains high accuracy in a quantized environment. QAT considers the impact of quantization on the model during training, ensuring that the trained model is more adaptable to quantized hardware deployments.

[0063] The fixed-point model conversion step converts the quantization-aware trained floating-point model into a fixed-point model for efficient operation on embedded hardware or other resource-constrained environments. Quantized model conversion: A quantization algorithm is used to convert the QAT-trained floating-point model into a fixed-point model. Finally, the converted fixed-point model is deployed on the target hardware platform for inference calculations.

[0064] The performance evaluation unit is designed to comprehensively monitor model training results and convergence during floating-point and quantization-aware training, providing guidance for further optimization. This unit consists of three main modules: a validation set metric self-test program, a key scenario visualization program, and a public dataset official metric testing program. By working together, these three programs provide detailed feedback on model training and optimization, ensuring excellent performance in complex application scenarios.

[0065] The validation set metrics self-test program is designed to monitor the model's performance on the validation set in real time during training, assessing the model's training effectiveness and convergence. The key scenario visualization program provides intuitive evaluation of model predictions in complex scenarios. The official public dataset metric testing program ensures the comparability and generalizability of the model under standard benchmarks (such as public datasets and evaluation metrics).

[0066] This paper provides a system and method for supporting rapid iteration and quantitative deployment of trajectory prediction algorithms, enabling offline testing of trajectory prediction models within a hardware-in-the-loop test platform. By offline testing of re-injected data, the accuracy and stability of the prediction model are evaluated, thereby promoting iterative optimization of the trajectory prediction algorithm. Figure 4 The three core modules of the hardware-in-the-loop test platform were demonstrated: first, a data re-injection program, which replays collected open-road driving data to simulate the perception process of the autonomous driving system; second, a trajectory prediction program, which performs model inference based on the re-injected data to predict the trajectories of traffic participants; and finally, a model performance evaluation program, which calculates and evaluates the performance of the trajectory prediction model, including qualitative and quantitative analysis. These programs work together to verify and optimize the trajectory prediction model in the autonomous driving system.

[0067] The specific hardware-in-the-loop testing steps include: (1) Open road driving and real vehicle data collection and storage. To fully evaluate the performance of the trajectory prediction model, the collected traffic scenes include pedestrians, vehicles, cyclists (including motorcycles and bicycles) and other traffic participants, and cover various types of traffic scenes such as intersections and two-way straight roads. After the safety officer drives the vehicle to a domestic open road that meets the above scenario requirements, the recording function of the specific software on the vehicle control platform is activated to collect more than 1000 seconds of perception and positioning data, and the perception data collected on the open road is saved as a data package.

[0068] (2) Real vehicle data playback. The collected data packets are played back using specific software pre-installed on the laptop computer and the perception packet data is published. The laptop computer is connected to the embedded platform via an Ethernet cable. The embedded platform is accessed using the Xshell tool. The program for subscribing to and parsing the perception packet data is cross-compiled and the executable file is transferred to the embedded platform. The subscription and parsing scripts are then run to implement subscription, parsing, and publishing of Protobuf-based perception data.

[0069] (3) Run the prediction model. First, complete the generation of the fixed-point model, cross-compile the board-side inference program, and write the configuration file. Transfer the executable file and its configuration file to the embedded platform. Run the inference script of the prediction model, receive the perception data, perform the prediction model inference, and save the perception and prediction results in real time.

[0070] (4) Sample processing and test sample construction. The saved serialized perception data is processed into predicted true value data, and then processed together with the prediction results into test samples. The total number of samples of the non-motor vehicle trajectory prediction model in the test samples shall be no less than 1000, of which no less than 500 are pedestrian samples and no less than 500 are cyclist samples.

[0071] (5) Check the calculated indicators. Complete the writing of the test indicator program in advance and calculate the prediction time domain and prediction accuracy indicators of non-motor vehicles respectively.

[0072] The accuracy of the non-motor vehicle lateral trajectory prediction is evaluated by the average lateral distance error (ADE-x). The specific calculation formula is:

[0073] Where N is the number of samples, T is the total prediction step size, represents the longitudinal position coordinate of the predicted trajectory in the i-th sample at the t-th time, and is the longitudinal true position coordinate of the i-th sample at the t-th time.

[0074] The average distance error (ADE) is used to evaluate the average distance difference between the predicted and actual trajectories of non-motor vehicles, thus measuring the accuracy of the trajectory prediction algorithm. The trajectory point error is calculated using the Euclidean distance.

[0075]

[0076] Where N is the number of samples, T is the total prediction step size, and represents the position of the predicted trajectory in the i-th sample at the t-th moment. is the true position of the i-th sample at the t-th moment.

[0077] Figure 5The overall framework of the autonomous driving real-vehicle test platform was demonstrated. The platform consists of several key components, including onboard sensing equipment, upstream perception programs, trajectory prediction programs, downstream regulation and control programs, and drive-by-wire actuators. These components work together to complete online testing of trajectory prediction models. Specifically, the following steps are involved: (1) Onboard sensor equipment is responsible for sensing the surrounding environment and publishing raw observation data in real time. This includes cameras, lidar, and integrated navigation systems. These sensors work together to collect environmental data in real time and publish the raw observation data to upstream perception programs through the onboard computing platform.

[0078] (2) Upstream perception program. The upstream perception program runs on the vehicle computing platform and is responsible for processing data from the vehicle's sensor devices and generating the output of the perception model. It subscribes to the raw observation data in real time, processes the raw data through various perception algorithms, detects and tracks dynamic and static targets, and publishes the processed perception data in a standard format (such as proto format) for use by downstream trajectory prediction programs.

[0079] (3) Trajectory prediction program. The trajectory prediction program runs on the vehicle computing platform, receives the perception result data from the upstream perception program, and performs trajectory prediction. The main processes include: data preprocessing, coordinate transformation, data normalization, time alignment, etc. of the received perception data; model inference, based on the preprocessed historical trajectory and map data, running the deployed trajectory prediction model to predict the future trajectory of traffic participants; inference result post-processing, post-processing the prediction results output by the model, including trajectory smoothing, credibility estimation, etc., to ensure the accuracy and stability of the prediction results; publishing the model prediction result data, and publishing the predicted trajectory results to the downstream regulatory control program.

[0080] (4) Downstream control program. The downstream control program is responsible for processing the control algorithm based on the trajectory prediction results to generate appropriate control instructions. These control instructions will then be sent to the wire control actuator to guide the operation of the vehicle. Its main functions include: Receiving prediction result data: receiving the prediction result data from the trajectory prediction program, including the predicted trajectory, position, speed and other information of the target object. Control algorithm processing: based on the prediction results, executing the control algorithm to generate a suitable driving trajectory or control command for the vehicle.

[0081] (5) Wire actuator. The wire actuator is responsible for receiving the control results issued by the downstream control program and driving various vehicle operations through a precise execution mechanism. Specific functions include: Receiving control result data: receiving control commands issued by the downstream control program, such as acceleration, steering angle, braking force, etc. Executing control commands: achieving dynamic control of the vehicle through the electronic control system (such as throttle, brake, steering and other control modules).

[0082] The platform can quickly verify the accuracy and real-time performance of trajectory prediction algorithms, providing strong support for performance improvement and iteration of autonomous driving systems. At the same time, through closed-loop control, it ensures the safety and stability of autonomous driving systems.

[0083] This embodiment achieves a systematic breakthrough through collaborative innovation of various modules. The beneficial effects are mainly reflected in the following dimensions: First, efficient compatibility and closed-loop updating of multi-source heterogeneous data. To address the poor model generalization issues inherent in existing technologies, which stem from single data sources and heterogeneous formats, this invention establishes a unified data representation framework through a standardized traffic flow data format conversion unit. This unit converts multi-source heterogeneous data collected from public datasets such as Argoverse and nuScenes, as well as real-world vehicles, into a standardized format that includes coordinate systems, lane topology, and kinematic parameters of traffic participants. This mechanism improves cross-dataset training efficiency, forming a closed-loop ecosystem of "data acquisition-model training-scenario feedback."

[0084] Secondly, the efficiency and accuracy of model training and quantization are both improved. Traditional methods have drawbacks during the model training phase, such as a high risk of overfitting and poor hardware adaptability. This invention supports a multi-stage training strategy consisting of self-supervised pre-training, supervised training, transfer learning, and quantization-aware training. It utilizes advanced model training techniques, such as masked trajectory modeling in the self-supervised pre-training phase and a dynamic curriculum learning mechanism in the supervised training phase. This significantly improves the convergence speed of both floating-point and fixed-point models, and enhances the robustness and generalization capabilities of the models.

[0085] Finally, the reliability of the model's real-vehicle deployment testing is systematically improved through a dual verification mechanism of hardware-in-the-loop testing and real-vehicle testing, which systematically improves the reliability and scenario adaptability of model deployment. Based on an embedded twin environment, hardware-in-the-loop testing simulates real-world hardware resource constraints in the laboratory phase, proactively identifying potential deployment risks such as abnormal model inference latency and excessive memory usage, significantly reducing debugging costs during the real-vehicle testing phase. Real-vehicle testing, through closed-loop linkage between the on-board computing platform and the drive-by-wire actuator, verifies the synergy between the prediction model and the regulatory control system in dynamic traffic scenarios, ensuring that the model's output trajectory complies with vehicle dynamics constraints and actual road rules.

[0086] Example 4 This application also provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, an optical disk, a server, an App store, etc., storing a computer program that implements corresponding functions when executed by a processor. When executed by a processor, the computer-readable storage medium of this embodiment implements the method of supporting a trajectory prediction model for an autonomous vehicle according to the method embodiment.

[0087] It should be pointed out that, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, or two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0088] The size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0089] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A system that supports rapid iteration and quantitative deployment of trajectory prediction algorithms, characterized by: include: The data sample processing module is used to integrate multi-source heterogeneous data, build a dynamically expandable sample library through a standardized format conversion mechanism, and generate a standardized pre-processed trajectory prediction sample set; The sample library includes public datasets and real-car datasets collected in stages; The model training and quantization module is used to perform multi-stage training on the pre-built autonomous vehicle trajectory prediction model using the pre-processed trajectory prediction sample set. This includes first performing floating-point model training and then performing quantization-aware training. The quantization-aware training includes quantizing and calibrating the parameters in the trajectory prediction model after floating-point model training, inserting pseudo-quantization nodes into the trajectory prediction model to generate a pseudo-quantized model, fine-tuning the parameters of the pseudo-quantized model, and finally converting it to a fixed-point model. The model deployment and testing module is used to perform hardware-in-the-loop testing and autonomous driving real-vehicle testing on the trajectory prediction model after multi-stage training, so as to further iterate and optimize the trajectory prediction model based on the test results.

2. The system for supporting rapid iteration and quantitative deployment of trajectory prediction algorithms according to claim 1, characterized in that: in, The specific training of the floating-point model is as follows: the vehicle trajectory encoder and map encoder in the trajectory prediction model are pre-trained through a self-supervised learning mechanism, followed by supervised training on a public dataset, and finally transfer learning using a real vehicle dataset to train the floating-point parameters in the trajectory prediction model that are adapted to the real vehicle data domain.

3. The system for supporting rapid iteration and quantitative deployment of trajectory prediction algorithms according to claim 1, characterized in that: Hardware-in-the-loop testing specifically uses an embedded twin environment to pre-verify model inference latency and resource occupancy; autonomous driving real-vehicle testing specifically uses the on-board computing platform and wire-controlled actuators to implement closed-loop scenario adaptability testing of the trajectory prediction model. The road scene data collected during the real-vehicle testing process will be fed back to the data sample processing module for dynamic expansion of the sample library.

4. The system for supporting rapid iteration and quantitative deployment of trajectory prediction algorithms according to claim 1, characterized in that: Multi-source heterogeneous data includes public road data and self-collected real vehicle data. The data sample processing module generates a public data set by summarizing and integrating the public road data, and integrates the real vehicle data collected in stages in real scenes by on-board sensor equipment into a real vehicle data set.

5. The system for supporting rapid iteration and quantitative deployment of trajectory prediction algorithms according to claim 4, characterized in that: On-board sensor equipment is used to perceive the surrounding environment and publish raw observation data in real time. It mainly includes cameras, lidars, inertial measurement units and wheel speed meters.

6. The system for supporting rapid iteration and quantitative deployment of trajectory prediction algorithms according to claim 1, characterized in that: The model training and quantization module also includes performance evaluation, which is used to conduct performance evaluation in stages during floating-point model training and quantization-aware training, monitor the model's training effect and convergence during the process, and provide intuitive evaluation and optimization direction of the model's prediction ability in complex scenarios.

7. The system for supporting rapid iteration and quantitative deployment of trajectory prediction algorithms according to claim 1, characterized in that: During hardware-in-the-loop testing, a data re-injection program is run to obtain and replay newly collected perception result data from the real vehicle data set, including motor vehicle and non-motor vehicle perception results, vehicle status information, and road structure information; the perception result data is pre-processed on a pre-built twin computing platform, and the trajectory of the pre-processed data is predicted through a multi-stage trained trajectory prediction model. The trajectory prediction results are then post-processed and the trajectory prediction result data is published.

8. The system for supporting rapid iteration and quantitative deployment of trajectory prediction algorithms according to claim 7, characterized in that: During hardware-in-the-loop testing, a model performance evaluation program is also run to evaluate the model prediction performance of the prediction results and perception result data. This includes quantitative performance evaluation and qualitative performance evaluation. The quantitative performance evaluation is calculated based on three indicators: trajectory prediction accuracy, trajectory prediction time domain, and trajectory prediction delay. The qualitative performance evaluation uses specific rendering tools to visualize the real-time perception results and prediction results.

9. A method for rapid iteration and quantitative deployment of an autonomous vehicle trajectory prediction algorithm, characterized in that: The following steps are involved: Integrate multi-source heterogeneous data, build a dynamically expandable sample library through a standardized format conversion mechanism, and generate a standardized pre-processed trajectory prediction sample set; The sample library includes public datasets and real-car datasets collected in stages; A pre-built autonomous vehicle trajectory prediction model is trained in multiple stages using a pre-processed trajectory prediction sample set. This includes first performing floating-point model training, followed by quantization-aware training. Quantization-aware training involves quantizing and calibrating the parameters in the trajectory prediction model after floating-point model training, inserting pseudo-quantization nodes into the trajectory prediction model to generate a pseudo-quantized model, fine-tuning the parameters of the pseudo-quantized model, and finally converting it to a fixed-point model. The trajectory prediction model after multi-stage training is subjected to hardware-in-the-loop testing and autonomous driving real vehicle testing to further optimize the trajectory prediction model based on the test results.

10. A computer storage medium, characterized in that A computer program executable by a processor is stored therein, and the computer program executes the method for supporting rapid iteration and quantitative deployment of trajectory prediction algorithms as described in claim 9.

Images