A vehicle trajectory prediction and monitoring method based on discriminator implicit future interaction learning guidance

Abstract

This invention relates to the field of trajectory prediction in autonomous driving, and particularly to a vehicle trajectory prediction and monitoring method based on discriminator implicit future interaction learning guidance. The method includes the following steps: Step 1: Designing a training framework for a cGAN-based trajectory prediction model; Step 2: Designing a sub-task network PRNet for evaluating future interactions between vehicles; Step 3: Alternating training of the generator G and discriminator D based on a dynamic weight strategy; Step 4: Lightening and optimizing the deployment of the trained trajectory prediction generator model G through quantized perception training; Step 5: Deploying the prediction model obtained in Step 4 on an in-vehicle embedded computing platform to achieve real-time prediction and inference, and visualizing the results on both the actual vehicle and regulatory ends, achieving advantages in trajectory accuracy and social compliance.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving trajectory prediction, and in particular to a method for predicting and monitoring vehicle trajectories based on discriminator implicit future interaction learning guidance. Background Technology

[0002] In real-world autonomous driving systems, the trajectory prediction module typically operates as an online functional unit on the vehicle. It needs to stably complete feature construction, model inference, and result output at preset intervals, provided that upstream perception data is continuously received. The prediction results are then provided to the downstream planning and control modules in a usable format. The performance of the trajectory prediction module in predicting the trajectories of future interacting vehicles relies on excellent trajectory prediction models and algorithms. Autonomous vehicle trajectory prediction not only needs to consider historical interaction relationships but also ensure that vehicle interactions within the prediction period are reasonable. That is, the model needs to consider future vehicle interactions, including oncoming traffic maneuvering and yielding, to avoid predicted trajectories from intersecting or overlapping, making the predicted multi-vehicle trajectories more socially compliant.

[0003] Based on the future interaction modeling method and trajectory output method, future interaction methods are divided into the following three types: iterative feedback future interaction method based on trajectory correction method, future interaction method based on synchronous prediction, and conditional prediction future interaction method based on dependency judgment.

[0004] Models based on trajectory correction methods typically first predict the future trajectories or intentions of multiple agents separately, then feed the initially predicted trajectories or intentions into a correction module. For each agent, the potential future actions of other agents on its own trajectory can be explicitly considered, and its own trajectory is corrected accordingly. Although this method provides a future interaction game approach through an iterative feedback mechanism and is interpretable, it suffers from poor real-time performance due to the need for multiple forward propagation updates during inference. This makes it difficult to run stably at fixed intervals on in-vehicle systems and hinders continuous online verification in real-world vehicle systems.

[0005] Simultaneous prediction-based future interaction methods predict the trajectory of the joint distribution of all multi-agents at once. These methods predict the joint distribution of multi-agents by directly modeling the joint probability distribution of future states or by extracting features of future interactions through attention mechanisms. This improves the consistency and rationality of the prediction results in terms of physical and social ethics, but it suffers from high computational complexity. For scenarios involving multiple agents, the dimension of the joint distribution increases exponentially with the number of agents, leading to a surge in training and inference costs. Secondly, its training relies on a large amount of high-quality interaction scenario data, and its generalization ability to unseen interaction patterns is limited. Finally, compared to the aforementioned interaction trajectory correction methods, black-box-based simultaneous prediction methods have lower interpretability, making it difficult to provide explicit explanations of interaction strategies, which is detrimental to debugging and verification on the vehicle side.

[0006] Dependency-based conditional prediction methods explicitly define the relationships between influencers and responders by constructing a marginal-conditional framework. This transforms the interaction logic into a traceable, structured relationship, overcoming the limitations of implicit modeling and making the model's decision-making process more consistent with human cognitive logic of traffic scenarios. Regarding computational complexity, on the one hand, conditional prediction decomposes the combinatorial explosion problem of joint prediction by breaking it down into multiple chained subproblems through a hierarchical prediction strategy, reducing computational complexity to linear or polynomial levels and effectively improving scalability for large-scale scenarios. On the other hand, conditional prediction requires explicit relationship prediction modules and hierarchical prediction, resulting in high computational costs and making it unsuitable for scenarios with high real-time requirements. Furthermore, obtaining relationship labels during training relies on manual annotation, which is costly on large-scale datasets. Additionally, the chain propagation of marginal prediction errors can lead to error accumulation, causing excessive sensitivity of conditional prediction and ultimately excessive errors, thus affecting the stable output and verifiability of the real-vehicle system. Summary of the Invention

[0007] To address the aforementioned problems, this invention proposes a vehicle trajectory prediction and monitoring method based on discriminator-guided implicit future interaction learning. On one hand, the method designs a high-real-time trajectory prediction algorithm to model future interactions, reducing computational complexity and time overhead. Specifically, this is achieved by employing a GAN architecture on a server platform, where the generator... Used for trajectory prediction, discriminator Through a multi-task design, the generator simultaneously performs trajectory authenticity judgment and future interaction relationship learning during adversarial training. In the discriminator Guided by this, trajectories that better conform to traffic interaction patterns can be generated. On the other hand, during the "monitoring" process of the onboard system, the algorithm of this invention does not require the operation of a discriminator in the inference phase. Therefore, it fundamentally avoids the complex overhead of calculating future interaction probabilities, effectively solving the problem of excessive inference latency. Furthermore, the predicted trajectory results obtained through inference are output at preset intervals and encapsulated in a standard communication format for distribution to downstream planning and control modules. Simultaneously, the predicted trajectory, vehicle status, and target status are visualized on the vehicle display terminal and remote monitoring system to support online verification, debugging, and backtracking analysis during actual vehicle operation. In the actual vehicle system, the trajectory prediction module stably outputs prediction results at preset intervals under the condition of continuous access to upstream sensing data, and provides them to the downstream planning and control modules in a usable form. The results are visualized at both the actual vehicle and regulatory ends, achieving advantages in trajectory accuracy and social compliance.

[0008] Technical solution

[0009] A method for predicting and monitoring vehicle trajectories based on discriminator-guided implicit future interaction learning includes the following steps:

[0010] Step 1: Design the training framework for the cGAN-based trajectory prediction model;

[0011] The cGAN (Conditional Generative Adversarial Network) framework includes a generator G and a discriminator D. The trajectory prediction model is obtained through adversarial training between the generator G and the discriminator D.

[0012] The generator G is used for trajectory prediction. It takes the vehicle's historical trajectory features and map features as conditional information y, and inputs them together with a noise vector sampled from a random noise distribution to generate a predicted future trajectory z.

[0013] The discriminator D is responsible for supervising and evaluating the generation effect of the generator G. It adopts a multi-task design and simultaneously completes trajectory authenticity judgment and future interaction relationship learning: the predicted future trajectory z, condition information y, and the real future trajectory x are input into the discriminator D, which judges their authenticity; the future interaction relationship sub-task network PRNet predicts the future interaction relationship between vehicles.

[0014] The total loss function of the discriminator D considers not only the loss d_loss from adversarial training, but also the loss pr_loss from evaluating future interaction relationships.

[0015] Step 2: Design a sub-task network PRNet to evaluate future interactions between vehicles, and calculate the loss for evaluating future interactions;

[0016] To evaluate future interaction relationships in the discriminator, a subtask network PRNet is designed to evaluate future interaction relationships between vehicles. The interaction probability rr of the real future trajectories between vehicles, calculated based on road topology rules, is used as the label. By minimizing the difference between the predicted future relationship value pr and the label rr, the network is guided to focus on the future interaction relationships between vehicles.

[0017] Step 3: Train the generator G and discriminator D alternately based on a dynamic weight strategy;

[0018] Step 4: Model quantization;

[0019] The trained trajectory prediction generator model G is lightweighted and optimized for deployment through quantized perception training.

[0020] Step 5: Deployment of actual vehicles and visualization of predicted trajectories;

[0021] The prediction model obtained after step 4 quantization Deployed in the vehicle-embedded computing platform of the in-vehicle device to achieve real-time predictive inference and visualize the results in the actual vehicle and at the regulatory end.

[0022] Furthermore, in step 1,

[0023] The optimization goals of the generator G and the discriminator D are opposite. The goal of D is to correctly distinguish between the real future trajectory and the predicted future trajectory, that is, to judge the predicted future trajectory as false and the real future trajectory as true, while the goal of G is to make the discriminator judge the predicted future trajectory as true.

[0024] The overall objectives of adversarial training are as follows:

[0025]

[0026] in, This represents the value function in a generative adversarial network. Indicates when the input sample From real data distribution At that time, the discriminator The expected value of the logarithmic probability of correctly judging it (i.e., judging it as true); while This indicates that when the input sample is generated by the generator From noise prior During generation, the discriminator The expected value of the logarithmic probability of correctly judging it (i.e., judging it as false); for the discriminator The goal of the generator is to maximize the expected probability of making the correct judgment, that is, to maximize the sum of the two aforementioned factors; while the goal of the generator G is exactly the opposite, that is, to minimize the sum of the two aforementioned factors.

[0027] In adversarial training, G and D are trained alternately to reach a Nash equilibrium. Due to the characteristics of adversarial generative networks, the generated predicted future trajectories are close to the real future trajectories and have higher diversity, making them suitable for multi-target trajectory prediction in complex scenarios. In order to save inference time, the discriminator D is used only in training, while in actual applications only the generator G is used for trajectory prediction.

[0028] Furthermore, step 1 includes:

[0029] Step 1.1 Design the basic structure of generator G;

[0030] The generator G extracts and fuses features from the input historical trajectory and map information, using them as the generator's conditional information y. At the same time, it samples various types of noise from a Gaussian distribution. The conditional information y and the noise are then input into the prediction network, and the final output is the predicted future trajectory z.

[0031] The specific process is as follows:

[0032] Feature extraction and fusion: In the feature extraction stage, convolutional networks or self-attention networks are used to extract trajectory features and map features respectively; in the feature fusion stage, convolutional networks or cross-attention networks are used to fuse the trajectory and map features in the temporal and spatial dimensions.

[0033] Sampling and processing Gaussian noise: Randomly generate a vector of a specific length, merge it with the fused trajectory and map features to form a new feature vector, which serves as the input to the trajectory generation module.

[0034] Generate predicted trajectories: First, an attention mechanism is used to process the merged feature vectors, and then a prediction network is used to output the predicted trajectories for all vehicles. Due to the uncertainty in trajectory prediction, the trajectory generation module adopts multimodal prediction, that is, predicting K different trajectories for the same vehicle simultaneously to cover all possibilities. Therefore, the generator G contains K prediction networks with the same structure but different parameters to generate K modes of multi-vehicle trajectory combinations.

[0035] Step 1.2 Design the loss function for generator G;

[0036] To avoid mode collapse, the generator's loss function combines the regression loss `reg_loss` from comparing generated and real trajectories, the classification loss `cls_loss` from multimodal trajectory prediction, and the loss `g_loss` generated during adversarial training. The loss function for `g` is:

[0037]

[0038] Among them, the regression loss reg_loss calculates the error between the generated trajectory and the true trajectory in multimodal trajectory prediction, and the formula is:

[0039]

[0040] Where N represents the total number of vehicles in a scene, K represents the K different multimodal trajectories generated, and T represents the prediction time (i.e., the total number of frames). Let represent the coordinates of the k-th predicted trajectory of the i-th vehicle at time t. Let represent the coordinates of the actual trajectory of the i-th vehicle at time t.

[0041] The classification loss cls_loss is used to optimize the generator's confidence assessment of multiple trajectories, so that the trajector that is closest to the real trajectory among the generated K modalities is assigned the highest confidence score, thereby improving the accuracy of trajectory ranking.

[0042] For classification loss, first calculate the loss for each vehicle. Predicted trajectory in each mode k With the actual trajectory average distance :

[0043]

[0044] Then, the mode with the smallest average distance is selected as the optimal mode:

[0045]

[0046] Finally, the classification loss is defined as the negative log-likelihood mean of the best modal confidence across all vehicles:

[0047]

[0048] in, This represents the confidence score given by generator G for the predicted trajectory of the i-th vehicle in the k-th mode. This score is obtained by mapping the trajectory features through an MLP. The confidence score of the trajectory selected as the best mode.

[0049] g_loss is the loss of G generated during adversarial training, and its calculation formula is:

[0050]

[0051] Step 1.3 Design the basic structure of discriminator D;

[0052] The discriminator D is used to score the reasonableness of the generator G's output and evaluate future interaction relationships. Its inputs are conditional information y (including historical trajectories), the true future trajectory x, and the predicted future trajectory output by generator G. .

[0053] First, feature extraction and feature fusion are performed on the input. Then, the features are fed into the neural network for scoring and evaluation of future interaction relationships. The final output is a trajectory true / false score between 0 and 1, and an interaction probability matrix between each pair of vehicles.

[0054] Specifically as follows:

[0055] Feature extraction and fusion: First, the historical trajectory is compared with the predicted future trajectory. Alternatively, the historical trajectory and the actual trajectory x are spliced ​​together in the time dimension, and features are extracted from the spliced ​​complete trajectory and map, followed by feature fusion.

[0056] True / False Scoring: The fused features are processed using a neural network, and then a fully connected layer is used to output a score of 0 to 1. The closer the score is to 1, the more real the discriminator considers the trajectory to be.

[0057] Future interaction relationship evaluation: The features of multiple vehicles are concatenated in pairs, and the concatenated features are processed by a neural network. The output of a fully connected layer is 0 to 1 to represent the predicted probability of interaction in the future. The closer the value is to 1, the more likely the interaction is to occur.

[0058] Step 1.4 Design the loss function for discriminator D;

[0059] For the discriminator D, in addition to the adversarial training loss d_loss, a loss pr_loss for evaluating future interaction relationships also needs to be added, resulting in a total loss function. Defined as follows:

[0060]

[0061] Where d_loss is the loss of the discriminator D during adversarial training, and its calculation formula is:

[0062]

[0063] pr_loss is the error between the predicted future interaction probability and the interaction probability label, which is calculated by the PRNet sub-task network for future interaction relationships (see step 2 for details).

[0064] Furthermore, step 2 includes:

[0065] Step 2.1 Design the network structure for evaluating the subtask of future interaction relationships;

[0066] The Future Interaction Relationship Evaluation Network (PRNet) predicts future interaction relationships between vehicles. Assuming there are N vehicles, its input consists of the historical trajectories, predicted trajectories, or actual trajectories of multiple vehicles in the same scene, as well as the fusion features of the map. The output is a matrix pr evaluating the interaction relationships between vehicles in future time periods.

[0067] The network processing procedure is as follows: Vehicle features are merged pairwise into a single feature. ,feature Characteristics of vehicle i Features of vehicle j The concatenated features are processed sequentially through a network consisting of multiple linear fully connected layers and their activation layers. Finally, a score is output through the output layer. The outputs of all vehicle combinations are arranged into an N×N matrix. Finally, each row of the matrix is ​​normalized to obtain the probability matrix pred_proximity, or pr, which is between 0 and 1.

[0068] Step 2.2 Calculate the ground truth of future interaction relationships as training labels;

[0069] The labels for future interaction relationships will be generated using rules based on road topology relationships:

[0070] Assuming there are N vehicles and M road centerlines, first calculate the initial occupancy matrix for each vehicle at the road center point based on the Euclidean distance from the vehicle to the road center. 1 indicates occupied, 0 indicates not occupied;

[0071] Then, to further model the vehicle's potential movement intentions, the initial road occupancy rate of the vehicle is determined based on the topological relationships between lanes (predecessor, successor, left neighbor, and friendly neighbor) and the prior probability of the vehicle appearing in adjacent lanes. The occupancy rate is propagated to its adjacent lanes in a weighted manner to obtain a smooth occupancy rate. (Values ​​between 0 and 1), the sum of vehicle occupancy rates across all roads is 1, where the smoothed occupancy rate represents the probability that a vehicle will appear at the centerline position of each road in the next moment; the smoothed occupancy rate matrix... Multiplying by itself yields the symmetric interaction probability matrix between vehicles. ;

[0072] Finally, normalization is performed to obtain the probability of a vehicle interacting with all other vehicles from its perspective, which is more consistent with the actual situation.

[0073] The specific calculation process is shown in the following formula:

[0074]

[0075]

[0076]

[0077] in, The coordinate matrix representing the vehicle. The coordinate matrix representing the centerline of the road. Represents the distance function. Indicates the distance threshold. This represents a topological relation matrix, where topological relations are categorized into predecessor (pre), successor (suc), left neighbor (left), and right neighbor (right). Pre and successor relations are further divided into direct adjacent nodes, second-order adjacent nodes, and so on up to fifth-order adjacent nodes. For each topological relation, an M×M 0 / 1 matrix is ​​used. This is expressed as an expression, multiplied by their respective influence coefficients. The subsequent superposition involves iterative optimization of the influence coefficients during training through backpropagation. The results are obtained through weighted summation of the topological relationships between roads. As shown in the following formula:

[0078]

[0079] Among them, the influence coefficient This represents the weight of a certain adjacent topological relationship in the relationship calculation. The magnitude of this coefficient depends on the probability that a vehicle will appear in that adjacent topological segment at the next time step. , respectively representing the road that appears before, the road that appears after, the road that appears on the left, and the road that appears on the right.

[0080] Generally speaking, the probability of a vehicle appearing on a subsequent road (suc) is much greater than the probability of it appearing on a previous road (pre). Generally more It is much larger. The specific coefficients are derived from prior statistics or training of the dataset.

[0081] Step 2.3 Calculate the loss for evaluating future vehicle interaction relationships;

[0082] The error pr_loss between the future interaction probability pred_proximity and the interaction probability label rule_proximity is calculated. By reducing this error, PRNet is trained to capture potential future interaction relationships between vehicles. Furthermore, through gradient backpropagation during adversarial training, the generator implicitly learns future interaction relationships between vehicles during feature extraction and feature fusion. The pr_loss design is as follows:

[0083]

[0084] Furthermore, step 3 includes:

[0085] Step 3.1 Training data processing;

[0086] Before training, the input map data and historical trajectory data need to be vectorized, and the true value of the probability of future interactions between vehicles needs to be calculated using a rule-based method.

[0087] Specifically, in the vectorization of vehicle historical data, the coordinates of all vehicles are first converted into a relative coordinate system with the coordinates of the target vehicle in the last frame of the historical time as the origin and the driving direction as the x-axis. Then, the increment of the x and y coordinates of each vehicle in each frame in the relative coordinate system is calculated. Finally, the time needs to be aligned. For vehicles observed in a certain frame, the effective position is 1, while for those not observed, the coordinates are set to (0,0) and the effective position is 0.

[0088] In the vectorization of map data, it is also necessary to convert the coordinate system of the road centerline into a relative coordinate system with the coordinates of the target vehicle in the last frame of history as the origin and the driving direction as the x-axis. Then, the displacement of each centerline coordinate point relative to the previous one is calculated, and a road is represented by a vector of relative coordinates plus displacement.

[0089] To calculate the true value of the future interaction probability of vehicles, a rule-based method is used: First, calculate the distance between the coordinates of each vehicle in the last frame of the future and the center line of each road, and use this to calculate the occupancy rate of each vehicle on each road. Then, smooth the occupancy rate matrix and multiply it by itself. After setting the diagonal to zero, normalize it to obtain the interaction probability matrix between vehicles, and use it as the true value.

[0090] Step 3.2 Fix the parameters of G and train the discriminator D;

[0091] Step 3.2.1 Generator G predicts future trajectories

[0092] First, in a generator G with fixed parameters, a self-attention network or convolutional network is used to extract historical trajectory features and map features of all vehicles. These features are then fused using methods such as cross-attention, and the fused features serve as conditional information y. Subsequently, the conditional information y is concatenated with a random noise vector and input into the prediction network of G, which outputs a predicted future trajectory. ;

[0093] Step 3.2.2 Input fake samples into the discriminator

[0094] Processing to predict future trajectories To piece it together with the historical trajectory to form a complete trajectory. (Fake sample);

[0095] Complete trajectory The conditional information y is input into the discriminator D to obtain D's discrimination against false samples. The score is obtained, and D is used to evaluate the future interaction relationship between vehicles. ;

[0096] Step 3.2.3 Input real samples into the discriminator

[0097] Input the actual future trajectory x, and combine it with the historical trajectory to form a complete trajectory. (True samples); representing the real future trajectory The conditional information y is input into the discriminator D to obtain D pairs of true samples. The rating, and the evaluation matrix for the probability of future interactions. ;

[0098] Step 3.2.4 Calculation of the loss of discriminator D

[0099] The goal of the discriminator D is to better distinguish between real and fake samples. Therefore, the true value of fake samples is set to 0 and the true value of real samples is set to 1. The average error between the scores of real and fake samples output by D and their respective true values ​​is calculated as the loss d_loss of the ScoreNet real and fake scoring module.

[0100] The average error between pr1, pr2 and the true value is directly calculated and used as the loss pr_loss of the future interaction relationship evaluation module;

[0101] Step 3.2.5 The DWA (Dynamic Weight Averaging) algorithm adjusts the loss weights of D and updates the parameters;

[0102] For the first batch of training data, d_loss and pr_loss are directly added together to obtain the loss D; starting from the second batch of data, the rate of change of d_loss and pr_loss recorded in this batch and the previous batch are calculated each time, and the weights are reversibly assigned according to the magnitude of the rate of change, and the weighted losses are added together to obtain the loss D.

[0103] After obtaining the loss of D, backpropagate to update the parameters of D;

[0104] Step 3.3 Fix the parameters of D and train the generator G;

[0105] Step 3.3.1 Generator G predicts future trajectories

[0106] The process is the same as step 3.2.1;

[0107] Step 3.3.2 Input fake samples into the discriminator

[0108] The process is the same as step 3.2.2;

[0109] Step 3.3.3 Calculation of the loss of generator G

[0110] The goal of generator G is the opposite of that of discriminator D, which is to make D identify fake samples as real. Therefore, the true value of fake samples is set to 1, and the average error between the score of fake samples output by D and the true value is calculated as the loss g_loss of generator G in adversarial training.

[0111] To avoid mode collapse and maintain the accuracy of the predicted trajectory, it is also necessary to calculate the regression loss reg_loss and classification loss cls_loss between the multimodal trajectory predicted by the generator G and the actual future trajectory.

[0112] Step 3.3.4 The DWA algorithm adjusts the loss weights of G and updates the parameters.

[0113] For the first batch of training data, g_loss, reg_loss and cls_loss are directly added together to obtain the loss G; starting from the second batch of data, the rate of change of the loss of g_loss, reg_loss and cls_loss recorded in the current batch and the previous batch are calculated each time, and the weights are reversed according to the magnitude of the rate of change, and the weighted losses are added together to obtain the loss G.

[0114] After obtaining the loss of G, backpropagation is used to update the parameters of G;

[0115] Step 3.4 Alternate training until the losses of G and D converge;

[0116] The Dynamic Weighted Average (DWA) algorithm calculates the loss for each subtask during training on each batch of data. It then calculates the rate of change of error based on the losses of each subtask during the previous backpropagation, and reassigns weights according to the magnitude of this rate of change. This dynamic balance of weights across the subtasks achieves collaborative optimization. The weight update formula for the DWA algorithm is shown below:

[0117]

[0118]

[0119]

[0120] in, Indicates task The ratio of the loss at time t-1 to the loss at time t-2 reflects the convergence speed of the task's loss. This represents the loss weight that can be assigned to task i at time t, which is determined by the weights at time t-1. The index of other tasks The sum and the exponent are related. I represents the number of tasks, and T is the temperature coefficient. When T is sufficiently large, The weights of each task are basically equal.

[0121] Furthermore, step 4 includes:

[0122] Step 4.1 Obtain the floating-point model;

[0123] Step 4.1.1 Obtain the optimal set of weight parameters from the generator G that has reached convergence in the adversarial training in Step 3, and load this set of parameters into a pre-defined prototype network structure that is isomorphic to the generator G to obtain the original floating-point model with complete logical structure and parameter weights.

[0124] Step 4.1.2: Perform structural modifications on the floating-point model:

[0125] The input layer of the original floating-point model is encapsulated using the QuantStub module to construct a quantization interface for converting input features from floating-point numbers to fixed-point numbers. The output layer of the original floating-point model is encapsulated using the DeQuantStub module to construct a dequantization interface for converting output results from fixed-point numbers to floating-point numbers, thereby obtaining the modified floating-point model.

[0126] Step 4.2 Quantize and calibrate the model;

[0127] Step 4.2.1 Utilize the Observer module to dynamically monitor the activation values ​​and weights of each layer of the modified floating-point model during the forward inference process; by traversing the calibration dataset, use the Observer module to extract the numerical boundaries of the data streams of each layer as the statistical basis for calculating the scaling factor and zero point. The numerical boundaries include the minimum value, maximum value, and distribution pattern.

[0128] Step 4.2.2 Use the calibration dataset to perform multiple forward inferences on the floating-point model, accumulate the observed data distribution, calculate the optimal quantization parameters for each layer, and establish the mapping relationship between floating-point data and fixed-point data;

[0129] Step 4.2.3 Validate the results of the calibrated floating-point model with quantization parameters on preset evaluation metrics, including minADE, minFDE, and MR:

[0130] Step 4.2.3.1 If the verification results meet the preset requirements, then based on the calibration results, convert the calibrated floating-point model with quantization parameters into a fixed-point model. And used for subsequent inference deployment;

[0131] Step 4.2.3.2 If the verification result does not meet the preset requirements, the calibration result is used as the initialization, and the process proceeds to step 4.3 to perform quantitative perception training to further optimize the inference accuracy after quantization.

[0132] Step 4.3 Quantize and train the model for perception;

[0133] Step 4.3.1 Enable pseudo-quantization nodes during the training phase to simulate the numerical behavior of fixed-point quantization and dequantization during forward inference, so that the model can take into account the numerical error caused by quantization during the training phase in Step 4.3.2; on this basis, continue to perform backpropagation and parameter update to enable the model parameters to be adaptively optimized to quantization error;

[0134] Step 4.3.2 During the quantization perception training process, the original trajectory prediction task objective remains unchanged, and the pseudo-quantization node is kept in the enabled state. The model weights are driven by error backpropagation to perform adaptive gradient compensation based on the numerical deviation generated by pseudo-quantization, so that the training process is consistent with the numerical characteristics of fixed-point inference.

[0135] After training is completed in step 4.3.3, the pseudo-quantized nodes are removed, and the quantization parameters and model weights are solidified to obtain the final fixed-point model for inference. Export the obtained fixed-point model as an hbm file for subsequent deployment on the actual vehicle board.

[0136] Furthermore, step 5 includes:

[0137] Step 5.1 Offline verification and consistency check;

[0138] In fixed-point model Before the vehicle is put into operation, the fixed-point model is first tested. Perform offline verification and consistency checks;

[0139] Step 5.2 Visualization of vehicle-mounted board operation and predicted trajectory;

[0140] By utilizing a multi-threaded asynchronous link organization, through two parallel logical links of "state maintenance" and "predictive inference", specifically, the dynamic maintenance tasks of data access and environmental state are executed by the state maintenance thread (hereinafter referred to as: thread A), and the trajectory prediction inference task is driven by the periodic triggering mechanism of the prediction inference thread (hereinafter referred to as: thread B), thereby realizing a complete closed loop from data access to result display.

[0141] Furthermore, step 5.2 includes:

[0142] Step 5.2.1 Data access and dynamic maintenance of environment status:

[0143] 5.2.1.1 Data Access:

[0144] The vehicle-mounted terminal uses thread A to receive structured environmental information output by the upstream sensing module in real time, including the vehicle's motion state information and the motion state information of surrounding vehicles, and then continuously writes the structured environmental information into the message buffer of the vehicle-mounted terminal.

[0145] 5.2.1.2 Dynamic maintenance of environmental status:

[0146] The target maintenance logic module on the vehicle end is used to parse the message buffer data, and the parsing results are updated and stored in the shared state library in real time to obtain global vehicle information. During parsing, the target identifier is used as an index, and the global vehicle information structure is formed by using a sliding window method, which includes the historical window sequence and the corresponding timestamp, and the dynamic maintenance of the environmental state is realized.

[0147] Step 5.2.2 Periodic Triggering and Data Snapshot Generation:

[0148] At the trigger moment, thread B uses a concurrent control method to perform a one-time read and copy of the data required for the current period from the shared state library, generating an independent data snapshot corresponding to the global vehicle information for the current period. This data snapshot includes the vehicle status and target information.

[0149] Step 5.2.3 Feature preprocessing and standardized input construction:

[0150] Thread B constructs a normalized tensor based on the data snapshot described in step 5.2.2, which serves as the fixed-point model in step 5.2.4. Input;

[0151] Step 5.2.4 Fixed-point model inference:

[0152] The normalized tensor input is fed into the fixed-point model using thread B. Perform inference and output the original prediction results; the original prediction results include multimodal trajectories in the future time domain and the confidence information corresponding to each trajectory;

[0153] Step 5.2.5 Post-processing of results and publication of standards:

[0154] Thread B uses post-processing logic combined with the target state information at the current moment to modify the fixed-point model. The output execution coordinates are restored, and the preferred predicted trajectory is extracted based on the confidence level filtering rules;

[0155] Subsequently, the preferred predicted trajectory is encapsulated into a standard communication format and published to the downstream vehicle planning and control module. The published content includes the predicted trajectory sequence, confidence level, and associated timestamp to ensure that the downstream planning and control module can achieve time-aligned decision consumption.

[0156] Step 5.2.6 Time Consumption Statistics and Prediction Results Visualization:

[0157] Thread B, through parallel time consumption statistics and trajectory visualization monitoring submodules, achieves real-time monitoring of the entire processing time and intuitive rendering of prediction results, realizing a fixed-point model. The closed-loop operation and interactive display on the vehicle side.

[0158] Furthermore, step 5.2.3, feature preprocessing and standardized input construction, specifically includes:

[0159] Step 5.2.3.1 State Alignment and Reference System Transformation:

[0160] For each target in the snapshot, its motion state within the historical window is retrieved in chronological order, and a state alignment process is performed using a spatial transformation operator. The alignment process is to uniformly transform the coordinate position and orientation angle of each target at different historical sampling times to a preset reference coordinate system with the current vehicle position as the origin.

[0161] Step 5.2.3.2 Feature Construction and Dimensional Regularization:

[0162] Feature engineering logic is used to construct features and map the motion states of each target after alignment in step 5.2.3.1. Specifically, by solving for the original physical quantities such as historical displacement, velocity, and orientation angle, feature vectors containing spatiotemporal evolution laws are generated, thereby enabling subsequent fixed-point models. It can capture the acceleration and deceleration trends and steering characteristics of target vehicles; at the same time, it performs dimension normalization processing on targets with inconsistent historical frame numbers, and performs zero-padding on missing frames to make the feature vectors of all targets length aligned on the time axis.

[0163] Step 5.2.3.3 Construction of Normalized Tensors:

[0164] Using a pre-allocated contiguous memory buffer, the feature vectors produced in step 5.2.3.2 are arranged and encapsulated according to the "target-time-feature" dimension; by mapping discrete features to contiguous memory space, a fixed-point model is directly constructed. The normalized tensors required for the input layer.

[0165] Furthermore, step 5.2.6 involves time consumption statistics and visualization of prediction results:

[0166] First, the performance of key nodes in each inference cycle is monitored using the time consumption statistics submodule. These key nodes include the entire process executed in thread B, which includes steps 5.2.2 (data snapshot generation), 5.2.3 (feature preprocessing and standardized input construction), 5.2.4 (fixed-point model inference), and 5.2.5 (result post-processing and standard release). By acquiring the computation time of each step in real time, the end-to-end processing logic is dynamically ensured to run stably within the preset scheduling cycle, thereby ensuring the timeliness of the prediction results.

[0167] Secondly, the trajectory visualization monitoring submodule receives the preferred predicted trajectory data released in step 5.2.5 in real time and performs dynamic rendering on the vehicle display terminal or remote monitoring system.

[0168] Beneficial effects

[0169] Compared with the prior art, the present invention has the following significant advantages:

[0170] (1) The present invention integrates the future interaction relationship evaluation module PRNet into the discriminator network. This module is used to predict the future interaction relationship between vehicles. The prediction of the future interaction relationship is introduced into the discriminator as a sub-task. By sharing model parameters, the discriminator is guided to pay attention to the potential interaction relationship between vehicles while scoring the rationality of the trajectory. This introduces the consideration of the social compliance of the trajectory into the evaluation process, thereby enabling the generator to generate more accurate future trajectories that conform to social compliance.

[0171] (2) Since the discriminator only participates in the training phase to assist the generator in optimization, it is not called in the inference phase. Therefore, although introducing the discriminator to calculate future interaction relationships during training will increase the computational overhead, the generator only needs to be used to generate the predicted trajectory during inference, without the participation of the discriminator, so there is no additional time cost. This solves the problem of excessive inference latency faced by existing trajectory prediction models when considering future interaction relationships.

[0172] (3) The trajectory prediction generator model G trained by the server is lightweighted and then deployed on the board as an efficient inference model.

[0173] (4) A highly reliable, low-latency on-board prediction pipeline was constructed. This invention effectively solves the resource contention problem between the asynchronous high-frequency access of upstream perception data and the periodic calculation of trajectory prediction through a dual-thread asynchronous architecture of "state maintenance" and "predictive inference" on the on-board end. By utilizing a data snapshot mechanism under concurrent control, the spatiotemporal consistency and reproducibility of the input data in each inference cycle are ensured. This mechanism not only improves the operational stability of the system on the embedded board, but also stably controls the single-frame inference latency within a preset period, ensuring the safe response of the actual vehicle operation.

[0174] (5) End-to-end closed-loop monitoring and efficient interaction of the prediction process are achieved. This invention integrates a high-performance fixed-point model with a visualization interaction module to achieve a complete technical closed loop from data access and feature extraction to trajectory display. The system supports dynamic rendering frequencies above 10Hz, and can synchronously present multimodal predicted trajectories with the real-time status of the vehicle, providing an intuitive online verification and backtracking analysis method for real vehicle operation, greatly enhancing the observability and interactivity of the autonomous driving system in complex traffic scenarios. Attached Figure Description

[0175] Figure 1 This is a schematic diagram of the training framework of the trajectory prediction model based on GAN in an embodiment of the present invention;

[0176] Figure 2 This is a design diagram of the discriminator network D in an embodiment of the present invention;

[0177] Figure 3 This is a visualization example of the smoothing coefficient matrix Topo in an embodiment of the present invention, (a) the original road image and (b) the smoothing coefficient diagram.

[0178] Figure 4 This is a visualization diagram of future interaction relationships in an embodiment of the present invention, wherein the thickness of the blue connecting lines represents the strength of the interaction;

[0179] Figure 5 This is an overall flowchart of the model quantization process in an embodiment of the present invention;

[0180] Figure 6 This is a flowchart illustrating the visualization of vehicle-mounted board operation and trajectory prediction in an embodiment of the present invention.

[0181] Figure 7 This is a visualization of the prediction results of the board-end quantization model in an embodiment of the present invention. Detailed Implementation

[0182] The technical solution provided in this application will be further described below with reference to specific embodiments. The advantages and features of this application will become clearer from the following description.

[0183] Example

[0184] The first training inference phase is deployed on a professional server and includes steps 1, 2, and 3.

[0185] LaneGCN was used as the baseline model and trained as a prototype for the generator G. Training and inference were performed on the Argoverse1 public dataset, predicting vehicle trajectories for the next 3 seconds (30 frames) based on the historical trajectories of vehicles over the past 2 seconds (20 frames). In the Argoverse1 dataset, vehicles are divided into three categories: "target vehicles (agents)" (the objects of prediction and evaluation), "own vehicles (av)" (the observation center), and "other vehicles" (the surrounding vehicles). To ensure fair comparison with benchmark methods, future trajectories of all vehicle types were predicted uniformly during training; however, in the final performance evaluation, only the prediction results for target vehicles (agents) were reported. Assuming there are N vehicles and M roads in the scene of the last frame of the historical trajectory, the future trajectories of N vehicles need to be predicted during training. The baseline model was trained and optimized. Experimental results show that the prediction accuracy of the baseline model can be effectively improved without significantly increasing the inference latency.

[0186] Step 1: Design the training framework for the cGAN-based trajectory prediction model (e.g., Figure 1 );

[0187] Step 1.1 Design the basic structure of generator G;

[0188] In this embodiment, the LaneGCN model is used as the prototype of generator G. The main components of generator G are: (1) a module for feature extraction and fusion, (2) a module for sampling and processing Gaussian noise, and (3) a module for generating predicted trajectories.

[0189] In the feature processing module, LaneGCN first abstracts historical trajectory information into vectors of shape N×20×3, where 20 represents the frame number and 3 contains the vehicle's coordinates (x, y) at the corresponding time and a validity indicator (1 indicates the vehicle was observed at that time, and the coordinates are valid; 0 indicates the vehicle was not observed at that time, and the coordinates are filled with zeros, indicating invalidity). After incremental calculation and other operations, a vector of shape N×19×128 is obtained. At the same time, map information is integrated into a vector of shape M×128, Nodes, representing roads, and a topological relationship matrix Graph containing four M×M 0 / 1 matrices. Then, a one-dimensional convolutional network with residual structure is used to extract trajectory features and map features respectively. Subsequently, feature fusion is performed using a cross-attention network, sequentially performing information transfer and feature fusion from trajectory to map (a2m), map to map (m2m), map to trajectory (m2a), and trajectory to trajectory (a2a), ultimately yielding an N×128 vector of features for N vehicles in the corresponding scene.

[0190] In the noise sampling, N vectors of a specific length are randomly sampled from the Gaussian distributed noise. Here, the length of the noise vector is set to 64. These vectors are then merged with the aforementioned feature vector of length N×128 to form a new feature vector of length N×192, which is used as the input to the trajectory generation module.

[0191] For the trajectory generation module, an attention mechanism is first used to process the merged feature vectors, and then a prediction network is used to output the predicted trajectories for all vehicles. Due to the uncertainty in trajectory prediction, most prediction models employ multimodal prediction, that is, predicting K different trajectories simultaneously for the same vehicle to cover all possibilities. Here, K=6, so the generator G contains 6 networks with the same structure but different parameters to generate 6 combinations of multi-vehicle trajectories in different modes. Assuming there are N vehicles in the last frame of this scene, the generator generates N×K trajectories, where each trajectory contains the vehicle coordinates for the next 3 seconds (i.e., 30 frames), meaning the generated vector shape is N×K×30×2.

[0192] Step 1.2 Design the loss function for generator G;

[0193] To avoid mode collapse, the generator's loss function combines the regression loss `reg_loss` (comparing to the true future trajectory), the classification loss `cls_loss` (from multimodal trajectory prediction), and the loss `g_loss` (from adversarial training). The loss function for `g` is:

[0194]

[0195]

[0196]

[0197] Where bz represents the number of samples in a training batch, K represents the number of different multimodal trajectories generated, T represents the prediction time (total number of frames), z represents the predicted future trajectory, and x represents the actual trajectory; It is the score of the k-th modality in the i-th sample, and This represents the mode that achieves the best result in the prediction.

[0198] Step 1.3 Design the basic structure of discriminator D;

[0199] The main components of the discriminator D are: (1) feature extraction and fusion module, (2) true / false scoring module, and (3) future interaction relationship evaluation module.

[0200] The feature processing for discriminator D is similar to that for G, except that, to obtain more complete trajectory information, the historical trajectory and the predicted trajectory are concatenated in the time dimension (or the historical trajectory and the true future trajectory are concatenated in the time dimension). The vector shape of the past trajectory is N×20×3; for the predicted future trajectory, one pattern is randomly selected from K patterns and input into D; for the true future trajectory, all patterns are input, and the valid bits of both the predicted and true future trajectories are set to 1, thus expanding the vector shape from N×30×2 to N×30×3. Concatenating these two vectors in the time dimension yields a vector of shape N×50×3, which contains richer and more complete trajectory information, aiding D in its judgment. Subsequently, feature extraction is performed on the trajectory vector and the map vector respectively, followed by feature fusion.

[0201] For the scoring module, a neural network is used to process the extracted features, and then a fully connected layer is used to output N scores of 0 to 1. The closer the score is to 1, the more realistic the discriminator considers the trajectory to be.

[0202] For the future interaction relationship evaluation module, the features of multiple vehicles are concatenated in pairs, and the concatenated features are processed by a neural network. The fully connected layer outputs a value of 0 to 1 to represent the probability of interaction in the future. The closer the value is to 1, the greater the probability of interaction. Finally, an M×M future interaction relationship evaluation matrix is ​​output.

[0203] Step 1.4 Design the loss function for discriminator D;

[0204] For the discriminator D, in addition to the adversarial training loss d_loss, we also need to pay attention to the loss pr_loss for future interaction relationship evaluation:

[0205]

[0206]

[0207]

[0208] Where pred_proximity represents the predicted probability of future interaction relationships, and rule_proximity represents the probability of interaction relationships calculated based on the trajectory.

[0209] Step 2: Design a sub-task network PRNet for the discriminator to predict future interactions between vehicles, and use it to calculate the loss for evaluating future interactions;

[0210] The sub-network PRNet is used to predict the future interaction relationships (pr) between each vehicle. Its inputs include real and fake samples (i.e., real future trajectories and predicted trajectories), historical trajectories, and map information. The goal of the prediction is to reduce the error between the predicted and actual future interaction probability labels. PRNet's position in the discriminator is as follows: Figure 2 As shown.

[0211] Step 2.1 Design the network structure for evaluating the subtask of future interaction relationships;

[0212] The Future Interaction Relationship Prediction Network (PRNet) predicts future interactions between vehicles. Assuming there are N vehicles, its input consists of historical trajectories, predicted trajectories, or actual trajectories of multiple vehicles in the same scene, as well as fused features of the map. The output is a prediction of the probability of pairwise interactions between vehicles in future time periods. .

[0213] The network processing procedure is as follows: vehicle features are combined pairwise into a single sample, for example, the features of vehicle i. Features of vehicle j spliced ​​into a feature The system alternately passes through multiple linear fully connected layers and their activation layers, and finally outputs a score through the output layer. The outputs of all vehicle combinations are arranged into an N×N matrix. Finally, each row of the matrix is ​​normalized to obtain a probability matrix pred_proximity between 0 and 1.

[0214] Step 2.2 Calculate the ground truth of future interaction relationships as training labels;

[0215] The labels for future interaction relationships are generated using rules based on road topology relationships. The core idea is that vehicles on adjacent roads have a higher probability of interaction. For the scenario to be predicted, the input consists of the endpoints of the true future trajectories of N vehicles, the midpoint coordinates of M road centerlines, and the topological relationship matrix of the road centerlines, specifically represented as four M×M 0 / 1 matrices: the predecessor relationship matrix `pre`, the successor relationship matrix `suc`, the left neighbor relationship matrix `left`, and the right neighbor relationship matrix `right`. First, the distance `d` between the endpoint of each vehicle's future trajectory and the midpoint coordinates of each road centerline is calculated. When `d` < 7 meters, the vehicle is considered to be on that road. This results in an N×M 0 / 1 road occupancy matrix. Subsequently, each road topology relationship matrix is ​​weighted and summed (e.g., 0.8×pre + 0.02×suc + 0.06×left + 0.15×right, where the weighted values ​​are derived from training) to obtain an M×M smoothing coefficient matrix `Topo`. The smoothing coefficient matrix represents the probability of traveling from one centerline `m1` to another centerline `m2`, and its visualization is shown below. Figure 3As shown, the blue lines indicate the possibility of roads reaching each other, while the thickness indicates the probability of a vehicle arriving at the next moment.

[0216] Multiply the road occ matrix by the smoothing coefficient matrix Topo and normalize each row of data to obtain the smoothed road occ matrix. At this point, each value in the matrix is ​​between 0 and 1, representing the probability that a vehicle will reach the road centerline in the next moment. It is generally assumed that the interaction probability between two vehicles reaching the same centerline is relatively high. Therefore, the smoothed road occupancy matrix is ​​multiplied by itself to obtain an N×N matrix. After normalizing each row by setting the diagonals to zero, the pairwise interaction probability matrix rr between vehicles is obtained, where... Let represent the ground truth probability of interaction between vehicle i and vehicle j, which will now be used as the training label. The visualization of future interaction relationships between vehicles calculated through rules is shown below. Figure 4 As shown, the dots represent vehicle positions, and the blue lines connecting them indicate future interaction relationships between vehicles. The thickness of the lines indicates the strength of the interaction relationship; the thicker the line, the stronger the interaction.

[0217] Step 2.3 Calculate the loss for evaluating future vehicle interaction relationships;

[0218] The error pr_loss between the future interaction probability pred_proximity and the interaction probability label rule_proximity is calculated. By reducing this error, PRNet is trained to capture potential future interaction relationships between vehicles. Furthermore, through gradient backpropagation during adversarial training, the generator implicitly learns future interaction relationships between vehicles during feature extraction and feature fusion. The pr_loss design is as follows:

[0219]

[0220] Step 3: Train the generator G and discriminator D alternately based on a dynamic weight strategy;

[0221] The adversarial training process between G and D is as follows:

[0222] Step 3.1 Training data processing;

[0223] Before training, the input map data and historical trajectory data need to be vectorized, and the true value of the probability of future interactions between vehicles needs to be calculated using a rule-based method.

[0224] Specifically, in the vectorization of vehicle historical data, the coordinates of all vehicles are first converted into a relative coordinate system with the coordinates of the target vehicle in the last frame of the historical time as the origin and the driving direction as the x-axis. Then, the increment of the x and y coordinates of each vehicle in each frame in the relative coordinate system is calculated. Finally, the time needs to be aligned. For vehicles observed in a certain frame, the effective position is 1, while for those not observed, the coordinates are set to (0,0) and the effective position is 0.

[0225] In the vectorization of map data, it is also necessary to convert the coordinate system of the road centerline into a relative coordinate system with the coordinates of the target vehicle in the last frame of history as the origin and the driving direction as the x-axis. Then, the displacement of each centerline coordinate point relative to the previous one is calculated, and a road is represented by a vector of relative coordinates plus displacement.

[0226] To calculate the true value of the future interaction probability of vehicles, a rule-based method is used: First, calculate the distance between the coordinates of each vehicle in the last frame of the future and the center line of each road, and use this to calculate the occupancy rate of each vehicle on each road. Then, smooth the occupancy rate matrix and multiply it by itself. After setting the diagonal to zero, normalize it to obtain the interaction probability matrix between vehicles, and use it as the true value.

[0227] Step 3.2 Fix the parameters of G and train the discriminator D;

[0228] Step 3.2.1 Generator G predicts future trajectories;

[0229] First, in a generator G with fixed parameters, a self-attention network or convolutional network is used to extract historical trajectory features and map features of all vehicles. These features are then fused using methods such as cross-attention, and the fused features serve as conditional information y. Subsequently, the conditional information y is concatenated with a random noise vector and input into the prediction network of G, which outputs a predicted future trajectory. ;

[0230] Step 3.2.2 Input fake samples into the discriminator;

[0231] Processing to predict future trajectories To piece it together with the historical trajectory to form a complete trajectory. (Fake sample);

[0232] Complete trajectory The conditional information y is input into the discriminator D to obtain D's discrimination against false samples. The score is obtained, and D is used to evaluate the future interaction relationship between vehicles. ;

[0233] Step 3.2.3 Input real samples into the discriminator;

[0234] Input a real future trajectory x (a true sample), and stitch it together with historical trajectories to form a complete trajectory. ; to determine the true trajectory of the future The conditional information y is input into the discriminator D to obtain D pairs of true samples. The rating, and the evaluation matrix for the probability of future interactions. ;

[0235] Step 3.2.4 Calculation of the loss of discriminator D;

[0236] The goal of the discriminator D is to better distinguish between real and fake samples. Therefore, the true value of fake samples is set to 0 and the true value of real samples is set to 1. The average error between the scores of real and fake samples output by D and their respective true values ​​is calculated as the loss d_loss of the ScoreNet real and fake scoring module.

[0237] The average error between pr1, pr2 and the true value is directly calculated and used as the loss pr_loss of the future interaction relationship evaluation module;

[0238] Step 3.2.5 The DWA algorithm adjusts the loss weights of D and updates the parameters;

[0239] For the first batch of training data, d_loss and pr_loss are directly added together to obtain the loss D; starting from the second batch of data, the rate of change of d_loss and pr_loss recorded in this batch and the previous batch are calculated each time, and the weights are reversibly assigned according to the magnitude of the rate of change, and the weighted losses are added together to obtain the loss D.

[0240] After obtaining the loss of D, backpropagate to update the parameters of D;

[0241] Step 3.3 Fix the parameters of D and train the generator G;

[0242] Step 3.3.1 Generator G predicts future trajectories

[0243] The process is the same as step 3.2.1;

[0244] Step 3.3.2 Input fake samples into the discriminator

[0245] The process is the same as step 3.2.2;

[0246] Step 3.3.3 Calculation of the loss of generator G

[0247] The goal of generator G is the opposite of that of discriminator D, which is to make D identify fake samples as real. Therefore, the true value of fake samples is set to 1, and the average error between the score of fake samples output by D and the true value is calculated as the loss g_loss of generator G in adversarial training.

[0248] To avoid mode collapse and maintain the accuracy of the predicted trajectory, it is also necessary to calculate the regression loss reg_loss and classification loss cls_loss between the multimodal trajectory predicted by the generator G and the actual future trajectory.

[0249] Step 3.3.4 The DWA algorithm adjusts the loss weights of G and updates the parameters;

[0250] For the first batch of training data, g_loss, reg_loss and cls_loss are directly added together to obtain the loss G; starting from the second batch of data, the rate of change of the loss of g_loss, reg_loss and cls_loss recorded in the current batch and the previous batch are calculated each time, and the weights are reversed according to the magnitude of the rate of change, and the weighted losses are added together to obtain the loss G.

[0251] After obtaining the loss of G, backpropagation is used to update the parameters of G;

[0252] Step 3.4 Alternate training until the losses of G and D converge;

[0253] The Dynamic Weighted Average (DWA) algorithm calculates the loss for each subtask during training on each batch of data. It then calculates the rate of change of error based on the losses of each subtask during the previous backpropagation, and reassigns weights according to the magnitude of this rate of change. This dynamic balance of weights across the subtasks achieves collaborative optimization. The weight update formula for the DWA algorithm is shown below:

[0254]

[0255]

[0256]

[0257] in, Indicates task The ratio of the loss at time t-1 to the loss at time t-2 reflects the convergence speed of the task's loss. This represents the loss weight that can be assigned to task i at time t, which is determined by the weights at time t-1. The index of other tasks The ratio of the exponents of the sum is related. I represents the number of tasks, and T is the temperature coefficient. In this case, I is 3 in G, 2 in D, and T is always 2.

[0258] The generator G obtained at this point is the optimized LaneGCN model. Experiments show that the minADE decreased from 0.71 meters to 0.70 meters, and the MR decreased from 10.3% to 10.2%, indicating a significant improvement in accuracy. The inference time of the model before the improvement was 4.0 ms, and the inference time after the improvement is [missing value]. The overhead is within a manageable range and can be ignored.

[0259] The example uses LaneGCN as the prototype for generator G, and completes model training and optimization within an adversarial training framework to obtain an improved model. The improved LaneGCN model maintains the same structure as the original model. To meet the deployment requirements of the actual vehicle platform, the improved model needs to be quantized on the server side, proceeding to the second part.

[0260] II. The model quantization stage, including step 4, is performed on the server.

[0261] Step 4: Quantize the trained model;

[0262] The trajectory prediction generator model G, trained in step 3, is used as a prototype of LaneGCN. Quantization-Aware Training (QAT) is employed to achieve model lightweighting, resulting in an efficient inference model that can be deployed on the board. The model quantization process is as follows: Figure 5 As shown.

[0263] Step 4.1 Obtain the floating-point model;

[0264] Step 4.1.1 Obtain the optimal set of weight parameters from the generator G that has reached convergence in the adversarial training in Step 3, and load this set of parameters into the preset LaneGCN prototype network structure to obtain the original floating-point model with complete logical structure and parameter weights.

[0265] Step 4.1.2: Perform structural modifications on the floating-point model:

[0266] The input layer of the original floating-point model is encapsulated using the QuantStub module to construct a quantization interface for converting input features from floating-point numbers to fixed-point numbers. The output layer of the original floating-point model is encapsulated using the DeQuantStub module to construct a dequantization interface for converting output results from fixed-point numbers to floating-point numbers, thereby obtaining the modified floating-point model.

[0267] Step 4.2 Quantize and calibrate the model;

[0268] Quantization calibration is performed to obtain the quantization parameters for each layer of the model.

[0269] Step 4.2.1 Utilize the Observer module to dynamically monitor the activation values ​​and weights of each layer of the modified floating-point model during forward inference. By traversing the calibration dataset, the Observer module extracts the numerical boundaries (including minimum, maximum, and distribution patterns) of the data streams at each layer, serving as the statistical basis for calculating the scaling factor and zero point.

[0270] Step 4.2.2 Perform multiple forward inferences on the floating-point model using the calibration dataset, accumulate the observed data distribution, calculate the optimal quantization parameters for each layer, and establish the mapping relationship between floating-point data and fixed-point data.

[0271] Step 4.2.3 Verify the results of the calibrated floating-point model with quantization parameters on preset evaluation indicators, including minADE, minFDE, and MR:

[0272] Step 4.2.3.1 If the verification results meet the preset requirements, then based on the calibration results, convert the calibrated floating-point model with quantization parameters into a fixed-point model. And used for subsequent inference deployment;

[0273] If the verification result does not meet the preset requirements, the calibration result is used as the initialization, and the process proceeds to step 4.3 to perform quantitative perception training to further optimize the inference accuracy after quantization.

[0274] Step 4.3 Quantization Aware Training (QAT) for Model.

[0275] Step 4.3.1 Enable the Fake Quantization Node during the training phase to simulate the numerical behavior of fixed-point quantization and dequantization during forward inference, so that the model can take into account the numerical error caused by quantization during the training phase in Step 4.3.2; on this basis, continue to perform backpropagation and parameter update to enable the model parameters to be adaptively optimized to the quantization error.

[0276] Step 4.3.2 During the quantization perception training process, the original trajectory prediction task objective remains unchanged, and the pseudo-quantization node is kept in the enabled state. The model weights are driven by error backpropagation to perform adaptive gradient compensation based on the numerical deviation generated by pseudo-quantization, so that the training process is consistent with the numerical characteristics of fixed-point inference.

[0277] After training is completed in step 4.3.3, the pseudo-quantized nodes are removed, and the quantization parameters and model weights are solidified to obtain the final fixed-point model for inference. Export the obtained fixed-point model as an hbm file for subsequent deployment on the actual vehicle board.

[0278] III. Inference Deployment Phase: Deployed on the actual vehicle's onboard system for inference operation and visualization, including step 5.

[0279] Step 5: Deployment of actual vehicles and visualization of predicted trajectories;

[0280] The fixed-point model obtained in step 4 Deployed on an in-vehicle embedded computing platform, the system performs trajectory prediction and inference at preset cycles during vehicle operation, and publishes the prediction results to downstream planning and control modules in a standard communication format. At the same time, the system enables visualization of the predicted trajectory on the in-vehicle display terminal and remote monitoring system.

[0281] Step 5.1 Offline verification and consistency check;

[0282] In fixed-point model Before the vehicle is put into operation, the fixed-point model is first tested. Perform offline verification and consistency checks. Select a verification dataset consistent with the real-vehicle operation scenario and test the fixed-point model. Offline inference is performed and preset evaluation metrics are calculated. These preset evaluation metrics include at least minADE, minFDE, and MR, and are used to evaluate the fixed-point model. Performance retention relative to the floating-point model.

[0283] At the same time, for fixed-point models Perform a consistency check on the input and output: For the input part, check the fixed-point model. Consistency with floating-point models in data normalization, scale mapping, and feature concatenation order; for the output, check the consistency of predicted trajectory point sequence, modal probability / confidence, and time series structure.

[0284] Once the offline verification results meet the preset requirements and the input-output consistency meets the preset rules, proceed to step 5.2 to perform vehicle-mounted terminal board inference.

[0285] Step 5.2 Visualization of vehicle-mounted board operation and predicted trajectory;

[0286] After completing model validation in step 5.1, proceed to step 5.2:

[0287] Fixed-point model Integrated into the vehicle system and running on an embedded computing platform, it enables real-time trajectory prediction and result visualization. This stage utilizes multi-threaded asynchronous link organization, through two parallel logical stages: "state maintenance" and "predictive inference," specifically as follows... Figure 6As shown, the dynamic maintenance tasks of data access and environmental status are executed by the status maintenance thread (hereinafter referred to as thread A in the figure), and the trajectory prediction and inference tasks are driven by the periodic triggering mechanism (10Hz) of the prediction and inference thread (hereinafter referred to as thread B in the figure), thereby realizing a complete closed loop from data access to result display.

[0288] Thread A involves step 5.2.1;

[0289] Thread B involves steps 5.2.2, 5.2.3, 5.2.4, 5.2.5, and 5.2.6.

[0290] Step 5.2.1 Data access and dynamic maintenance of environment status

[0291] 5.2.1.1 Data Access:

[0292] The vehicle-mounted terminal uses thread A to receive structured environmental information output by the upstream sensing module in real time, including the vehicle's motion state information and the motion state information of surrounding vehicles, and then continuously writes the structured environmental information into the message buffer of the vehicle-mounted terminal.

[0293] 5.2.1.2 Dynamic maintenance of environmental status:

[0294] The target maintenance logic module on the vehicle end is used to parse the message buffer data, and the parsing results are updated and stored in the shared state library in real time to obtain global vehicle information. During parsing, the target identifier is used as an index, and the global vehicle information structure is formed by using a sliding window method, which includes historical window sequences and corresponding timestamps, and the dynamic maintenance of the environment state is realized.

[0295] Step 5.2.2 Periodic Triggering and Data Snapshot Generation:

[0296] At the trigger moment, thread B uses concurrent control to perform a one-time read and copy of the data required for the current period from the shared state library, generating an independent data snapshot corresponding to the global vehicle information for the current period. This data snapshot includes the vehicle's status and target information. This embodies the asynchronous decoupling mechanism, ensuring that feature processing and model inference within this period are both performed based on static snapshots, avoiding inconsistencies in input caused by real-time updates of perception data in step 5.2.1, and ensuring the reproducibility of the inference process.

[0297] Step 5.2.3 Feature preprocessing and standardized input construction:

[0298] Thread B constructs a normalized tensor based on the data snapshot described in step 5.2.2, which serves as the fixed-point model in step 5.2.4. The input specifically includes:

[0299] Step 5.2.3.1 State Alignment and Reference System Transformation:

[0300] For each target in the snapshot, its motion state within the historical window is retrieved in chronological order, and a spatial transformation operator is used to perform state alignment processing. Specifically, the alignment processing transforms the coordinate positions and orientation angles of each target at different historical sampling times to a preset reference coordinate system with the current vehicle position as the origin. This eliminates the observation perspective deviation caused by the vehicle's motion, achieving physical position alignment of multiple target historical trajectories under the same spatiotemporal reference, and providing a standardized spatial level for subsequent feature construction.

[0301] Step 5.2.3.2 Feature Construction and Dimensional Regularization:

[0302] Feature engineering logic is used to construct features and map the motion states of each target after alignment in step 5.2.3.1. Specifically, by solving for original physical quantities such as historical displacement, velocity, and orientation angle, feature vectors containing spatiotemporal evolution patterns are generated, thereby enabling subsequent fixed-point models. It can capture the acceleration / deceleration trends and steering characteristics of target vehicles. Simultaneously, it performs dimension normalization for targets with inconsistent historical frame counts, ensuring that the feature vectors of all targets are aligned in length along the time axis by zero-padding missing frames, thus making their dimensions fully consistent with subsequent fixed-point models. The calculation requirements.

[0303] Step 5.2.3.3 Construction of Normalized Tensors:

[0304] Using a pre-allocated contiguous memory buffer, the feature vectors produced in step 5.2.3.2 are arranged and encapsulated according to the "target-time-feature" dimension. This step directly constructs a fixed-point model by mapping discrete features to a contiguous memory space. The standardized tensors required by the input layer are designed to eliminate the overhead of intermediate data transport before inference, ensuring the response latency of the end-to-end link from the underlying implementation level.

[0305] Step 5.2.4 Fixed-point model inference:

[0306] The normalized tensor input is fed into the fixed-point model using thread B. The inference process is performed on the onboard hardware computing resources and outputs the raw prediction results. These raw prediction results include at least the multimodal trajectories in the future time domain and the confidence information corresponding to each trajectory. The multimodal trajectories refer to multiple possible paths predicted for the same target, representing different driving intentions (such as going straight, turning left, turning right, etc.), to cover the uncertainty of the target's future movement. The trajectories are expressed in the form of trajectory offsets, that is, the relative displacement vectors of the target relative to its current position at various future times, thereby reducing the convergence difficulty of the model predicting absolute coordinates. After inference, the raw prediction results, containing multiple sets of offset sequences and probability distributions, are passed to the post-processing stage.

[0307] Step 5.2.5 Post-processing of results and publication of standards:

[0308] Thread B uses post-processing logic combined with the target state information at the current moment to modify the fixed-point model. The output execution coordinates are restored, and the preferred predicted trajectory is extracted based on the confidence level filtering rules.

[0309] Subsequently, the optimized predicted trajectory is encapsulated in a standard communication format and published to the downstream vehicle planning and control module. The published content includes at least the predicted trajectory sequence, confidence level, and associated timestamp, aiming to ensure that the downstream planning and control module can make time-aligned decisions.

[0310] Step 5.2.6 Time Consumption Statistics and Prediction Results Visualization:

[0311] This step involves thread B using parallel time-consuming statistics and trajectory visualization monitoring submodules to achieve real-time monitoring of the entire processing time and intuitive rendering of prediction results, thus realizing a fixed-point model. Closed-loop operation and interactive display on the vehicle side:

[0312] First, the performance of key nodes in each inference cycle is monitored using a time consumption statistics submodule. These key nodes include the entire process executed in thread B, including steps 5.2.2 (data snapshot generation), 5.2.3 (feature preprocessing and standardized input construction), 5.2.4 (fixed-point model inference), and 5.2.5 (result post-processing and standard release). By acquiring the computation time of each step in real time, the end-to-end processing logic is dynamically ensured to run stably within the preset scheduling cycle, thereby ensuring the timeliness of the prediction results.

[0313] Secondly, the trajectory visualization monitoring submodule receives the optimized predicted trajectory data released in step 5.2.5 in real time and performs dynamic rendering on the vehicle display terminal or remote monitoring system to intuitively present the target's driving intention. For example... Figure 7As shown, the system uses different colors to distinguish vehicles: blue vehicles represent target vehicles to be predicted, and red vehicles represent the vehicle itself. In the diagram, blue represents the future trajectory predicted by the model, and arrows indicate the vehicle's direction of travel. The visualization interface updates dynamically at a 10Hz refresh rate, achieving real-time synchronization of prediction results and vehicle status.

[0314] This step quantifies the fixed-point model. A complete closed-loop operation, from real-time data input and predictive inference to result display, was achieved on the vehicle-mounted device. Implementation examples demonstrate that this fixed-point model... It can run stably in real-world embedded environments, with an average single-frame inference latency of less than 100 milliseconds. The prediction error is consistent with that of the floating-point model, effectively supporting the real-time requirements of vehicle trajectory prediction, decision planning, and visualization interaction.

[0315] The above description is merely a description of preferred embodiments of this application and is not intended to limit the scope of this application in any way. Any changes or modifications made by those skilled in the art based on the above-disclosed technical content should be considered as equivalent and valid embodiments and fall within the scope of protection of the technical solution of this application.

Claims

1. A method for vehicle trajectory prediction and monitoring based on discriminator implicit future interaction learning guidance, characterized in that, Includes the following steps: Step 1: Design the training framework for the trajectory prediction model based on the cGAN framework; The cGAN framework includes a generator G and a discriminator D. The trajectory prediction model is obtained through adversarial training between the generator G and the discriminator D. The generator G is used for trajectory prediction. It takes the vehicle's historical trajectory features and map features as conditional information y, and inputs them together with a noise vector sampled from a random noise distribution to generate a predicted future trajectory z. Discriminator D is responsible for supervising and evaluating the generation effect of generator G. The predicted future trajectory z, condition information y, and the real future trajectory x are input into discriminator D, which judges whether they are true or false. In the discriminator D, the future interaction relationship between vehicles is predicted by the sub-task network PRNet of future interaction relationship. The total loss function considers not only the loss d_loss of adversarial training, but also the loss pr_loss of future interaction relationship evaluation. Step 2: Design a sub-task network PRNet to evaluate future interactions between vehicles, and calculate the loss for evaluating future interactions; To evaluate future interaction relationships in the discriminator, a subtask network PRNet is designed to evaluate future interaction relationships between vehicles. The interaction probability rr of the real future trajectories between vehicles, calculated based on road topology rules, is used as the label. By minimizing the difference between the predicted future relationship value pr and the label rr, the network is guided to focus on the future interaction relationships between vehicles. Step 3: Train the generator G and discriminator D alternately based on a dynamic weight strategy; Step 4: Model Quantization The trained trajectory prediction generator model G is lightweighted and optimized for deployment through quantized perception training. Step 5: Real vehicle deployment and visualization of predicted trajectory The prediction model obtained after step 4 quantization Deployed in the vehicle-embedded computing platform of the in-vehicle device to achieve real-time predictive inference and visualize the results in the actual vehicle and at the regulatory end; Step 2 includes: Step 2.1 Design the network structure for evaluating the subtask of future interaction relationships; The Future Interaction Relationship Evaluation Network (PRNet) predicts future interaction relationships between vehicles. Assuming there are N vehicles, its input consists of the historical trajectories, predicted trajectories, or actual trajectories of multiple vehicles in the same scene, as well as the fusion features of the map. The output is a matrix pr evaluating the interaction relationships between vehicles in future time periods; The network processing procedure is as follows: Vehicle features are merged pairwise into a single feature. ,feature Characteristics of vehicle i Features of vehicle j The concatenated features are processed sequentially through a network consisting of multiple linear fully connected layers and their activation layers. Finally, a score is output through the output layer. The outputs of all vehicle combinations are arranged into an N×N matrix. Finally, each row of the matrix is ​​normalized to obtain the probability matrix pred_proximity, or pr, which is between 0 and 1. Step 2.2 Calculate the ground truth of future interaction relationships as training labels; The labels for future interaction relationships are generated using rules based on road topology relationships: Assuming there are N vehicles and M road centerlines, first calculate the initial occupancy matrix for each vehicle at the road centerline based on the Euclidean distance from the vehicle to the road center. 1 indicates occupied, 0 indicates not occupied; Then, to further model the potential movement intentions of vehicles, the initial road occupancy rate of vehicles is determined based on the predecessor, successor, left neighbor, and friendly neighbor topological relationships between lanes, as well as the prior probability of a vehicle appearing in an adjacent road. The occupancy rate is propagated to its adjacent lanes in a weighted manner to obtain a smooth occupancy rate. The sum of vehicle occupancy rates across all roads is 1. The smoothed occupancy rate essentially represents the probability that a vehicle will appear at the centerline position of each road in the next moment. Since vehicles with high occupancy probabilities on the same road centerline have a higher likelihood of interaction, the smoothed occupancy rate matrix... Multiplying by itself yields the symmetric interaction probability matrix between vehicles. ; Finally, normalization is performed to obtain the probability of a vehicle interacting with all other vehicles from the perspective of a certain vehicle, which is more in line with the actual situation. The calculation process is shown in the following formula: in, The coordinate matrix representing the vehicle. The coordinate matrix representing the centerline of the road. Represents the distance function. Indicates the distance threshold. The matrix represents the topological relationships, which are categorized into predecessor (pre), successor (suc), left neighbor (left), and right neighbor (right). Pre and successor relationships are further divided into direct adjacent nodes, second-order adjacent nodes, and so on up to fifth-order adjacent nodes. For each type of topological relationship, an M×M 0 / 1 matrix is ​​used. This is expressed as an expression, and multiplied by their respective influence coefficients. The subsequent superposition, where the influence coefficients are iteratively optimized through backpropagation during training; is obtained by weighted summation of the topological relationships between roads. As shown in the following formula: Among them, the influence coefficient This represents the weight of a certain adjacent topological relationship in the relationship calculation. The magnitude of this coefficient depends on the probability that a vehicle will appear in that adjacent topological segment in the next time step. , respectively representing the road that appears before, the road that appears after, the road that appears on the left, and the road that appears on the right; Step 2.3 Calculate the loss for evaluating future vehicle interaction relationships; The error pr_loss between the future interaction probability pred_proximity and the interaction probability label rule_proximity is calculated. By reducing this error, PRNet is trained to capture potential future interaction relationships between vehicles. Furthermore, gradient backpropagation during adversarial training allows the generator to implicitly learn future vehicle interaction relationships during feature extraction and feature fusion. The pr_loss design is as follows: 。 2. The vehicle trajectory prediction and monitoring method based on discriminator implicit future interaction learning guidance as described in claim 1, characterized in that, In step 1, The overall objectives of adversarial training are as follows: in, This represents the value function in a generative adversarial network. Indicates when the input sample From real data distribution At that time, the discriminator The expected value of the logarithmic probability of making a correct judgment about it; and This indicates that when the input sample is generated by the generator From noise prior During generation, the discriminator The expected value of the logarithmic probability of correctly judging it; for the discriminator The goal is to maximize the expected probability of making the correct judgment, that is... and The goal of the generator G is to maximize the sum of the two elements; while the goal of the generator G is exactly the opposite. and Minimize the summation; In adversarial training, G and D are trained alternately to reach Nash equilibrium; in order to save inference time, the discriminator D is used only in training, while in practical applications only the generator G is used for trajectory prediction.

3. The vehicle trajectory prediction and monitoring method based on discriminator implicit future interaction learning guidance according to claim 1, characterized in that, Step 1 includes: Step 1.1 Design the basic structure of generator G; The generator G extracts and fuses features from the input historical trajectory and map information, using them as the generator's conditional information y. At the same time, it samples various noises from a Gaussian distribution. The conditional information y and the noise are input into the prediction network, and the final output is the predicted future trajectory z. The specific process is as follows: Feature extraction and fusion: In the feature extraction stage, convolutional networks or self-attention networks are used to extract trajectory features and map features respectively; in the feature fusion stage, convolutional networks or cross-attention networks are used to fuse the trajectory and map features in the temporal and spatial dimensions. Sampling and processing Gaussian noise: Randomly generate a vector of a specific length, merge it with the fused trajectory and map features to form a new feature vector, which serves as the input to the trajectory generation module; Generate predicted trajectories: First, the merged feature vectors are processed using an attention mechanism, and then the predicted trajectories of all vehicles are output using a prediction network; the generator G contains K prediction networks with the same structure but different parameters to generate K combinations of multi-vehicle trajectories. Step 1.2 Design the loss function for generator G; The loss function of G is: Among them, the regression loss reg_loss calculates the error between the generated trajectory and the true trajectory in multimodal trajectory prediction, and the formula is: Where N represents the total number of vehicles in a scene, K represents the K different multimodal trajectories generated, and T represents the prediction time, i.e., the total number of frames. Let represent the coordinates of the k-th predicted trajectory of the i-th vehicle at time t. Let represent the coordinates of the actual trajectory of the i-th vehicle at time t; The classification loss cls_loss is used to optimize the generator's confidence evaluation of multiple trajectories, so that the trajector that is closest to the real trajectory among the generated K modalities is assigned the highest confidence score, thereby improving the accuracy of trajectory ranking. For classification loss, first calculate the loss for each vehicle. Predicted trajectory in each mode k With the actual trajectory average distance : Then, the mode with the smallest average distance is selected as the optimal mode: Finally, the classification loss is defined as the negative log-likelihood mean of the best modal confidence across all vehicles: in, This represents the confidence score given by generator G for the predicted trajectory of the i-th vehicle in the k-th mode. This score is obtained by mapping the trajectory features through an MLP. The confidence score of the trajectory selected as the best mode; g_loss is the loss of G generated during adversarial training, and its calculation formula is: Step 1.3 Design the basic structure of discriminator D; The discriminator D is used to score the reasonableness of the generator G's output and evaluate future interaction relationships. Its inputs are conditional information y, the true future trajectory x, and the predicted future trajectory output by generator G. ; First, feature extraction and feature fusion are required for the input. Then, the features are fed into the neural network for scoring and evaluation of future interaction relationships. The final output is a trajectory true / false score between 0 and 1, and an interaction probability matrix between each pair of vehicles. Specifically as follows: Feature extraction and fusion: First, the historical trajectory is compared with the predicted future trajectory. Alternatively, historical trajectories and real trajectories x are spliced ​​together in the time dimension, and features are extracted from the spliced ​​complete trajectory and map respectively, followed by feature fusion; True / False Scoring: The fused features are processed using a neural network, and then a fully connected layer is used to output a score of 0 to 1. The closer the score is to 1, the more real the discriminator considers the trajectory to be. Future interaction relationship evaluation: The features of multiple vehicles are concatenated in pairs, and the concatenated features are processed by a neural network. The output of a fully connected layer is 0 to 1 to represent the prediction of the probability of interaction in the future. The closer to 1, the more likely the interaction is to occur. Step 1.4 Design the loss function for discriminator D; For the discriminator D, in addition to the adversarial training loss d_loss, a loss pr_loss for evaluating future interaction relationships also needs to be added, resulting in a total loss function. Defined as follows: Where d_loss is the loss of the discriminator D during adversarial training, and its calculation formula is: pr_loss is the error between the predicted future interaction probability and the interaction probability label, which is calculated by the PRNet sub-task network for future interaction relationships.

4. The vehicle trajectory prediction and monitoring method based on discriminator implicit future interaction learning guidance according to claim 1, characterized in that, Step 3 includes: Step 3.1 Training data processing; Before training, the input map data and historical trajectory data need to be vectorized, and the true value of the probability of future interactions between vehicles needs to be calculated using a rule-based method. Specifically, in the vectorization of vehicle historical data, the coordinates of all vehicles are first converted into a relative coordinate system with the coordinates of the target vehicle in the last frame of the historical time as the origin and the driving direction as the x-axis. Then, the increment of the x and y coordinates in the relative coordinate system is calculated for each vehicle in each frame. Finally, the time needs to be aligned. For vehicles observed in a certain frame, the effective position is 1, while for those not observed, the coordinates are set to (0,0) and the effective position is 0. In the vectorization of map data, it is also necessary to convert the coordinate system of the road centerline into a relative coordinate system with the coordinates of the target vehicle in the last frame of history as the origin and the driving direction as the x-axis. Then, the displacement of each centerline coordinate point relative to the previous one is calculated, and a road is represented by a vector of relative coordinates + displacement. To calculate the true value of the future interaction probability of vehicles, a rule-based method is used: First, calculate the distance between the coordinates of each vehicle in the last frame of the future and the center line of each road, and use this to calculate the occupancy rate of each vehicle for each road. Then, smooth the occupancy rate matrix and multiply it by itself. After setting the diagonal to zero, normalize it to obtain the interaction probability matrix between vehicles, and use it as the true value. Step 3.2 Fix the parameters of G and train the discriminator D; Step 3.2.1 Generator G predicts future trajectories; First, in a generator G with fixed parameters, a self-attention network or convolutional network is used to extract historical trajectory features and map features of all vehicles. These features are then fused using methods such as cross-attention, and the fused features serve as conditional information y. Subsequently, the conditional information y is concatenated with a random noise vector and input into the prediction network of G, which outputs a predicted future trajectory. ; Step 3.2.2 Input fake samples into the discriminator; Processing to predict future trajectories To piece it together with the historical trajectory to form a complete trajectory. That is, fake samples; Complete trajectory The conditional information y is input into the discriminator D, and D's conditional information for false samples is obtained. The score is obtained, and D is used to evaluate the future interaction relationship between vehicles. ; Step 3.2.3 Input real samples into the discriminator; Input the actual future trajectory x, and combine it with the historical trajectory to form a complete trajectory. That is, true samples; representing the real future trajectory. The conditional information y is input into the discriminator D to obtain D pairs of true samples. The rating, and the evaluation matrix for the probability of future interactions. ; Step 3.2.4 Calculation of the loss of discriminator D; The goal of the discriminator D is to better distinguish between real and fake samples. Therefore, the true value of fake samples is set to 0 and the true value of real samples is set to 1. The average error between the scores of real and fake samples output by D and their respective true values ​​is calculated as the loss d_loss of the ScoreNet real and fake scoring module. The average error between pr1, pr2 and the true value is directly calculated and used as the loss pr_loss of the future interaction relationship evaluation module; Step 3.2.5 The DWA algorithm adjusts the loss weights of D and updates the parameters; For the first batch of training data, d_loss and pr_loss are directly added together to obtain the loss D; starting from the second batch of data, the rate of change of d_loss and pr_loss recorded in this batch and the previous batch are calculated each time, and the weights are reversibly assigned according to the magnitude of the rate of change, and the weighted losses are added together to obtain the loss D. After obtaining the loss of D, backpropagate to update the parameters of D; Step 3.3 Fix the parameters of D and train the generator G; Step 3.3.1 Generator G predicts future trajectories; The process is the same as step 3.2.1; Step 3.3.2 Input fake samples into the discriminator; The process is the same as step 3.2.2; Step 3.3.3 Calculation of the loss of generator G; The goal of generator G is the opposite of that of discriminator D, which is to make D identify fake samples as real. Therefore, the true value of fake samples is set to 1, and the average error between the score of fake samples output by D and the true value is calculated as the loss g_loss of generator G in adversarial training. To avoid mode collapse and maintain the accuracy of the predicted trajectory, it is also necessary to calculate the regression loss reg_loss and classification loss cls_loss between the multimodal trajectory predicted by the generator G and the actual future trajectory. Step 3.3.4 The DWA algorithm adjusts the loss weights of G and updates the parameters; For the first batch of training data, g_loss, reg_loss and cls_loss are directly added together to obtain the loss G; starting from the second batch of data, the rate of change of the loss of g_loss, reg_loss and cls_loss recorded in the current batch and the previous batch are calculated each time, and the weights are reversed according to the magnitude of the rate of change, and the weighted losses are added together to obtain the loss G. After obtaining the loss of G, backpropagation is used to update the parameters of G; Step 3.4 Alternate training until the losses of G and D converge; The DWA algorithm calculates the loss for each subtask during training on each batch of data, and calculates the error change rate based on the loss of each subtask in the previous backpropagation. Weights are then reassigned based on the magnitude of the change rate, resulting in a dynamic balance of weights across the subtasks and achieving collaborative optimization. The weight update formula for the Dynamic Weight Averaging algorithm is shown below: in, Indicates task The ratio of the loss at time t-1 to the loss at time t-2 reflects the convergence speed of the task's loss. This represents the loss weight that can be assigned to task i at time t, which is determined by the weights at time t-1. The index of other tasks The sum and the exponent are related; I represents the number of tasks, and T is the temperature coefficient. When T is sufficiently large, The weights of each task are equal.

5. The vehicle trajectory prediction and monitoring method based on discriminator implicit future interaction learning guidance according to claim 1, characterized in that, Step 4 includes: Step 4.1 Obtain the floating-point model Step 4.1.1 Obtain the optimal set of weight parameters from the generator G that has reached convergence in the adversarial training in Step 3, and load this set of parameters into a pre-defined prototype network structure that is isomorphic to the generator G to obtain the original floating-point model with complete logical structure and parameter weights. Step 4.1.2: Perform structural modifications on the floating-point model: The input layer of the original floating-point model is encapsulated using the QuantStub module to construct a quantization interface for converting input features from floating-point numbers to fixed-point numbers. The output layer of the original floating-point model is encapsulated using the DeQuantStub module to construct a dequantization interface for converting output results from fixed-point numbers to floating-point numbers, thereby obtaining the modified floating-point model. Step 4.2 Quantize and calibrate the model. Step 4.2.1 Utilize the Observer module to dynamically monitor the activation values ​​and weights of each layer of the modified floating-point model during the forward inference process; by traversing the calibration dataset, use the Observer module to extract the numerical boundaries of the data streams of each layer as the statistical basis for calculating the scaling factor and zero point. The numerical boundaries include the minimum value, maximum value, and distribution pattern. Step 4.2.2 Use the calibration dataset to perform multiple forward inferences on the floating-point model, accumulate the observed data distribution, calculate the optimal quantization parameters for each layer, and establish the mapping relationship between floating-point data and fixed-point data; Step 4.2.3 Verify the results of the calibrated floating-point model with quantization parameters on preset evaluation indicators, including minADE, minFDE, and MR: Step 4.2.3.1 If the verification results meet the preset requirements, then based on the calibration results, convert the calibrated floating-point model with quantization parameters into a fixed-point model. And used for subsequent inference deployment; Step 4.2.3.2 If the verification result does not meet the preset requirements, the calibration result is used as the initialization, and the process proceeds to step 4.3 to perform quantitative perception training to further optimize the inference accuracy after quantization. Step 4.3 Model Quantization Perception Training Step 4.3.1 Enable pseudo-quantization nodes during the training phase to simulate the numerical behavior of fixed-point quantization and dequantization during forward inference, so that the model can take into account the numerical error caused by quantization during the training phase in Step 4.3.2; on this basis, continue to perform backpropagation and parameter update to enable the model parameters to be adaptively optimized to quantization error; Step 4.3.2 During the quantization perception training process, the original trajectory prediction task objective remains unchanged, and the pseudo-quantization node is kept in the enabled state. The model weights are driven by error backpropagation to perform adaptive gradient compensation based on the numerical deviation generated by pseudo-quantization, so that the training process is consistent with the numerical characteristics of fixed-point inference. After training is completed in step 4.3.3, the pseudo-quantization nodes are removed, and the quantization parameters and model weights are solidified to finally obtain the fixed-point model used for inference. Export the obtained fixed-point model as an hbm file for subsequent deployment on the actual vehicle board.

6. The vehicle trajectory prediction and monitoring method based on discriminator implicit future interaction learning guided by claim 1, characterized in that, Step 5 includes: Step 5.1 Offline verification and consistency check In fixed-point model Before the vehicle is put into operation, the fixed-point model is first tested. Perform offline verification and consistency checks; Step 5.2 Visualization of vehicle-mounted board operation and predicted trajectory By utilizing a multi-threaded asynchronous link organization, through two parallel logical links of "state maintenance" and "predictive inference", specifically, the dynamic maintenance tasks of data access and environmental state are executed by the state maintenance thread (hereinafter referred to as: thread A), and the trajectory prediction inference task is driven by the periodic triggering mechanism of the prediction inference thread (hereinafter referred to as: thread B), thereby realizing a complete closed loop from data access to result display.

7. The vehicle trajectory prediction and monitoring method based on discriminator implicit future interaction learning guided by claim 5, characterized in that, Step 5.2 includes: Step 5.2.1 Data access and dynamic maintenance of environment status: 5.2.1.1 Data Access: The vehicle-mounted terminal uses thread A to receive structured environmental information output by the upstream sensing module in real time, including the vehicle's motion state information and the motion state information of surrounding vehicles, and then continuously writes the structured environmental information into the message buffer of the vehicle-mounted terminal. 5.2.1.2 Dynamic maintenance of environmental status: The target maintenance logic module on the vehicle end is used to parse the message buffer data, and the parsing results are updated and stored in the shared state library in real time to obtain global vehicle information. During parsing, the target identifier is used as an index, and the global vehicle information structure is formed by using a sliding window method, which includes the historical window sequence and the corresponding timestamp, and the dynamic maintenance of the environment state is realized. Step 5.2.2 Periodic Triggering and Data Snapshot Generation: At the trigger moment, thread B uses a concurrent control method to perform a one-time read and copy of the data required for the current period from the shared state library, generating an independent data snapshot corresponding to the global vehicle information for the current period. This data snapshot includes the vehicle status and target information. Step 5.2.3 Feature preprocessing and standardized input construction: Thread B constructs a normalized tensor based on the data snapshot described in step 5.2.2, which serves as the fixed-point model in step 5.2.

4. Input; Step 5.2.4 Fixed-point model inference The normalized tensor input is fed into the fixed-point model using thread B. Perform inference and output the original prediction results; the original prediction results include multimodal trajectories in the future time domain and the confidence information corresponding to each trajectory; Step 5.2.5 Post-processing of results and publication of standards Thread B uses post-processing logic combined with the target state information at the current moment to modify the fixed-point model. The output execution coordinates are restored, and the predicted trajectory is extracted according to the confidence level filtering rules; Subsequently, the predicted trajectory is encapsulated into a standard communication format and published to the downstream vehicle planning and control module. The published content includes the predicted trajectory sequence, confidence level, and associated timestamp to ensure that the downstream planning and control module can make time-aligned decisions. Step 5.2.6 Time Consumption Statistics and Visualization of Prediction Results Thread B, through parallel time consumption statistics and trajectory visualization monitoring submodules, achieves real-time monitoring of the entire processing time and intuitive rendering of prediction results, realizing a fixed-point model. The closed-loop operation and interactive display on the vehicle side.

8. The vehicle trajectory prediction and monitoring method based on discriminator implicit future interaction learning guided by claim 7, characterized in that, Step 5.2.3 Feature preprocessing and standardized input construction specifically includes: Step 5.2.3.1 State Alignment and Reference System Transformation: For each target in the snapshot, its motion state within the historical window is retrieved in chronological order, and a state alignment process is performed using a spatial transformation operator. The alignment process is to uniformly transform the coordinate position and orientation angle of each target at different historical sampling times to a preset reference coordinate system with the current vehicle position as the origin. Step 5.2.3.2 Feature Construction and Dimensional Regularization: Feature engineering logic is used to construct features and map the motion states of each target after alignment in step 5.2.3.

1. Specifically, by solving for the original physical quantities such as historical displacement, velocity, and orientation angle, feature vectors containing spatiotemporal evolution laws are generated, thereby enabling subsequent fixed-point models. It can capture the acceleration and deceleration trends and steering characteristics of target vehicles; at the same time, it performs dimension normalization processing on targets with inconsistent historical frame numbers, and performs zero-padding on missing frames to make the feature vectors of all targets length aligned on the time axis. Step 5.2.3.3 Construction of Normalized Tensors: Using a pre-allocated contiguous memory buffer, the feature vectors produced in step 5.2.3.2 are arranged and encapsulated according to the "target-time-feature" dimension; by mapping discrete features to contiguous memory space, a fixed-point model is directly constructed. The normalized tensors required for the input layer.

9. The vehicle trajectory prediction and monitoring method based on discriminator implicit future interaction learning guidance according to claim 8, characterized in that, Step 5.2.6 Time Consumption Statistics and Prediction Results Visualization: First, the performance of key nodes in each inference cycle is monitored using the time consumption statistics submodule. These key nodes include the entire process executed in thread B, which includes steps 5.2.2 (data snapshot generation), 5.2.3 (feature preprocessing and standardized input construction), 5.2.4 (fixed-point model inference), and 5.2.5 (result post-processing and standard release). By acquiring the computation time of each step in real time, the end-to-end processing logic is dynamically ensured to run stably within the preset scheduling cycle, thereby ensuring the timeliness of the prediction results. Secondly, the trajectory visualization monitoring submodule receives the predicted trajectory data released in step 5.2.5 in real time and performs dynamic rendering on the vehicle display terminal or remote monitoring system.

Images