Autonomous driving trajectory prediction method and device based on suggestion-optimization framework

Abstract

The present invention relates to the field of computer vision, and in particular to a method and system for autonomous driving trajectory prediction based on a suggestion-optimization framework. The method comprises: constructing a novel suggestion-optimization framework; obtaining learnable feature codes, inputting the learnable feature codes into a preset baseline model, and outputting decoded data through the baseline model; inputting the decoded data into the novel suggestion-optimization framework, performing trajectory prediction and trajectory optimization on the decoded data; and outputting trajectory prediction and trajectory optimization data to complete autonomous driving trajectory prediction based on the novel suggestion-optimization framework. The suggestion-optimization framework provided by the present invention introduces a clustering-based suggestion trajectory generation module, utilizes region partition predictions and region trajectory prototypes as refined prior anchor frames, and a complexity-based optimization module for distinguishing simple from complex trajectories, so that the optimization module can focus on learning the features of complex trajectories, thereby improving the prediction accuracy of long-tail scenarios.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and device for predicting autonomous driving trajectories based on a suggestion-optimization framework. Background Art

[0002] An autonomous driving system primarily consists of three modules: perception, prediction, and planning. The trajectory prediction module plays a connecting role. It receives environmental information identified by the perception module and predicts the movement trajectories of other traffic participants, providing the planning module with accurate and timely trajectory information about other traffic participants so that the planning module can better formulate appropriate action strategies. Therefore, accurately predicting the trajectories of surrounding vehicles is crucial for autonomous driving safety. The goal of the trajectory prediction task is to predict the future two-dimensional coordinates of a multimodal trajectory, given the target vehicle's historical information (including trajectory coordinates, speed, yaw angle, etc.), high-definition map information within a certain observation radius (including lane coordinates and type, etc.), and interaction information with surrounding vehicles.

[0003] Although existing models can accurately predict future trajectories to a large extent, they often fail to capture the characteristics of rare but potentially dangerous scenarios, also known as long-tail scenarios, such as sudden stops, sharp turns, and avoiding pedestrians, due to data imbalance. Although these overlooked scenarios do not occur frequently, they may have a significant impact on safety and decision-making. In other words, for non-long-tail scenarios, there are often no major safety hazards, but for those long-tail scenarios, if they cannot be accurately predicted, they are likely to cause huge safety hazards.

[0004] Long-tail learning is a technique used to improve model performance when dealing with imbalanced data. To address the difficulty of predicting long-tail data, existing mainstream methods mainly include resampling, reweighting, and contrastive learning. Resampling is a straightforward approach that increases the number of rare data samples in the training dataset, thereby balancing the distribution of top and tail samples. However, it increases training time and computing resource consumption. Reweighting methods adjust the loss weights of samples to increase the focus on rare data, thereby improving the model's generalization ability. However, this places certain requirements on parameter tuning and model selection. Contrastive learning has received significant attention in self-supervised feature learning. Previous work has separated simple and long-tail trajectories in the latent space, enabling models to more effectively identify and share information between challenging trajectories. However, when the dataset is large, different types of challenging scenarios may involve different features and context, which can make it difficult to effectively share information between them. Summary of the Invention

[0005] To address the technical problem that existing solutions can improve the overall prediction level of the model, but because the number of difficult scene samples is smaller than that of simple scenes, the model cannot fully learn their features, resulting in overlap or confusion between simple data and long-tail data in the feature space, and cannot effectively distinguish them, the embodiment of the present invention provides an autonomous driving trajectory prediction method and device based on a suggestion-optimization framework. The technical solution is as follows:

[0006] In one aspect, a method for autonomous driving trajectory prediction based on a suggestion-optimization framework is provided. The method is implemented by an autonomous driving trajectory prediction device based on the suggestion-optimization framework, and the method includes:

[0007] S1. Construct a new suggestion-optimization framework;

[0008] S2. Obtain a learnable feature code, input the learnable feature code into a preset baseline model, and output decoded data through the baseline model;

[0009] S3, input the decoded data into the novel suggestion-optimization framework to perform trajectory prediction and trajectory optimization on the decoded data;

[0010] S4: Output trajectory prediction and trajectory optimization data to complete autonomous driving trajectory prediction based on the new suggestion-optimization framework.

[0011] Optionally, in S1, a new suggestion-optimization framework is constructed, including:

[0012] Build a novel suggestion-optimization framework with two core modules;

[0013] Among them, the two core modules include: a proposed trajectory generation module based on clustering areas and a trajectory optimization module based on scene complexity;

[0014] The proposed trajectory generation module based on clustered regions includes: a proposed trajectory output head and a region predictor; the trajectory optimization module based on scene complexity includes: a complexity predictor and an optimized trajectory output head.

[0015] Optionally, in S2, a learnable feature code is obtained, the learnable feature code is input into a preset baseline model, and decoding data is outputted through the baseline model, including:

[0016] The encoder of the baseline model is used to encode and fuse the historical trajectory information of the target vehicle, scene map information, and the interaction features of surrounding vehicles, and all anchor-free proposed trajectory feature encodings are input into the decoder for query decoding.

[0017] Optionally, S3 inputs the decoded data into a novel suggestion-optimization framework to perform trajectory prediction and trajectory optimization on the decoded data, including:

[0018] Input the decoded data into the novel suggestion-optimization framework;

[0019] Train an offline clustering model to divide the regions;

[0020] Collect the endpoints of all true trajectories from the dataset and cluster them into M clusters using K-means.

[0021] The clustered region-based proposed trajectory generation module outputs the decoded and updated proposed trajectory feature encoding according to the region, generates multimodal proposed trajectories in each region, and predicts the most likely region through the region predictor;

[0022] The proposed trajectory in the prediction area is converted into a feature encoding as an anchor box and input into the trajectory optimization module based on scene complexity to predict the trajectory difficulty for branch optimization. Only difficult trajectories are optimized and simple proposed trajectories are directly output. Before optimization, the trajectory prototype of the prediction area is introduced as the regional prior.

[0023] Optionally, a trace output header is recommended, including:

[0024] The proposed trajectory output head is represented by M MLPs, each of which is responsible for outputting K multimodal trajectories within a region and the probability of each trajectory.

[0025] Optionally, before optimization, the trajectory prototype of the prediction area is introduced as a regional prior, including:

[0026] At the dataset level, the common features of all future trajectories in a region are extracted as regional trajectory prototype priors;

[0027] Generate a regional pseudo label for all trajectories in the dataset through the trained clustering model; the regional pseudo label provides the sample truth value for the regional feature extractor;

[0028] The LSTM model is used to extract the features of all future trajectories in the region, and the hidden features of the last time step are selected as the features of each future trajectory.

[0029] Aggregate the features of all future trajectories in the current area by performing a maximum pooling operation on the time dimension;

[0030] The feature prototypes of all future trajectories extracted from the current area are concatenated with the corresponding optimized trajectory feature codes to enhance the effect of the anchor frame. The concatenated prototypes are then input into the optimization module for bias calculation.

[0031] Optionally, a trajectory optimization module based on scene complexity includes:

[0032] We define scene complexity prediction as a self-supervised task, test the entire dataset using a baseline, and define a difficulty pseudo-label for each trajectory.

[0033] The learnable feature encoding after the maximum pooling operation in the modal dimension is used as input, and the multimodal trajectory suggestion features of the region are extracted through LSTM, and initialized with the hidden features of the last time step;

[0034] The difficulty prediction module is used to predict the difficulty of the trajectory, and the prediction result is compared with the difficulty pseudo-label to calculate the loss.

[0035] In another aspect, a system for autonomous driving trajectory prediction based on a suggestion-optimization framework is provided. The system is applied to the method for autonomous driving trajectory prediction based on a suggestion-optimization framework. The system includes:

[0036] Framework building unit, used to build a new suggestion-optimization framework;

[0037] a data processing unit, configured to obtain a learnable feature code, input the learnable feature code into a preset baseline model, and output decoded data through the baseline model;

[0038] The trajectory prediction and optimization unit is used to input the decoded data into the novel suggestion-optimization framework to perform trajectory prediction and trajectory optimization on the decoded data;

[0039] The data output unit is used to output trajectory prediction and trajectory optimization data to complete autonomous driving trajectory prediction based on the suggestion-optimization framework.

[0040] On the other hand, an autonomous driving trajectory prediction device based on a suggestion-optimization framework is provided, and the autonomous driving trajectory prediction device based on the suggestion-optimization framework includes: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned autonomous driving trajectory prediction methods based on the suggestion-optimization framework is implemented.

[0041] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned autonomous driving trajectory prediction methods based on the suggestion-optimization framework.

[0042] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0043] In an embodiment of the present invention, a new suggestion-optimization framework is proposed: the present invention introduces a clustering-based suggestion trajectory generation module, which uses region partition predictions and region trajectory prototypes as further refined prior anchor frames, and a complexity-based optimization module for distinguishing simple and complex trajectories, so that the optimization module can focus on learning the features of complex trajectories, thereby improving the prediction accuracy of long-tail scenarios. The proposed method can be easily combined with most existing methods to improve their performance in challenging scenarios. Experimental results show that the framework of the present invention improves the performance of three baseline models on the long-tail scenarios of the Argoverse1 and Argoverse2 datasets, and outperforms the existing best methods, while maintaining the prediction indicators on the overall dataset. The present invention also incorporates the GradNorm algorithm for automatically calculating loss weights for multiple tasks into the trajectory prediction direction, so as to better solve the problem of uneven weight distribution of multiple tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 This is a flow chart of an autonomous driving trajectory prediction method based on a suggestion-optimization framework provided by an embodiment of the present invention;

[0046] Figure 2 This is a diagram of the process architecture of the suggestion-optimization framework provided by an embodiment of the present invention;

[0047] Figure 3 1 is a framework diagram of a regional feature extractor provided by an embodiment of the present invention;

[0048] Figure 4 is a visual comparison diagram of different methods provided by embodiments of the present invention;

[0049] Figure 5 is a comparison diagram of the effectiveness of the region-based suggested trajectory generation module provided by an embodiment of the present invention;

[0050] Figure 6 This is a block diagram of an autonomous driving trajectory prediction system based on a suggestion-optimization framework provided by an embodiment of the present invention;

[0051] Figure 7 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0053] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0054] In the embodiments of the present invention, the terms "image" and "picture" may be used interchangeably. It should be noted that, when the distinction between them is not emphasized, their intended meanings are the same. The terms "of," "corresponding," and "corresponding" may be used interchangeably. It should be noted that, when the distinction between them is not emphasized, their intended meanings are the same.

[0055] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0056] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0057] The embodiment of the present invention provides an autonomous driving trajectory prediction method based on a suggestion-optimization framework. The method can be implemented by an autonomous driving trajectory prediction device based on a suggestion-optimization framework. The autonomous driving trajectory prediction device based on a suggestion-optimization framework can be a terminal or a server. Figure 1 The flowchart of the autonomous driving trajectory prediction method based on the suggestion-optimization framework is shown. The processing flow of the method may include the following steps:

[0058] S1. Construct a new suggestion-optimization framework;

[0059] In a feasible implementation, in S1, a novel suggestion-optimization framework is constructed, including:

[0060] Build a novel suggestion-optimization framework with two core modules;

[0061] Among them, the two core modules include: a proposed trajectory generation module based on clustering areas and a trajectory optimization module based on scene complexity;

[0062] The proposed trajectory generation module based on clustered regions includes: a proposed trajectory output head and a region predictor; the trajectory optimization module based on scene complexity includes: a complexity predictor and an optimized trajectory output head.

[0063] In a feasible implementation, the present invention designs a novel suggestion-optimization framework, which mainly includes two core modules: a clustering region-based suggestion trajectory generation module and a scene complexity-based trajectory optimization module. The main process is as follows: Figure 2 As shown in the figure, (1) the baseline model adopts a sequence-to-sequence structure, and a learnable feature code is input into the decoder as the feature code of the future proposed trajectory to decode the output of the encoder, where the number of groups of the proposed trajectory feature code is equal to the number of regions, and the dimension of each group is [modes, hidden_dims]; (2) The proposed trajectory generation module based on the clustered region outputs the proposed trajectory feature code after decoding and updating according to the region, generates multimodal proposed trajectories in each region, and predicts a most likely region through the region predictor; (3) The proposed trajectory in the predicted region is converted into a feature code as an anchor box and input into the trajectory optimization module based on scene complexity to predict the trajectory difficulty for branch optimization, so that only difficult trajectories are optimized and simple proposed trajectories are directly output. Before optimization, the trajectory prototype of the predicted region is introduced as a region prior to further enhance the model's understanding of the scene context; (4) Finally, the multimodal trajectory prediction head and classification head output multimodal trajectories and probabilities.

[0064] S2. Obtain a learnable feature code, input historical data into a preset baseline model, and output decoded data through the baseline model;

[0065] In one feasible implementation, in S2, a learnable feature code is obtained, the learnable feature code is input into a preset baseline model, and decoding data is outputted through the baseline model, including:

[0066] The encoder of the baseline model is used to encode and fuse the historical trajectory information of the target vehicle, scene map information, and the interaction features of surrounding vehicles, and all anchor-free proposed trajectory feature encodings are input into the decoder for query decoding.

[0067] S3, input the decoded data into the novel suggestion-optimization framework to perform trajectory prediction and trajectory optimization on the decoded data;

[0068] In one feasible implementation, the decoded data is input into a novel suggestion-optimization framework to perform trajectory prediction and trajectory optimization on the decoded data, including:

[0069] Input the decoded data into the novel suggestion-optimization framework;

[0070] Train an offline clustering model to divide the regions;

[0071] Collect the endpoints of all true trajectories from the dataset and cluster them into M clusters using K-means.

[0072] The clustered region-based proposed trajectory generation module outputs the decoded and updated proposed trajectory feature encoding according to the region, generates multimodal proposed trajectories in each region, and predicts the most likely region through the region predictor;

[0073] The proposed trajectory in the prediction area is converted into a feature encoding as an anchor box and input into the trajectory optimization module based on scene complexity to predict the trajectory difficulty for branch optimization. Only difficult trajectories are optimized and simple proposed trajectories are directly output. Before optimization, the trajectory prototype of the prediction area is introduced as the regional prior.

[0074] In a feasible implementation, it is suggested that the track output head includes:

[0075] The proposed trajectory output head is represented by M MLPs, each of which is responsible for outputting K multimodal trajectories within a region and the probability of each trajectory.

[0076] In one feasible implementation, before optimization, the trajectory prototype of the prediction area is introduced as the area prior, including:

[0077] At the dataset level, the common features of all future trajectories in a region are extracted as regional trajectory prototype priors;

[0078] Generate a regional pseudo label for all trajectories in the dataset through the trained clustering model; the regional pseudo label provides the sample truth value for the regional feature extractor;

[0079] The LSTM model is used to extract the features of all future trajectories in the region, and the hidden features of the last time step are selected as the features of each future trajectory.

[0080] Aggregate the features of all future trajectories in the current area by performing a maximum pooling operation on the time dimension;

[0081] The feature prototypes of all future trajectories extracted from the current area are concatenated with the corresponding optimized trajectory feature codes to enhance the effect of the anchor frame. The concatenated prototypes are then input into the optimization module for bias calculation.

[0082] In a feasible implementation, the trajectory optimization module based on scene complexity further includes:

[0083] We define scene complexity prediction as a self-supervised task, test the entire dataset using a baseline, and define a difficulty pseudo-label for each trajectory.

[0084] The learnable feature encoding after the maximum pooling operation in the modal dimension is used as input, and the multimodal trajectory suggestion features of the region are extracted through LSTM, and initialized with the hidden features of the last time step;

[0085] The difficulty prediction module is used to predict the difficulty of the trajectory, and the prediction result is compared with the difficulty pseudo-label to calculate the loss.

[0086] In one feasible implementation, multimodal trajectories are widely distributed, and driver behavior varies across different scenarios. However, existing methods often generate an insufficient number of trajectories when predicting future multimodal trajectories, which limits randomness and makes it difficult to cover the long-tail region. To address this issue, this paper proposes a clustering-based suggested trajectory method.

[0087] First, the present invention trains an offline clustering model to partition regions. The present invention collects the endpoints of all real trajectories from the dataset and clusters them into M clusters using K-means (K-means is a computationally efficient classic clustering technique that can be substituted with other clustering algorithms). This allows the clustering model to adapt to this distribution, allowing the model to partition regions according to the same distribution during inference. Trajectory endpoints are chosen for partitioning regions because, for long-term predictions, the deviation in prediction results for the last time step is the greatest due to accumulated errors. Therefore, only by correctly predicting the region in which the last time step falls will the most accurate result. The present invention then uses the baseline model's encoder to encode and fuse the target vehicle's historical trajectory information, scene map information, and surrounding vehicle interaction features. The encoded features of all anchor-free proposed trajectories are then input into a decoder for query decoding. Subsequently, the present invention uses a proposed trajectory output header to generate multimodal proposed trajectories for different regions. These proposed trajectories are then used as input to the optimization module. The proposed trajectory output header is also represented by M multimodal projection layers (MLPs), each of which is responsible for outputting K multimodal trajectories within a region and the probability of each trajectory, focusing on learning the characteristics of the region. When optimizing the proposed trajectory output head, the present invention calculates the sum of the losses of all multimodal trajectories within the prediction area, rather than adopting a winner-takes-all strategy. This allows each set of proposed trajectory feature encoding and output heads to independently learn the characteristics of its own region during the optimization process, without affecting each other, thus achieving a more even trajectory distribution.

[0088] In a feasible implementation, a large number of proposed trajectories have been obtained according to the region. Therefore, the present invention predicts a region as the region of most interest and selects the multimodal trajectory therein as a coarse-grained prior anchor frame to obtain the region where the driver's final driving position may be located, thereby allowing the model to be further refined. The present invention pre-trains a regional prediction model by designing a self-supervised task. In order to make the method of the present invention easier to insert into the existing method, the present invention defines the input of the regional prediction module as the proposal embeddings after decoder fusion. The cluster prediction module here is represented by a 2-layer MLP, so that the cluster where the agent's future trajectory endpoint is located can be output, thereby obtaining the multimodal trajectory in the cluster. The present invention defines the input of the regional prediction module as the feature encoding of the proposed trajectory in all regions after decoder fusion decoding. Since the dimension of the proposed trajectory feature encoding is [A, M, F], where A represents the number of agents, M represents the number of regions, and F represents the hidden feature, the present invention also performs a maximum pooling aggregation operation on it in the modal dimension before inputting it into the regional prediction module of the model. The prediction probability of each region and the loss function of the regional prediction head are shown in the following formula:

[0089] x i =f(F i ) (1)

[0090]

[0091] The function f() is implemented using a 2-layer MLP, where Fi is the feature encoding of the proposed trajectory of the i-th vehicle. As the loss function, the present invention uses a cross-entropy function, where the true value of the target region xi is 1 and the value of other regions is 0.

[0092] For the trajectories in the same driving area, they basically have the same trajectory pattern, so the present invention can extract the common features of all future trajectories in a region at the data set level as the regional trajectory prototype prior and introduce it into the subsequent optimization module; Figure 3 As shown in the figure, the clustering model trained by the present invention is shown above. It generates a regional pseudo-label for all trajectories in the dataset, providing sample true values ​​for the regional feature extractor below. The present invention then uses the LSTM model to extract the features of all future trajectories in the region, and selects the hidden features of the last time step as the features of each future trajectory. It then aggregates the features of all future trajectories in the current region by performing a maximum pooling operation on the time dimension, and then splices the optimized trajectory feature encoding to further enhance the effect of the anchor frame. Finally, the whole is input into the optimization module to calculate the bias.

[0093] The method of using the suggested trajectory as an anchor frame to continuously optimize the calculation bias is very popular. The suggested trajectory can be used as a coarse-grained prior trajectory, and the optimization module will perform fine-grained optimization on it. Although this method will improve the trajectories of all features to a certain extent, it will also increase the learning difficulty of the model. In addition, the number of samples of simple trajectories and difficult trajectories is too different, and the samples are too far apart in the feature space. Therefore, the learning of simple trajectories and difficult trajectories by an optimization module will affect the learning of difficult trajectory features to a certain extent. As a result, the previous method will cause the optimization model to sacrifice a certain ability to learn the characteristics of simple behaviors, which will reduce the ability to predict those difficult trajectories that really cause danger. Therefore, the present invention proposes a trajectory optimization structure based on scene complexity: first predicting the difficulty of the trajectory, and then selecting the difficult trajectory for optimization according to the difficulty to solve this problem. The following will introduce the complexity predictor and the complexity-based branch optimization method one by one.

[0094] In a feasible implementation, for the complexity predictor, first, the present invention defines it as a self-supervised task, tests the entire dataset using a baseline, and calculates the overall average fde as a threshold k. Samples with fde greater than k are defined as difficult trajectories, otherwise they are simple samples, thereby defining a difficulty pseudo-label for each trajectory; this complexity predictor, like the previous suggestion module, uses a learnable feature encoding after a maximum pooling operation in the modal dimension as input, but unlike the previous feature encoding, the present invention changes the random initialization to extracting multimodal proposed trajectory features of the region through LSTM, and initializes it with the hidden features of the last time step. These learnable feature encodings based on the proposed trajectory as anchor boxes provide rich contextual information for the difficulty prediction module and provide a clear regional prior for the subsequent optimization module. Then the difficulty prediction module will predict whether the trajectory is simple or difficult, and compare it with the pseudo-label to calculate the loss; wherein, the difficulty prediction module is also only implemented through MLP;

[0095] The difficulty prediction probability of the trajectory and the loss function of the region prediction head are shown in the following formula:

[0096] c i =g(F i ) (3)

[0097]

[0098] The function g() is implemented using a 2-layer MLP, and Fi is the optimized trajectory feature encoding of the i-th vehicle. As the loss function, the present invention also uses the cross-entropy function, where the true value of complex trajectories is 1 and that of simple trajectories is 0.

[0099] After the present invention has trained this offline complexity prediction model, the present invention then performs branch optimization according to the difficulty of the predicted trajectory. If the predicted target vehicle trajectory is a simple trajectory, then the previously obtained recommended trajectory is directly output, because through experiments, for simple trajectories, even without optimizing to calculate the bias, a good prediction effect can be obtained, and the task goal of the present invention is to solve the long-tail trajectory problem, so the present invention hopes that the model can put more losses on the difficult learning trajectory; if the predicted target vehicle trajectory is a difficult trajectory, then the optimized feature encoding based on the recommended trajectory is spliced ​​on the feature dimension, and the feature prototype of the corresponding prediction area obtained by the offline feature extractor is used as the prior anchor frame of the final trajectory to calculate the bias for optimization. In this way, the contextual information of the area can be fully grasped, and the selective optimization strategy can effectively reduce the learning pressure of the optimization module, allowing it to focus on solving those long-tail trajectory problems. Among them, the optimization module here also uses the same structure as the previous baseline model decoder for query decoding.

[0100] Many existing methods use GMM to model trajectory distribution, but it cannot pay enough attention to the tail data. In order to represent the uncertainty of multimodal trajectories, this paper uses a mixed Laplace distribution to represent the distribution of trajectories. The Laplace distribution is steeper than the Gaussian distribution, which makes it more suitable for modeling long-tail data. The probability of extremely large values ​​is much greater than that of the Gaussian distribution, so it can better handle noise and better depict the distribution of long-tail data. The formula is as follows:

[0101]

[0102] pi is the probability of the i-th Laplace distribution model, μx and μy are the means of the probability density of the i-th Laplace distribution model at time step t, and b_x and b_y are scaling factors. Then, the present invention uses Laplace NLLloss as the loss function:

[0103]

[0104] In the task of the present invention, the loss of the regression task can be defined as follows:

[0105]

[0106] N is the number of multimodal trajectories in each region, and k is the complexity weight coefficient. For trajectories predicted as simple by the complexity predictor, k = 0, and for trajectories predicted as complex, k = 1. For the multimodal trajectory confidence classification task, the present invention uses the NLL loss to optimize the Laplace mixing coefficient. This loss function minimizes the negative log-likelihood value in Equation 5.

[0107] When defining the loss function, because trajectory prediction involves multiple tasks, including regression and classification tasks, there is a problem: how to distribute the loss weights of multiple tasks. The simple way is to directly add up the losses of multiple tasks: but the actual situation is that the magnitudes of the loss gradients of different tasks are different, causing some tasks to dominate the gradient backpropagation, and the model to over-learn the task and ignore other tasks. In addition, the inconsistent convergence speeds of different tasks may cause some tasks to be underfitting, while some tasks have been overfitting; manually setting hyperparameters, but because the gradient magnitude and convergence speed of each task also change dynamically during the training process, it is obvious that this fixed-value weight definition method does not solve the problem well; therefore, in order to solve the problem of uneven weight distribution of multiple tasks, the present invention introduces the GradNorm method into the trajectory prediction task for the first time, which can dynamically adjust the algorithm of multi-task loss weights; the overall loss function of the task can be defined as:

[0108]

[0109] The weights w1 and w2 are automatically updated using the GradNorm algorithm. The update process is as follows:

[0110]

[0111] Where Li is the gradient norm of each task and ri is the loss weight adjustment factor.

[0112] S4: Output trajectory prediction and trajectory optimization data to complete autonomous driving trajectory prediction based on the new suggestion-optimization framework.

[0113] In a feasible implementation, in order to demonstrate the effectiveness of the proposed method, the present invention verifies the effectiveness of the method of the present invention through three baseline models on the two data sets of Argoverse1 and Argoverse2 (represented by LTPR in the table), and compares it with other best methods for solving the long-tail problem, mainly including: data resampling, loss reweighting, and comparative learning. In order to further evaluate the performance of the model of the present invention on long-tail data, the present invention also divides the data set into a variety of different levels of complexity. The present invention uses a baseline model to evaluate the min-FDE6 indicator, and sets trajectories of different complexities according to the size of the indicator value. Specifically, the present invention selects the top 1-4% of the trajectories with the highest indicator errors as complex scenarios for testing, and regards the 30% with the lowest errors as simple trajectories for testing. Finally, the performance of the model in the entire data set is evaluated accordingly.

[0114] Table 1 Comparison of various methods for solving long-tail problems on the Argoverse2 dataset using QCNet as the baseline model

[0115]

[0116] Table 1 shows a comparison of the proposed method with other common approaches to solving the long-tail problem, using QCNet as the baseline model. The proposed method performs best under long-tail data of various complexity levels, while maintaining the same overall average error compared to the baseline model.

[0117] We also applied our method to two other baseline models and compared them with the best contrastive learning long-tail prediction method. The comparison results are shown in Tables 2 and 3. These experimental results demonstrate that our method outperforms contrastive learning on both the Argoverse1 and Argoverse2 datasets at varying complexity levels, performing better than both baseline models.

[0118] Table 2 Comparison of the best methods using Forecast-MAE as the baseline model on the Argoverse2 dataset

[0119]

[0120] Table 3 Comparison of the best methods using HiVT as the baseline model on the Argoverse1 dataset

[0121]

[0122] As shown in Table 4, we conducted ablation experiments on the Argoverse2 dataset using QCNet as the baseline model to further validate the effectiveness of each module. The Clu module represents the region-based trajectory proposal generation module, and the Com module represents the scene complexity-based trajectory optimization module. Three observations emerge from the results. First, when the proposal module alone is used without the optimization module, the prediction performance for simple trajectories with a complexity of the bottom 30% is almost the same as without it. Therefore, for simple trajectories, adding the optimization module does not provide significant improvement. Second, after integrating the Com module, we observed improvements in trajectory prediction accuracy for scenes with top 1%, 2%, 3%, and 4% complexity compared to the baseline model. This demonstrates that the optimization module effectively learns the characteristics of complex scenes and avoids interference from simple trajectories. Third, the addition of the Clu module not only enhances the prediction capability for complex scenarios, but also improves performance across the entire dataset. This demonstrates that the region-based trajectory selection and optimization approach enables the output head to learn the characteristics of each region without interfering with each other, and provides the optimization module with anchor boxes with rich regional context.

[0123] Table 4 Ablation experiments of each module

[0124]

[0125] Figure 4 The visualization of prediction results using QCnet, QCNet+Contrastive Learning (the previous best long-tail prediction method) and QCNet+LTPR (the method of the present invention) is shown. The first four columns plot the top 1-4% challenging scenarios with complex road topology and interactions. It can be seen that the trajectories predicted by the method of the present invention are significantly better than QCNet+Contrastive Learning and the baseline model QCNet. In addition, the last column is the prediction result of a simple scenario, which shows that the prediction ability of the model of the present invention is basically consistent with the baseline model, while the contrastive learning performs poorly. This is due to the complexity-based optimization module of the present invention, which enables the optimization module to focus on learning the features of complex scenes and prevent the confusion of features of simple and complex trajectories. This enables the present invention to almost maintain the prediction performance for simple scenarios while effectively improving the model's prediction performance for long-tail data.

[0126] In order to better demonstrate the role of the region-based proposed trajectory generation module of the present invention, the present invention uses QCNet as the baseline model and compares the effects of the module with and without it. Figure 5The left picture is after adding, and the present invention displays all six multimodal suggested trajectories for each area in the picture, where the actual trajectory is represented by pink, and the predicted trajectories of different areas are represented by different colors respectively. The right picture is the effect picture without adding, and for the sake of fairness, the present invention allows the baseline model to also generate 36 trajectories, and the predicted trajectories are represented by yellow. It can be seen from the visualization diagram that when this module is not added, the coverage area of ​​the multimodal trajectory is very limited; after adding, the regional multimodal trajectory of the present invention reasonably covers all possible driving areas, and each regional output head learns the characteristics of that area very well, without being affected by different areas, and only outputs trajectories in its own area.

[0127] In an embodiment of the present invention, a new suggestion-optimization framework is proposed: the present invention introduces a clustering-based suggestion trajectory generation module, uses region partition predictions and region trajectory prototypes as a priori anchor boxes for further refinement, and a complexity-based optimization module for distinguishing simple and complex trajectories, allowing the optimization module to focus on learning the features of complex trajectories, thereby improving the prediction accuracy of long-tail scenarios.

[0128] The proposed method can be easily combined with most existing methods to improve their performance in challenging scenarios.

[0129] Experimental results show that the proposed framework improves the performance of three baseline models on the long-tail scenarios of the Argoverse1 and Argoverse2 datasets and outperforms the existing best methods while maintaining the prediction indicators on the overall dataset.

[0130] The present invention also integrates the GradNorm algorithm for automatically calculating loss weights of multiple tasks into the trajectory prediction direction, thereby better solving the problem of uneven weight distribution of multiple tasks.

[0131] Figure 6 This is a block diagram of an autonomous driving trajectory prediction system based on a suggestion-optimization framework according to an exemplary embodiment. The system is used for an autonomous driving trajectory prediction method based on a suggestion-optimization framework. Figure 6 The system includes a data framework construction module 310, a data processing module 320, a trajectory prediction optimization module 330, a data output module 340, and a resource allocation module 350. Among them:

[0132] A framework construction unit 310 is used to construct a novel suggestion-optimization framework;

[0133] The data processing unit 320 is used to obtain a learnable feature code, input historical data into a preset baseline model, and output decoded data through the baseline model;

[0134] The trajectory prediction and optimization unit 330 is used to input the decoded data into the novel suggestion-optimization framework to perform trajectory prediction and trajectory optimization on the decoded data;

[0135] The data output unit 340 is used to output trajectory prediction and trajectory optimization data to complete the autonomous driving trajectory prediction based on the new suggestion-optimization framework.

[0136] Optionally, the framework construction unit 310 is configured to construct a novel suggestion-optimization framework having two core modules;

[0137] Among them, the two core modules include: a proposed trajectory generation module based on clustering areas and a trajectory optimization module based on scene complexity;

[0138] The proposed trajectory generation module based on clustered regions includes: a proposed trajectory output head and a region predictor; the trajectory optimization module based on scene complexity includes: a complexity predictor and an optimized trajectory output head.

[0139] Optionally, the data processing unit 320 is configured to use the encoder of the baseline model to encode and fuse the historical trajectory information, scene map information, and surrounding vehicle interaction features of the target vehicle, and input all anchor-free suggested trajectory feature encodings into the decoder for query decoding.

[0140] Optionally, the trajectory prediction optimization unit 330 is configured to input the decoded data into the novel suggestion-optimization framework;

[0141] Train an offline clustering model to divide the regions;

[0142] Collect the endpoints of all true trajectories from the dataset and cluster them into M clusters using K-means.

[0143] The clustered region-based proposed trajectory generation module outputs the decoded and updated proposed trajectory feature encoding according to the region, generates multimodal proposed trajectories in each region, and predicts the most likely region through the region predictor;

[0144] The proposed trajectory in the prediction area is converted into a feature encoding as an anchor box and input into the trajectory optimization module based on scene complexity to predict the trajectory difficulty for branch optimization. Only difficult trajectories are optimized and simple proposed trajectories are directly output. Before optimization, the trajectory prototype of the prediction area is introduced as the regional prior.

[0145] Optionally, a trace output header is recommended, including:

[0146] The proposed trajectory output head is represented by M MLPs, each of which is responsible for outputting K multimodal trajectories within a region and the probability of each trajectory.

[0147] Optionally, before optimization, the trajectory prototype of the prediction area is introduced as a regional prior, including:

[0148] At the dataset level, the common features of all future trajectories in a region are extracted as regional trajectory prototype priors;

[0149] Generate a regional pseudo label for all trajectories in the dataset through the trained clustering model; the regional pseudo label provides the sample truth value for the regional feature extractor;

[0150] The LSTM model is used to extract the features of all future trajectories in the region, and the hidden features of the last time step are selected as the features of each future trajectory.

[0151] Aggregate the features of all future trajectories in the current area by performing a maximum pooling operation on the time dimension;

[0152] The feature prototypes of all future trajectories extracted from the current area are concatenated with the corresponding optimized trajectory feature codes to enhance the effect of the anchor frame. The concatenated prototypes are then input into the optimization module for bias calculation.

[0153] Optionally, a trajectory optimization module based on scene complexity includes:

[0154] We define scene complexity prediction as a self-supervised task, test the entire dataset using a baseline, and define a difficulty pseudo-label for each trajectory.

[0155] The learnable feature encoding after the maximum pooling operation in the modal dimension is used as input, and the multimodal trajectory suggestion features of the region are extracted through LSTM, and initialized with the hidden features of the last time step;

[0156] The difficulty prediction module is used to predict the difficulty of the trajectory, and the prediction result is compared with the difficulty pseudo-label to calculate the loss.

[0157] The proposed method can be easily combined with most existing methods to improve their performance in challenging scenarios.

[0158] Experimental results show that the proposed framework improves the performance of three baseline models on the long-tail scenarios of the Argoverse1 and Argoverse2 datasets and outperforms the existing best methods while maintaining the prediction indicators on the overall dataset.

[0159] The present invention also integrates the GradNorm algorithm for automatically calculating loss weights of multiple tasks into the trajectory prediction direction, thereby better solving the problem of uneven weight distribution of multiple tasks.

[0160] Figure 5 is a structural diagram of an autonomous driving trajectory prediction device based on a suggestion-optimization framework provided by an embodiment of the present invention, such as Figure 7 As shown, the autonomous driving trajectory prediction device based on the suggestion-optimization framework may include the above Figure 6 The autonomous driving trajectory prediction device based on the suggestion-optimization framework is shown. Optionally, the autonomous driving trajectory prediction device 410 based on the suggestion-optimization framework may include a first processor 2001.

[0161] Optionally, the autonomous driving trajectory prediction device 410 based on the suggestion-optimization framework may further include a memory 2002 and a transceiver 2003 .

[0162] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0163] The following combination Figure 7 The components of the autonomous driving trajectory prediction device 410 based on the suggestion-optimization framework are described in detail:

[0164] The first processor 2001 is the control center of the autonomous driving trajectory prediction device 410 based on the suggestion-optimization framework, and can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs) or one or more field programmable gate arrays (FPGAs).

[0165] Optionally, the first processor 2001 can perform various functions of the autonomous driving trajectory prediction device 410 based on the suggestion-optimization framework by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0166] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 7CPU0 and CPU1 are shown in FIG.

[0167] In a specific implementation, as an embodiment, the autonomous driving trajectory prediction device 410 based on the suggestion-optimization framework may also include multiple processors, such as Figure 7 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0168] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0169] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently and accessed through the interface circuit ( Figure 7 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0170] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0171] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 7 (not shown separately in the figure). The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0172] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and communicate with the autonomous driving trajectory prediction device 410 based on the suggestion-optimization framework through an interface circuit ( Figure 7 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0173] It should be noted that Figure 7 The structure of the autonomous driving trajectory prediction device 410 based on the suggestion-optimization framework shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0174] In addition, the technical effects of the autonomous driving trajectory prediction device 410 based on the suggestion-optimization framework can refer to the technical effects of the autonomous driving trajectory prediction method based on the suggestion-optimization framework described in the above method embodiment, and will not be repeated here.

[0175] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0176] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0177] The above embodiments can be implemented in whole or in part through software, hardware (such as circuits), firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired method (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, or magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0178] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0179] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0180] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0181] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0182] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0183] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0184] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0185] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0186] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0187] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for autonomous driving trajectory prediction based on a suggestion-optimization framework, characterized in that: The method comprises: S1. Construct a new suggestion-optimization framework; In S1, a new suggestion-optimization framework is constructed, including: Build a novel suggestion-optimization framework with two core modules; The two core modules include: a proposed trajectory generation module based on clustering areas and a trajectory optimization module based on scene complexity; The proposed trajectory generation module based on clustered regions includes: a proposed trajectory output head and a region predictor; the trajectory optimization module based on scene complexity includes: a complexity predictor and an optimized trajectory output head; S2. Obtain a learnable feature code, input the learnable feature code into a preset baseline model, and output decoded data through the baseline model; S3, inputting the decoded data into the novel suggestion-optimization framework, and performing trajectory prediction and trajectory optimization on the decoded data; The step S3, inputting the decoded data into the suggestion-optimization framework, and performing trajectory prediction and trajectory optimization on the decoded data, includes: inputting the decoded data into the novel suggestion-optimization framework; Train an offline clustering model to divide the regions; Collect the endpoints of all true trajectories from the dataset and cluster them into M clusters using K-means. The clustered region-based proposed trajectory generation module outputs the decoded and updated proposed trajectory feature encoding according to the region, generates multimodal proposed trajectories in each region, and predicts the most likely region through the region predictor; The proposed trajectory in the prediction area is converted into a feature code as an anchor box and input into the trajectory optimization module based on scene complexity to predict the trajectory difficulty for branch optimization. Only difficult trajectories are optimized and simple proposed trajectories are directly output. Before optimization, the trajectory prototype of the prediction area is introduced as the regional prior. S4: Output trajectory prediction and trajectory optimization data to complete autonomous driving trajectory prediction based on the new suggestion-optimization framework.

2. The autonomous driving trajectory prediction method based on the suggestion-optimization framework according to claim 1, characterized in that: In S2, a learnable feature code is obtained, the learnable feature code is input into a preset baseline model, and decoded data is output through the baseline model, including: The encoder of the baseline model is used to encode and fuse the historical trajectory information of the target vehicle, scene map information, and the interaction features of surrounding vehicles, and all anchor-free proposed trajectory feature encodings are input into the decoder for query decoding.

3. The autonomous driving trajectory prediction method based on the suggestion-optimization framework according to claim 2, characterized in that: The proposed trajectory output header includes: The proposed trajectory output head is represented by M MLPs, each of which is responsible for outputting K multimodal trajectories within a region and the probability of each trajectory.

4. The autonomous driving trajectory prediction method based on the suggestion-optimization framework according to claim 3 is characterized in that: The method of introducing the trajectory prototype of the prediction area as the area prior before the optimization includes: At the dataset level, the common features of all future trajectories in a region are extracted as regional trajectory prototype priors; Generate a regional pseudo label for all trajectories in the dataset through the trained clustering model; the regional pseudo label provides the sample truth value for the regional feature extractor; The LSTM model is used to extract the features of all future trajectories in the region, and the hidden features of the last time step are selected as the features of each future trajectory. Aggregate the features of all future trajectories in the current area by performing a maximum pooling operation on the time dimension; The feature prototypes of all future trajectories extracted from the current area are concatenated with the corresponding optimized trajectory feature codes to enhance the effect of the anchor frame. The concatenated prototypes are then input into the optimization module for bias calculation.

5. The autonomous driving trajectory prediction method based on the suggestion-optimization framework according to claim 4 is characterized in that: The trajectory optimization module based on scene complexity includes: We define scene complexity prediction as a self-supervised task, test the entire dataset using a baseline, and define a difficulty pseudo-label for each trajectory. The learnable feature encoding after the maximum pooling operation in the modal dimension is used as input, and the multimodal trajectory suggestion features of the region are extracted through LSTM, and initialized with the hidden features of the last time step; The difficulty prediction module predicts the difficulty of the trajectory, and the prediction result is compared with the difficulty pseudo label to calculate the loss.

6. An autonomous driving trajectory prediction system based on a suggestion-optimization framework, wherein the autonomous driving trajectory prediction system based on the suggestion-optimization framework is used to implement the autonomous driving trajectory prediction method based on the suggestion-optimization framework according to any one of claims 1 to 5, characterized in that: The system comprises: Framework building unit, used to build a new suggestion-optimization framework; a data processing unit, configured to obtain a learnable feature code, input the learnable feature code into a preset baseline model, and output decoded data through the baseline model; a trajectory prediction and optimization unit, configured to input the decoded data into the novel suggestion-optimization framework and perform trajectory prediction and trajectory optimization on the decoded data; The data output unit is used to output trajectory prediction and trajectory optimization data to complete autonomous driving trajectory prediction based on the new suggestion-optimization framework.

7. An autonomous driving trajectory prediction device based on a suggestion-optimization framework, characterized in that: The autonomous driving trajectory prediction device based on the suggestion-optimization framework includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 5.

Images