Method for predicting tracks of various traffic participants in autonomous driving system based on flexible anchor points

Abstract

The invention provides a track prediction method for various traffic participants in an autonomous driving system based on flexible anchor points, which comprehensively absorbs the advantages of two target track prediction modes without anchor points and based on anchor points, and provides accurate track prediction by modeling and fusing track data to process scene information and agent behaviors. Meanwhile, local and global attention mechanisms are introduced to improve the understanding ability of the model for complex scenes; and finally, generating a more refined and accurate query vector based on a Transform and a multi-layer perceptron, and obtaining a final prediction trajectory. Compared with the prior art, the method gets rid of excessive dependence on a complex physical model, so that the calculation cost and the requirement for related hardware resources are effectively reduced.

Description

Technical Field

[0001] The present invention belongs to the field of perception and processing technology of vehicle-mounted autonomous driving systems, and specifically aims at future trajectory prediction technology of traffic participants in autonomous driving scenarios and related processing methods. Background Art

[0002] At present, the trajectory prediction of traffic participants in autonomous driving scenarios is mainly based on learning methods, which can provide good efficiency and adaptability for various objects such as vehicles, pedestrians and cyclists. The existing learning-based traffic participant trajectory prediction methods are mainly divided into two categories: process-based trajectory prediction methods and planning-based trajectory prediction methods. The former predicts by simulating the future trajectory generation process. For example, the MotionDiffuser model based on diffusion representation is used for regression to directly generate a complete trajectory. No anchor points are used in the process, so it has good flexibility. However, when outputting multiple possible trajectories, it is easy to cause mode collapse, resulting in a single generated trajectory. At the same time, due to the distribution complexity in the training process, the stability of the model is poor and the dependence on the optimization algorithm is high. The latter method usually uses key points in the target trajectory to generate a complete trajectory. For example, the DenseTNT model constrains the trajectory through predefined anchor points and is considered to be a typical anchor-based prediction method. However, the performance of this type of method is heavily dependent on the accuracy and coverage of the anchor point setting. If the anchor point distribution is not reasonable or the number is insufficient, the prediction result may lack robustness and it is difficult to cope with the diverse needs in complex scenarios. It can be seen that the existing technologies either lack control over the prediction process, which easily leads to uncertainty in the prediction results; or they rely too much on anchor points, which limits their application in dynamic and complex scenarios. It is currently difficult to balance flexibility and stability, and therefore cannot meet the needs of high-precision trajectory prediction. Summary of the invention

[0003] In view of this, in order to solve the technical problems existing in the art, the present invention provides a method for predicting trajectories of various traffic participants in an autonomous driving system based on flexible anchor points, which specifically includes the following steps:

[0004] Step 1: Receive the historical trajectory data of different traffic participants, the scene data consisting of road vectors, obstacle positions and other information, and the corresponding time data, and use these three types of data to construct an input feature matrix X with dimensions (N, F, T) to represent the historical trajectory sequence of the intelligent agent to be tested; where N represents the number of traffic participants, F is the feature dimension of each traffic participant, and T is the time dimension;

[0005] Step 2: Use a vector group encoder based on a multi-layer perceptron (MLP) to process and model the input feature matrix X to generate a latent space representation of each agent to be tested, which contains a high-dimensional vector of information such as position, velocity, acceleration, etc.; and obtain the corresponding target scene modeling output, including a parameter vector used to describe the behavior pattern of the target agent and the environmental interaction characteristics; perform similar processing on the scene data therein to generate the corresponding target scene modeling output, which contains feature vectors of information such as static obstacle positions, dynamic obstacle trajectories, and traffic signal status; these target scene modeling outputs and scene modeling outputs will be used as inputs in the subsequent trajectory prediction and decision-making steps to improve prediction accuracy and the rationality of decision-making;

[0006] Step 3: After filtering the input feature matrix X and removing invalid elements, the corresponding input feature tensor is obtained, and the local attention and global attention mechanisms are introduced to process it to capture the key local features and global interaction information around the agent;

[0007] Step 4: Take the key local features and global interaction information obtained in step 3 as input, fuse them through the Transformer algorithm and output the corresponding query vector Q′;

[0008] Step 5: Input the historical trajectory data of the agent, the current state including position, velocity and acceleration, and the scene data into the multi-layer perceptron MLP, so that it can integrate the historical trajectory and the agent behavior to output a set of dynamic anchor points A = {A1, A2, ..., A K},

[0009] Step 6: Calculate the contribution of each anchor point by analyzing the similarity between the query vector Q obtained in step 4 and the dynamic anchor point obtained in step 5, assign a corresponding weight to each anchor point based on the contribution, and weightedly fuse the anchor points to obtain the refined query vector Q′;

[0010] Step 7: Input the query vector Q′ obtained in step 6 into the multi-layer perceptron MLP, and obtain the final trajectory prediction result T after passing through the hidden layer and the output layer.

[0011] Furthermore, the process of processing and modeling the input feature matrix X in step 2 specifically includes:

[0012] (1) First-layer processing: The input feature matrix X is processed by the first layer of MLP to generate the preliminary latent space representation Z1 of the target;

[0013] (2) Layer-by-layer deep feature learning: After L layers of MLP structure, deeper features are extracted layer by layer, and finally a latent space representation Z is generated, which contains the temporal information of the target and its relationship with the scene;

[0014] (3) Projection output: Generate the target scene modeling output Y through the projection layer a :

[0015] Y a =Projection(Z)

[0016] The scene data is processed similarly to generate the scene modeling output Y s .

[0017] Furthermore, in step 3, local attention is specifically defined as:

[0018] H l =Attention l (X filtered ,P,N)

[0019] Attention l The specific formula is:

[0020]

[0021] Among them, X filtered is the input feature tensor generated after filtering invalid elements, P is the position embedding tensor, N is the neighbor index pair tensor, and W Q is the weight matrix of the query vector, which is used to transform the input features into the query vector; W K is the weight matrix of the key vector, which is used to transform the input features into the key vector; W V is the weight matrix of the value vector, which is used to transform the input features into the value vector; d k is the dimension of the query vector and the key vector, which is used to normalize the dot product result between the query vector and the key vector to avoid numerical instability and ensure the stability of the attention calculation;

[0022] Global attention is defined as:

[0023] H g =Attention g (H l ,P c )

[0024] Among them, P c It is the global context location information.

[0025] Furthermore, in step 5, the dynamic anchor point set A is generated through the following K-means clustering process:

[0026] (1) Collect trajectory data: Obtain a dataset containing N historical trajectories, each of which consists of two-dimensional coordinate points with T time steps:

[0027]

[0028] Each track X i It is expressed as:

[0029] X i ={(x i,1 ,y i,1 ),(x i,2 ,y i,2 ),…,(x i,T ,y i,T )}

[0030] Flatten all trajectory points into a two-dimensional array D for clustering:

[0031] D={d1,d2,…,d M},d m =(x m ,y m ),M=N×T

[0032] (2) Initialize K-means clustering:

[0033] Select the number of anchor points K;

[0034] Initialize the centroid: Randomly select K data points as the initial centroid

[0035] (3) Assign data points to the nearest centroid:

[0036] Calculate the distance: For each data point d m , calculate its relationship with each centroid μ k The Euclidean distance of:

[0037] ||d m -μ k || 2 =(x m -μ k,x ) 2 +(y m -μ k,y ) 2

[0038] Assign clusters: Assign each data point d m Assign to the cluster C with the nearest centroid k :

[0039]

[0040] (4) Update the centroid:

[0041] Recalculate the centroid: For each cluster C k , calculate the new center of mass μ k :

[0042]

[0043] Repeat iteration: Repeat steps (3) and (4) until the position of the centroid no longer changes significantly or the preset maximum number of iterations is reached;

[0044] (5) Generate anchor point set:

[0045] Determine the final centroid: When the K-means algorithm converges, the final centroid μ k As an anchor point:

[0046] A k =μ k

[0047] Construct an anchor set:

[0048] A={A1,A2,…,A k}.

[0049] Furthermore, the specific process of obtaining the refined query vector Q′ in step 6 includes:

[0050] (1) Calculate the Euclidean distance: measure the actual distance between two vectors in space. The smaller the distance, the higher the similarity:

[0051]

[0052] (2) Calculate similarity: The query vector is Q, and the dynamic anchor point set is A = {A1, A2, ..., A K}, where K = 32, each anchor point A k For a d-dimensional vector, the cosine similarity is used to calculate the similarity between the query vector and each anchor point:

[0053]

[0054] Among them, s k Represents the similarity score between the query vector and the kth anchor point; the contribution of each anchor point is determined based on the similarity, and a corresponding weight is assigned to each anchor point based on the contribution, and the refined query vector Q′ is obtained by weighted fusion of each anchor point:

[0055]

[0056] Among them, w k is the weight determined based on the similarity between the query vector and each anchor point.

[0057] Furthermore, in step 7, the final trajectory prediction result T is obtained based on the following activation function:

[0058] T=W of(Q′)+b o

[0059] Among them, W o and b o are the weight matrix and bias vector of the output layer respectively.

[0060] The method for predicting the trajectories of various traffic participants in an autonomous driving system based on flexible anchor points provided by the present invention above fully absorbs the advantages of both anchor-free and anchor-based target trajectory prediction methods, and provides accurate trajectory prediction by integrating scene information and intelligent agent behavior through modeling of trajectory data; at the same time, it introduces local and global attention mechanisms to improve the model's ability to understand complex scenes; finally, it generates a more refined and accurate query vector based on Transformer and multi-layer perceptron and obtains the final predicted trajectory. Compared with the existing technology, this method gets rid of the excessive reliance on complex physical models, thereby effectively reducing the computing cost and the demand for related hardware resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of the trajectory prediction process of the method provided by the present invention;

[0062] Figure 2 It is a closed-loop schematic diagram of real vehicle training verification and data collection based on the present invention;

[0063] Figure 3 It is a schematic diagram of the trajectory prediction effect based on the present invention. DETAILED DESCRIPTION

[0064] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] The method for predicting trajectories of various traffic participants in an autonomous driving system based on flexible anchor points provided by the present invention is as follows: Figure 1 As shown, the specific steps include:

[0066] Step 1: Receive the historical trajectory data of different traffic participants, the scene data consisting of road vectors, obstacle positions and other information, and the corresponding time data, and use these three types of data to construct an input feature matrix X with dimensions (N, F, T) to represent the historical trajectory sequence of the intelligent agent to be tested; where N represents the number of traffic participants, F is the feature dimension of each traffic participant, and T is the time dimension;

[0067] Step 2: Use a vector group encoder based on a multi-layer perceptron (MLP) to process and model the input feature matrix X to generate a latent space representation of each agent to be tested, which contains a high-dimensional vector of information such as position, velocity, acceleration, etc.; and obtain the corresponding target scene modeling output, including a parameter vector used to describe the behavior pattern of the target agent and the environmental interaction characteristics; perform similar processing on the scene data therein to generate the corresponding target scene modeling output, which contains feature vectors of information such as static obstacle positions, dynamic obstacle trajectories, and traffic signal status; these target scene modeling outputs and scene modeling outputs will be used as inputs in the subsequent trajectory prediction and decision-making steps to improve prediction accuracy and the rationality of decision-making;

[0068] Step 3: After filtering the input feature matrix X and removing invalid elements, the corresponding input feature tensor is obtained, and the local attention and global attention mechanisms are introduced to process it to capture the key local features and global interaction information around the agent;

[0069] Step 4: Take the key local features and global interaction information obtained in step 3 as input, fuse them through the Transformer algorithm and output the corresponding query vector Q;

[0070] Step 5: Input the historical trajectory data of the agent, the current state including position, velocity and acceleration, and the scene data into the multi-layer perceptron MLP, so that it can integrate the historical trajectory and the agent behavior to output a set of dynamic anchor points A = {A1, A2, ..., A K},

[0071] Step 6: Calculate the contribution of each anchor point by analyzing the similarity between the query vector Q obtained in step 4 and the dynamic anchor point obtained in step 5, assign a corresponding weight to each anchor point based on the contribution, and weightedly fuse the anchor points to obtain the refined query vector Q′;

[0072] Step 7: Input the query vector Q′ obtained in step 6 into the multi-layer perceptron MLP, and obtain the final trajectory prediction result T after passing through the hidden layer and the output layer.

[0073] In a preferred embodiment of the present invention, the process of processing and modeling the input feature matrix X in step 2 specifically includes:

[0074] (1) First-layer processing: The input feature matrix X is processed by the first layer of MLP to generate the preliminary latent space representation Z1 of the target;

[0075] (2) Layer-by-layer deep feature learning: After L layers of MLP structure, deeper features are extracted layer by layer, and finally a latent space representation Z is generated, which contains the temporal information of the target and its relationship with the scene;

[0076] (3) Projection output: Generate the target scene modeling output Y through the projection layer a :

[0077] Y a =Projection(Z)

[0078] The scene data is processed similarly to generate the scene modeling output Y s .

[0079] In a preferred embodiment of the present invention, the local attention is specifically defined in step 3 as:

[0080] H l =Attention l (X filtered ,P,N)

[0081] Attention l The specific formula is:

[0082]

[0083] Among them, X filtered is the input feature tensor generated after filtering invalid elements, P is the position embedding tensor, N is the neighbor index pair tensor, and W Q is the weight matrix of the query vector, which is used to transform the input features into the query vector; W K is the weight matrix of the key vector, which is used to transform the input features into the key vector; W V is the weight matrix of the value vector, which is used to transform the input features into the value vector; d k is the dimension of the query and key vectors, used to normalize the dot product between the query and the key to avoid numerical instability and ensure the stability of the attention calculation;

[0084] Global attention is defined as:

[0085] H g =Attention g (H l ,P c )

[0086] Among them, P c It is the global context location information.

[0087] In a preferred embodiment of the present invention, in step 5, the dynamic anchor point set A is generated by the following K-means clustering process:

[0088] (1) Collect trajectory data: Obtain a dataset containing N historical trajectories, each of which consists of two-dimensional coordinate points with T time steps:

[0089]

[0090] Each track X i It is expressed as:

[0091] X i ={(x i,1 ,y i,1 ),(x i,2 ,y i,2 ),…,(x i,T ,y i,T )}

[0092] Flatten all trajectory points into a two-dimensional array D for clustering:

[0093] D={d1,d2,…,d M},d m =(x m ,y m ),M=N×T

[0094] (2) Initialize K-means clustering:

[0095] Select the number of anchor points: Set the number of anchor points K = 32;

[0096] Initialize the centroid: Randomly select K data points as the initial centroid

[0097] (3) Assign data points to the nearest centroid:

[0098] Calculate the distance: For each data point d m , calculate its relationship with each centroid μ k The Euclidean distance of:

[0099] ||d m -μ k || 2 =(x m -μ k,x ) 2 +(y m -μ k,y ) 2

[0100] Assign clusters: Assign each data point d m Assign to the cluster C with the nearest centroid k :

[0101]

[0102] (4) Update the centroid:

[0103] Recalculate the centroid: For each cluster C k , calculate the new center of mass μk :

[0104]

[0105] Repeat iteration: Repeat steps (3) and (4) until the position of the centroid no longer changes significantly or the preset maximum number of iterations is reached;

[0106] (5) Generate anchor point set:

[0107] Determine the final centroid: When the K-means algorithm converges, the final centroid μ k As an anchor point:

[0108] A k =μ k ,k=1,2,…,32

[0109] Construct an anchor set:

[0110] A={A1,A2,…,A 32}.

[0111] In a preferred embodiment of the present invention, the specific process of obtaining the refined query vector Q′ in step 6 includes:

[0112] (1) Calculate the Euclidean distance: measure the actual distance between two vectors in space. The smaller the distance, the higher the similarity:

[0113]

[0114] (2) Calculate similarity: The query vector is Q, and the dynamic anchor point set is A = {A1, A2, ..., A K}, where K = 32, each anchor point A k For a d-dimensional vector, the cosine similarity is used to calculate the similarity between the query vector and each anchor point:

[0115]

[0116] Among them, s k Represents the similarity score between the query vector and the kth anchor point; the contribution of each anchor point is determined based on the similarity, and a corresponding weight is assigned to each anchor point based on the contribution, and the refined query vector Q′ is obtained by weighted fusion of each anchor point:

[0117]

[0118] Among them, w k is the weight determined based on the similarity between the query vector and each anchor point.

[0119] In a preferred embodiment of the present invention, in step seven, the final trajectory prediction result T is obtained based on the following activation function:

[0120] T=W o f(Q′)+b o

[0121] Among them, W o and b o are the weight matrix and bias vector of the output layer respectively.

[0122] Figure 2 The actual vehicle training verification and data closed-loop collection process based on the present invention is shown. Specifically, the prediction model is first trained in a virtual environment and then applied to the actual vehicle-mounted automatic driving system. The signals collected in real time by the sensor are used to predict the trajectories of various traffic participants, and the actuators on the vehicle are controlled to perform corresponding actions.

[0123] Figure 3 The prediction results of target trajectories in different scenarios based on the examples of the present invention are shown.

[0124] It should be understood that the size of the serial numbers of the steps in the embodiment of the present invention does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0125] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for predicting trajectories of diverse traffic participants in an autonomous driving system based on flexible anchor points, characterized by: The specific steps include: Step 1: Receive the historical trajectory data of different traffic participants, the scene data consisting of road vectors, obstacle location information, and the corresponding time data, and use these three types of data to construct an input feature matrix X with dimensions (N, F, T) to represent the historical trajectory sequence of the intelligent agent to be tested; where N represents the number of traffic participants, F is the feature dimension of each traffic participant, and T is the time dimension; Step 2: Use the vector group encoder based on multi-layer perceptron MLP to process and model the input feature matrix X to generate a latent space representation of each agent to be tested, which contains a high-dimensional vector of position, velocity, and acceleration information; and obtain the corresponding target scene modeling output, including a parameter vector used to describe the behavior pattern of the target agent and the environmental interaction characteristics; perform similar processing on the scene data therein to generate the corresponding target scene modeling output, including feature vectors of static obstacle positions, dynamic obstacle trajectories, and traffic signal status information; Step 3: After filtering the input feature matrix X and removing invalid elements, the corresponding input feature tensor is obtained, and the local attention and global attention mechanisms are introduced to process it to capture the key local features and global interaction information around the agent; Step 4: Take the key local features and global interaction information obtained in step 3 as input, fuse them through the Transformer algorithm and output the corresponding query vector Q; Step 5: Input the historical trajectory data of the agent, the current state including position, velocity and acceleration, and the scene data into the multi-layer perceptron MLP, so that it can integrate the historical trajectory and the agent behavior to output a set of dynamic anchor points A = {A1, A2, ..., A K }; Step 6: Calculate the contribution of each anchor point by analyzing the similarity between the query vector Q obtained in step 4 and the dynamic anchor point obtained in step 5, assign a corresponding weight to each anchor point based on the contribution, and weightedly fuse the anchor points to obtain the refined query vector Q′; Step 7: Input the query vector Q′ obtained in step 6 into the multi-layer perceptron MLP, and obtain the final trajectory prediction result T after passing through the hidden layer and the output layer.

2. The method according to claim 1, characterized in that: The process of processing and modeling the input feature matrix X in step 2 specifically includes: (1) First-layer processing: The input feature matrix X is processed by the first layer of MLP to generate the preliminary latent space representation Z1 of the target; (2) Layer-by-layer deep feature learning: After L layers of MLP structure, deeper features are extracted layer by layer, and finally a latent space representation Z is generated, which contains the temporal information of the target and its relationship with the scene; (3) Projection output: Generate the target scene modeling output Y through the projection layer a : Y a =Projection(Z) The scene data is processed similarly to generate the scene modeling output Y s .

3. The method according to claim 2, characterized in that: In step 3, local attention is specifically defined as: H l =Attention l (X filtered ,P,N) Attention l The specific formula is: Among them, X filtered is the input feature tensor generated after filtering invalid elements, P is the position embedding tensor, N is the neighbor index pair tensor; W Q is the weight matrix of the query vector, which is used to transform the input features into the query vector; W K is the weight matrix of the key vector, which is used to transform the input features into the key vector; W V is the weight matrix of the value vector, which is used to transform the input features into the value vector; d k is the dimension of the query vector and key vector, used to normalize the dot product result between the query vector and the key vector; Global attention is defined as: H g =Attention g (H l ,P c ) Among them, P c It is the global context location information.

4. The method according to claim 2, characterized in that: In step 5, the dynamic anchor point set A is generated through the following K-means clustering process: (1) Collect trajectory data: Obtain a dataset containing N historical trajectories, each of which consists of two-dimensional coordinate points with T time steps: Each track X i It is expressed as: X i ={(x i,1 ,t i,1 ),(x i,2 ,t i,2 ),…,(x i,T ,y i,T )} Flatten all trajectory points into a two-dimensional array D for clustering: D={d1,d2,…,d M },d m =(x m ,y m ),M=N×T (2) Initialize K-means clustering: Select the number of anchor points K; Initialize the centroid: Randomly select K data points as the initial centroid (3) Assign data points to the nearest centroid: Calculate the distance: For each data point d m , calculate its relationship with each centroid μ k The Euclidean distance of: ||d m -m k || 2 =(x m -m k,x ) 2 +(y m -m k,y ) 2 Assign clusters: Assign each data point d m Assign to the cluster C with the nearest centroid k : (4) Update the centroid: Recalculate the centroid: For each cluster C k , calculate the new center of mass μ k : Repeat iteration: Repeat steps (3) and (4) until the position of the centroid no longer changes significantly or the preset maximum number of iterations is reached; (5) Generate anchor point set: Determine the final centroid: When the K-means algorithm converges, the final centroid μ k As an anchor point: A k =μ k Construct an anchor set: <h2 style=";text-align:left;direction:ltr">A = {A1,A2,…,A<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr">}。 5. The method according to claim 4, characterized in that: The specific process of obtaining the refined query vector Q′ in step 6 includes: (1) Calculate the Euclidean distance: measure the actual distance between two vectors in space. The smaller the distance, the higher the similarity: (2) Calculate similarity: The query vector is Q, and the dynamic anchor point set is A = {A1, A2, ..., A K }, where K = 32, each anchor point A k For a d-dimensional vector, the cosine similarity is used to calculate the similarity between the query vector and each anchor point: Among them, s k Represents the similarity score between the query vector and the kth anchor point; the contribution of each anchor point is determined based on the similarity, and a corresponding weight is assigned to each anchor point based on the contribution, and the refined query vector Q′ is obtained by weighted fusion of each anchor point: Among them, w k is the weight determined based on the similarity between the query vector and each anchor point.

6. The method according to claim 5, characterized in that: In step 7, the final trajectory prediction result T is obtained based on the following activation function: T=W o ·f(Q′)+b o Among them, W o and b o are the weight matrix and bias vector of the output layer respectively.

Images