Vehicle track prediction method based on interactive perception and map adaptation

Abstract

The invention relates to a vehicle track prediction method based on interactive perception and map adaptation, and belongs to the technical field of automatic driving. The method comprises the steps of collecting a historical track and geographical semantic information of a vehicle; the method comprises the following steps of: modeling interaction between dynamic objects by utilizing Transform, and fusing map information when a state sequence is decoded; a hierarchical attention mechanism is adopted, the future state of the vehicle is considered, and the consistency and accuracy of prediction are ensured; in addition, a multi-resolution map coding module is also constructed, the module fully utilizes a Vision Transform module to efficiently capture local and global scene context information, and then map representation is closely associated with a vehicle state to accurately model an interaction process of a vehicle and a scene, so that a more reasonable future trajectory prediction result is generated. The method shows wide application potential, can effectively improve the calculation performance, and is suitable for a more complex traffic environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving and relates to a vehicle trajectory prediction method based on interactive perception and map adaptation. Background Art

[0002] Autonomous driving technology aims to achieve autonomous driving of vehicles, and usually includes functional modules such as perception, positioning, planning, decision-making and control. The perception module collects environmental information through sensors such as cameras, lidar, and radar. The positioning module combines GPS and inertial measurement units to provide the precise location of the vehicle. The planning module generates a feasible driving path based on the current environment and target location. The control module executes the planning results and operates the vehicle's steering, acceleration, braking and other actions. In the autonomous driving system, trajectory prediction, as a key module connecting perception and decision-making, plays a connecting role. By performing high-level abstraction and understanding of the environmental information collected by sensors, the vehicle can accurately assess potential risks when making decisions, make safe and efficient decisions, and ensure that autonomous driving vehicles can drive safely on crowded city streets or highways.

[0003] Vehicle trajectory prediction methods are mainly divided into physical model-based methods and data-driven methods. Physical model-based methods usually use physical characteristics such as vehicle speed, acceleration and direction, combined with kinematic models for prediction. This method is simple and direct, with low computational complexity, but it is difficult to handle complex interactive behaviors and scene changes. With the rapid development of deep learning, data-driven methods have become mainstream. This type of method uses a large amount of historical trajectory data to learn the behavior pattern of the target through machine learning or deep learning models to achieve trajectory prediction. Common models include recurrent neural networks, long short-term memory networks, graph convolutional networks, and attention models. These methods can achieve more accurate trajectory prediction in multi-agent and multi-scenario interactions, especially in complex traffic environments.

[0004] However, in practice, when predicting vehicle trajectories, multiple factors usually need to be considered, such as complex interactions between vehicles, dynamic road conditions, etc. In real-world scenarios, the key challenge in vehicle trajectory prediction is to accurately predict the behavior of surrounding vehicles and understand the scene context. There are complex interactions between vehicles, and accurately modeling these interactions is crucial to predicting vehicle trajectories. In addition, static and dynamic scenes in the road environment have a direct impact on the vehicle's trajectory. Combining scene information for prediction can help the system better understand the vehicle's movement pattern in different environments. Vehicle trajectory prediction is of great significance to the safety and efficiency of autonomous driving. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a vehicle trajectory prediction method based on interactive perception and map adaptation. The method collects and processes the historical trajectory and map information of the vehicle through sensors, uses the Transformer encoder and Vision Transformer model to model the interaction relationship between vehicles and map information, and predicts the future trajectory of the vehicle based on this information.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A vehicle trajectory prediction method based on interactive perception and map adaptation, the prediction method comprising the following steps:

[0008] S1: Collect vehicle historical trajectory data and geographic semantic information, and pre-process the collected historical trajectory data and geographic semantic information;

[0009] S2: Use Transformer encoder to model the interaction between vehicles, generate embedding vectors, and process the input sequence through multi-head self-attention mechanism and residual connection to capture vehicle characteristics;

[0010] S3: Uses a multi-resolution map encoding module and Vision Transformer to segment map image blocks and encode map features, combines position encoding to process map information, and generates global and local embeddings;

[0011] S4: Estimate the future trajectory based on context information and joint distribution, combine the vehicle's historical status, interaction features and map information, use the Transformer decoder to calculate the predicted probability distribution, and generate the final trajectory prediction.

[0012] Further, in step S1, historical trajectory data and geographic semantic information of the vehicle are collected through sensors and vehicle systems, the historical trajectory data including vehicle location information, speed, and acceleration; the geographic semantic information includes the road network, topography, buildings, and landmarks of the target area;

[0013] The historical trajectory data and geographic semantic information data are organized into a sequence data format, and based on the position information of the vehicle in the global coordinate system and the positioning information of the surrounding traffic participants in their respective local coordinate systems, the position information of the surrounding traffic participants is uniformly mapped to the global coordinate system;

[0014] The collected historical trajectory data and geographic semantic information are then preprocessed, where the preprocessing includes: noise removal, standardization, and interpolation operations.

[0015] Further, in step S2, the following steps are included:

[0016] S21. Use the Transformer encoder to model the interaction between vehicles and generate a historical trajectory embedding vector for each vehicle. The interaction between vehicles is modeled by the Transformer encoder to obtain a historical trajectory sequence:

[0017]

[0018] Among them, A represents the total number of vehicles, τ is the time step; the state of each vehicle Add index information I a , and then transformed through a linear layer to obtain d m The embedding vector of dimension is reorganized into a historical trajectory embedding vector

[0019]

[0020]

[0021] Among them, E 1 (t),E 2 (t),...,E A (t) represents the coded information of each vehicle, Represents a concatenation operation, and Linear represents a linear connection;

[0022] S22, combining the historical trajectory embedding vector with the learned position encoding; wherein the learned position encoding refers to a set of position vectors that can provide additional timing information of the vehicle state at each time step through training; the combination process of the historical trajectory embedding vector and the position encoding is to combine the historical trajectory embedding vector with the position encoding. It is concatenated with the learned position encoding PE according to the time step to obtain a joint representation H with timing information and vehicle information, and passed to a Transformer module;

[0023]

[0024]

[0025]

[0026]

[0027] Among them, pos is the position information, ξ is the dimension, total is the total dimension of the input embedding; h t 1 ,h t 2 ,...,h t A is the vehicle feature;

[0028] S23, in the Transformer module, the input sequence is processed by a multi-head self-attention mechanism and a fully connected feed-forward network to capture vehicle features;

[0029] S24, using a multi-head self-attention mechanism to enhance the modeling of complex interactive relationships, where a multi-head self-attention mechanism is used to aggregate information from different feature subspaces. This process is performed λ times, and the output of each attention head is aggregated using a fully connected layer. Merge to form the final multi-head attention output φ S :

[0030]

[0031] Among them, W o is a learnable weight matrix; represents the total attention weight obtained in each calculation, and concat(·) represents the concatenation operation.

[0032] Further, in step S23, the Transformer encoder includes a plurality of identical TEBlocks, each TEBlock includes a multi-head self-attention layer and a fully connected feedforward network, and each multi-head self-attention mechanism and the fully connected feedforward network are connected to a normalized residual connection;

[0033] In the first TEBlock, the feature vectors at each moment are first learned in the multi-head self-attention layer:

[0034]

[0035] in, are the query vector, key vector and value vector of the i-th vehicle at time t, Represents the characteristics of the vehicle, W q , W k , W v are three learnable weight matrices;

[0036] right and Perform a scaled dot product, normalize the weights of the spatial edges between vehicle i and its surrounding vehicles j, and sum them up to get the attention weight of a single attention head of node i The calculation method is:

[0037]

[0038] Then the attention weight Input the fully connected feedforward layer and perform nonlinear processing; get the output a of each attention head i′; and then the final attention output a is obtained after connection processing through the normalized residual module after the fully connected feedforward layer final :

[0039]

[0040] Among them, LayerNorm(·) represents the layer normalization operation.

[0041] Further, in step S3, the following steps are included:

[0042] S31. Use the multi-resolution map encoding module to divide the map image into several image blocks; and use VisionTransformer to process the image blocks to generate global and local feature embeddings of the map;

[0043] S32, using a binary mask to represent a map of the drivable area, and adding pixel index and distance information;

[0044] S33, add position encoding to the embedding vector to capture spatial position information, embedding the global g and local embedding Γ l The sequence is input into the Vision Transformer to generate map features φ representing different relationships between vehicles and maps m .

[0045] Further, in step S31, the map information in the traffic scene is processed, the image is segmented into a series of first-category image blocks, a feature map is generated according to the first-category image blocks, and the feature map is used to generate local and global embeddings;

[0046] Among them, the first type of image blocks are transformed into dimension d through a linear layer m , we get the global embedding Γ of the entire scene g ;

[0047] The first type of image blocks are further extracted into a second type of image blocks of size M×M, where the second type of image blocks are smaller than the first type of image blocks, and the second type of image blocks are linearly mapped to transform the dimension into d m , thus obtaining the local embedding Γ l ;

[0048] In step S32, the drivable area map is first represented using a binary mask, and the pixel index and the Euclidean distance between each pixel and the center of the map are added in each channel;

[0049] Secondly, through a sparse global graph, each edge is linearly interpolated to create a topological map in an automatic or semi-automatic way;

[0050] In step S33, the positional encoding is added to the embedding vector:

[0051] E′=PE+E

[0052] Among them, E represents the processed embedding vector, PE is the positional encoding;

[0053] Embed the global g and local embedding Γ l The sequence is input into the Vision Transformer to generate a series of map features φ m ;

[0054] φ m =ViT(concat(Γ g ,Γ l ))

[0055] Where ViT(·) represents Vision Transformer;

[0056] The loss is back-propagated through the single attention head that is closest to the true value in terms of the final displacement error, where the query vector is the interaction feature between vehicles, and the key vector and value vector are the feature vectors from the map encoder.

[0057] Further, in step S4, the following steps are included:

[0058] S41, matching the vehicle interaction features with the map features, processing the combined features through a fully connected layer to form a final feature representation;

[0059] S42, use the Transformer decoder to calculate the predicted trajectory distribution, and further refine the prediction through multiple cross-attention modules, generate trajectory coordinates for each time step, and generate probability distribution through softmax for the prediction results;

[0060] S43. Optimize the parameters using maximum likelihood estimation to maximize the marginal data log likelihood, and use the optimized parameters to perform trajectory prediction to obtain the final prediction result.

[0061] Further, in step S41, the vehicle interaction features are matched with the map features, and the combined features are processed through the fully connected layer to form the final feature representation. The process is as follows:

[0062] Each vehicle uses ω discrete patterns, each pattern is represented as a one-hot binary vector, the road features are input into a fully connected layer, and the maximum pooling is used to aggregate the information on the same lane;

[0063] The lane features are processed by another fully connected layer, which concatenates the waypoint features, aggregate features, and lane features and passes them through a fully connected layer to obtain the final feature vector of the waypoint;

[0064] The index of the vehicle is concatenated to the K-sized pattern representation of the vehicle. All vectors are then flattened to form a vector of size A×K and dimension 2 for all vehicles. The vector is then passed through a linear layer and multiplied with φ S Connect to form the intention embedding φ i , the intention embedding φ i Capture the vehicle's intention in the current traffic scenario based on the vehicle's historical status, interaction features, and map information.

[0065] Further, in step S42, according to the context feature φ S and φ m Calculate the joint distribution Z S , each vehicle uses K learnable vectors P;

[0066] The index of each vehicle is also concatenated to the vector P and projected to d using a linear layer. m dimension, input the sequence into the Transformer decoder block TDBlock, calculate all Z S The probability of ; Among them, TDBlock contains two cross attention modules;

[0067] The predicted vehicle trajectory depends on three features, namely the historical state of the target vehicle, the interaction between vehicles, and the map information; the shape of the map feature is matched according to the interaction feature and the target vehicle feature along the mode axis, and then these three features are connected to form the final feature representation;

[0068] The trajectory decoder outputs the trajectory coordinates for each time step, and the scores of all predicted trajectories are grouped and passed through a softmax layer to produce a probability distribution:

[0069] P(Z s |φ)=Softmax(FFN(TDBlack d (ρ,φ)))

[0070]

[0071] Among them, φ represents φ S and φ m , ρ is a learnable vector, TDBlack d (·) denotes a TDBlock block, and FFN(·) denotes a fully connected layer.

[0072] Further, in step S43, at each time step, the trajectory prediction of the target vehicle is conditioned on all other vehicles, and then the output sequence is passed to N feed-forward layers to generate parameters of the binary Gaussian distribution, and the maximum likelihood estimation is used to find the parameter θ that maximizes the marginal data log likelihood, and the optimized parameters are used for trajectory prediction to obtain the final prediction result.

[0073] The beneficial effects of the present invention are:

[0074] The present invention uses Transformer to model the interactive relationship between dynamic objects in traffic scenes, effectively captures the complex interaction between vehicles, and improves the overall accuracy of trajectory prediction; incorporates map information when decoding state sequences, and uses a hierarchical attention mechanism to dynamically regulate the weight and effect of the vehicle's past trajectory and scene encoding, making the prediction result more reasonable and in line with the actual scene; through a multi-resolution map encoding module, this method can efficiently capture local and global scene context information, and uses a Vision Transformer module that closely combines map information with vehicle status, thereby modeling the complex relationship between vehicles and scenes at different scales.

[0075] The future trajectory prediction generated by the present invention is not only highly accurate in the short term, but also stable and consistent in the long-term prediction, making the vehicle trajectory prediction more reliable. The present invention significantly improves the accuracy and rationality of vehicle trajectory prediction through innovative methods and module design, is suitable for complex traffic scenarios, has good flexibility and scalability, and has broad application prospects.

[0076] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0078] Figure 1 is an overall flow chart of the trajectory prediction method of the present invention;

[0079] Figure 2 A schematic diagram of a vehicle prediction coding module of the present invention;

[0080] Figure 3 This is a schematic diagram of a map encoding module of the present invention;

[0081] Figure 4It is a schematic diagram of the vehicle prediction decoding module of the present invention. DETAILED DESCRIPTION

[0082] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0083] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0084] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0085] See also Figure 1 to Figure 4 , which is a vehicle trajectory prediction method based on interactive perception and map adaptation.

[0086] Example

[0087] This embodiment discloses a specific implementation method of a vehicle trajectory prediction method based on interactive perception and map adaptation, which adopts the Transformer method to model the interactive relationship between dynamic objects in the scene, and incorporates map information when decoding the state sequence. This fusion process is achieved through a hierarchical attention mechanism, in which the state of the vehicle dynamically regulates the weight and contribution of past trajectories and scene encoding in the final prediction result in an autoregressive manner. In addition, a multi-resolution map encoding module is constructed, which makes full use of the Vision Transformer module to efficiently capture local and global scene context information, and then closely links the map representation with the vehicle state to accurately model the interaction process between the vehicle and the scene, thereby generating more reasonable future trajectory prediction results. Figure 1 As shown, the method specifically includes the following processes:

[0088] S1: Collect vehicle historical trajectory data and geographic semantic information, and pre-process the data to ensure continuity and accuracy;

[0089] S2: Transformer encoder is used to model the interaction between vehicles, generate embedding vectors, and process input sequences through multi-head self-attention mechanism and residual connection to capture the complex interaction characteristics between vehicles;

[0090] S3: Uses a multi-resolution map encoding module and Vision Transformer to segment map image blocks and encode map features, combines position encoding to process map information, and generates global and local embeddings;

[0091] S4: Estimate the future trajectory based on context information and joint distribution, combine the vehicle's historical status, interaction features and map information, use the Transformer decoder to calculate the predicted probability distribution, and generate the final trajectory prediction.

[0092] In step S1 of this embodiment, the historical trajectory data of the vehicle is comprehensively collected through sensors and vehicle systems, including key indicators such as vehicle location information, speed, acceleration, and geographic semantic information such as road network, topography, buildings and landmarks in the target area, and these data are orderly organized into a sequence data form. Based on the precise position of the vehicle in the global coordinate system and the positioning information of the surrounding traffic participants in their respective local coordinate systems, the position information of the surrounding traffic participants is uniformly mapped to the global coordinate system.

[0093] In addition, in step S1, the collected historical trajectory data and geographic semantic information data are preprocessed, including noise removal, standardization processing, and interpolation operations to ensure the continuity and accuracy of the data.

[0094] In step S2 of this embodiment, a trajectory prediction coding module is used to obtain vehicle trajectory features, such as Figure 2 As shown, the specific steps include:

[0095] S21. Use the Transformer encoder to model the interactions between vehicles and generate the historical trajectory embedding vector of each vehicle.

[0096] Specifically, the Transformer encoder is used to model the interaction between vehicles to obtain the historical trajectory sequence. Where A represents the total number of vehicles, τ is the time step, and the state of each vehicle Add index information I a , and then transformed through a linear layer to obtain d m The embedding vector of dimension is reorganized into a historical trajectory embedding vector The calculation method is:

[0097]

[0098]

[0099] Among them, E 1 (t),E 2 (t),...,E A (t) represents the coded information of each vehicle, represents a concatenation operation, and Linear represents a linear connection; the sequence covers the trajectories of all vehicles within the observation range, and flexibly handles a variable number of vehicles through index allocation.

[0100] S22, combine the historical trajectory embedding vector with the learned position encoding;

[0101] Specifically, the learned position encoding refers to a set of position vectors obtained through training, which can provide additional timing information for the vehicle state at each time step; the combination process of the historical trajectory embedding vector and the position encoding is to concatenate the historical embedding S′ and the learned position encoding PE according to the time step, so as to obtain a joint representation H with timing information and vehicle information, and pass it to a Transformer module.

[0102]

[0103]

[0104]

[0105]

[0106] Among them, pos is the position information, v is the dimension, ξ total is the total dimension of the input embedding, Characteristics of the vehicle.

[0107] S23. In the Transformer module, the input sequence is processed by a multi-head self-attention mechanism and a fully connected feed-forward network to capture the complex interaction features between vehicles.

[0108] Specifically, the Transformer encoder is composed of multiple identical TEBlocks stacked together, each of which contains a multi-head self-attention mechanism and a fully connected feedforward network. Each TEBlock's multi-head self-attention mechanism and fully connected feedforward network are followed by a normalized residual connection to effectively process and encode the input sequence. In the first TEBlock, the encoded input sequence is first processed by the multi-head self-attention layer, and then the processing result is passed to the fully connected feedforward neural network. First, the feature vectors at each moment are learned in the multi-head self-attention layer. Indicates the characteristics of the vehicle; and Perform a scaled dot product, normalize the weights of the spatial edges between vehicle i and its surrounding vehicles j, and sum them up to get the attention weight of a single attention head of node i The calculation method is: The above three vectors are the query vectors of the i-th vehicle at time t Key Vector Value vector W q , W k , W v are three learnable weight matrices. Input the fully connected feedforward layer and perform nonlinear processing; get the output a of each attention head i ′; and then the final attention output is obtained after connection processing through the normalized residual module after the fully connected feedforward layer:

[0109]

[0110] Among them, LayerNorm(·) represents the layer normalization operation.

[0111] S24, using multi-head self-attention mechanism to enhance the modeling of complex interactive relationships;

[0112] Specifically, in order to enhance the modeling of complex interactive relationships from multiple perspectives, a multi-head self-attention mechanism is used to aggregate information from different feature subspaces. This process is performed λ times, and the output of each attention head is aggregated using a fully connected layer. Each TEBlock uses a residual connection to connect the input of each layer to its output and merge them before applying layer normalization, which can capture the complex interactions between vehicles. The output of the trajectory encoder is φ S ,φ S Represents the interaction characteristics between vehicles:

[0113]

[0114] Among them, W o is a learnable weight matrix; represents the total attention weight obtained in each calculation, and concat(·) represents the concatenation operation.

[0115] In step S3 of this embodiment, a multi-resolution map encoding module and a Vision Transformer are used to segment map image blocks and encode map features, and map information is processed in combination with position encoding to generate global and local embeddings, such as Figure 3 As shown, the specific steps include:

[0116] S31. Use the multi-resolution map encoding module to divide the map image into several image blocks; and use VisionTransformer to process the image blocks to generate global and local feature embeddings of the map;

[0117] Specifically, when using Vision Transformer to process map information, the map image is segmented into a series of image patches, which are processed by the encoder of Vision Transformer to capture the global and local features in the map. First, the map information in the traffic scene is processed, the image is segmented into a series of first-class image patches, and a feature map is generated. The feature map is used to generate local and global embeddings, and these image patches are transformed into dimension d through a linear layer. m , thereby creating a global embedding Γ for the entire scene g Then extract the second type of image blocks of size M×M, where M is smaller than the size of the feature map, and perform linear mapping on these image blocks to transform the dimension into d m , thus obtaining the local embedding Γ l .

[0118] S32, using a binary mask to represent a map of the drivable area, and adding pixel index and distance information;

[0119] Specifically, for map information, simple semantics are used to help the model avoid unrealistic predictions, such as drivable areas. The present invention uses a binary mask to represent the map of the drivable area, adds pixel indexes and the Euclidean distance between each pixel and the center of the map in each channel to further enrich the map information, and the model can directly obtain the spatial information of each pixel. Through a sparse global graph, each edge is linearly interpolated to create a dense topological map. Such a topological map can also be created in an automatic or semi-automatic manner, using relatively simple and low-cost topological maps to achieve effective trajectory prediction and navigation functions, which makes our method more flexible and scalable in practical applications.

[0120] S33. In order to capture the spatial location information, position encoding is added to the embedding vector, and the sequence of global embedding and local embedding is input into the Vision Transformer to generate map features to represent the different relationships between the vehicle and the map;

[0121] In addition, in order to process the sequence sequentially, a positional encoding E′=PE+E is added to the embedding vector so that the model can capture the information of different positions in the image. g and local embedding Γ l The sequence is input into VisionTransformer to generate a series of map features φ m ,φ m =ViT(concat(Γ g ,Γ l )). The output is a pattern feature, each with different characteristics corresponding to different relationships between the target vehicle and the map. To ensure the diversity of these patterns, we focus on different parts of the map and backpropagate the loss through the single attention head that is closest to the true value in terms of the final displacement error. The query vector is the interaction feature between vehicles, while the key vector and value vector are the feature vectors from the map encoder.

[0122] In step S4 of this embodiment, the future trajectory is estimated based on the context information and the joint distribution, and the predicted probability distribution is calculated using the Transformer decoder in combination with the vehicle's historical state, interaction features, and map information, and the final trajectory prediction is generated, such as Figure 4 As shown, the specific steps include:

[0123] S41, matching the vehicle interaction features with the map features, processing the combined features through a fully connected layer to form a final feature representation;

[0124] The goal of prediction is to estimate the joint distribution over the future trajectories of all vehicles based on the context information, which is the observed state φS and map φ m . To estimate the joint distribution, ω discrete patterns are used for each vehicle, each pattern is represented as a one-hot binary vector, the road features are input into a fully connected layer, and the information on the same lane is aggregated using maximum pooling. The lane features are processed by another fully connected layer. The waypoint features, aggregated features, and lane features are concatenated and passed through a fully connected layer to obtain the final feature vector of the waypoint. Similar to interaction modeling, the index of the vehicle is connected to the pattern representation of size K of the vehicle. Then, all vectors are flattened to form a vector of size A×K and dimension 2 for all vehicles. These vectors are then passed through a linear layer and combined with φ S Before joining, form the intention embedding φ i The intention embedding is based on the vehicle's historical status, interaction features, and map information, capturing the vehicle's intention in the current traffic scenario.

[0125] S42, use the Transformer decoder to calculate the predicted trajectory distribution, and further refine the prediction through multiple cross-attention modules, generate trajectory coordinates for each time step, and generate probability distribution through softmax for the prediction results;

[0126] Specifically, according to the context feature φ S and φ m Calculate the joint distribution Z S , each vehicle uses K learnable vectors V. The index of each vehicle is also concatenated to this vector and projected to d using a linear layer. m dimension, this sequence is fed into the Transformer decoder block TDBlock to compute all Z S The TDBlock contains two cross-attention modules. This approach allows the model to learn different driving modes in an unsupervised manner, which can represent different driving behaviors and decisions, such as going straight, going left, going right, accelerating, decelerating, etc. The predicted vehicle trajectory depends on three features, namely the historical state of the target vehicle, the interaction between vehicles, and the map information. The shape of the map feature is matched along the mode axis according to the interaction feature and the target vehicle feature, and then the three features are connected to form the final feature representation. The trajectory decoder outputs the trajectory coordinates of each time step, and the scores of all predicted trajectories are grouped and passed through the softmax layer to produce a probability distribution.

[0127] P(Z s |φ)=Softmax(FFN(TDBlack d (V,φ)))

[0128]

[0129] Among them, φ represents φ S and φ m , ρ is a learnable vector, TDBlack d (·) denotes a TDBlock block, and FFN(·) denotes a fully connected layer.

[0130] S43. Use maximum likelihood estimation to optimize parameters to maximize the marginal data log likelihood and ensure model accuracy and stability.

[0131] Specifically, each possible future trajectory of the vehicle depends on a pattern that is more consistent with the goal of the entire trajectory intention in the latent space. Since layer normalization operations delay convergence, the second decoder block does not have layer normalization operations. At each time step, the trajectory prediction of the target vehicle is conditioned on all other vehicles, and then the output sequence is passed to N feed-forward layers to generate the parameters of the binary Gaussian distribution. Maximum likelihood estimation is used to find the parameters θ that maximize the marginal data log likelihood. The optimized parameters are used for trajectory prediction to obtain the final prediction result.

[0132]

[0133]

[0134] in, represents the predicted trajectory, is the average value, is the standard deviation;

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A vehicle trajectory prediction method based on interactive perception and map adaptation, characterized by: The prediction method comprises the following steps: S1: Collect vehicle historical trajectory data and geographic semantic information, and pre-process the collected historical trajectory data and geographic semantic information; S2: Use Transformer encoder to model the interaction between vehicles, generate embedding vectors, and process the input sequence through multi-head self-attention mechanism and residual connection to capture vehicle characteristics; S3: Uses a multi-resolution map encoding module and Vision Transformer to segment map image blocks and encode map features, combines position encoding to process map information, and generates global and local embeddings; S4: Estimate the future trajectory based on context information and joint distribution, combine the vehicle's historical status, interaction features and map information, use the Transformer decoder to calculate the predicted probability distribution, and generate the final trajectory prediction.

2. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 1 is characterized in that: In step S1, historical trajectory data and geographic semantic information of the vehicle are collected through sensors and vehicle systems. The historical trajectory data includes vehicle location information, speed, and acceleration; the geographic semantic information includes the road network, topography, buildings, and landmarks of the target area; The historical trajectory data and geographic semantic information data are organized into a sequence data format, and based on the position information of the vehicle in the global coordinate system and the positioning information of the surrounding traffic participants in their respective local coordinate systems, the position information of the surrounding traffic participants is uniformly mapped to the global coordinate system; The collected historical trajectory data and geographic semantic information are then preprocessed, where the preprocessing includes: noise removal, standardization, and interpolation operations.

3. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 1 is characterized in that: In step S2, the following steps are included: S21. Use the Transformer encoder to model the interaction between vehicles and generate a historical trajectory embedding vector for each vehicle. The interaction between vehicles is modeled by the Transformer encoder to obtain a historical trajectory sequence: Among them, A represents the total number of vehicles, τ is the time step; the state of each vehicle Add index information I a , and then transformed through a linear layer to obtain d m The embedding vector of dimension is reorganized into a historical trajectory embedding vector Among them, E 1 (t),E 2 (t),...,E A (t) represents the coded information of each vehicle, Represents a concatenation operation, and Linear represents a linear connection; S22, combining the historical trajectory embedding vector with the learned position encoding; wherein the learned position encoding refers to a set of position vectors that can provide additional timing information of the vehicle state at each time step through training; the combination process of the historical trajectory embedding vector and the position encoding is to combine the historical trajectory embedding vector with the position encoding. It is concatenated with the learned position encoding PE according to the time step to obtain a joint representation H with timing information and vehicle information, and passed to a Transformer module; Among them, pos is the position information, v is the dimension, ξ total is the total dimension of the input embedding; is the vehicle feature; S23, in the Transformer module, the input sequence is processed by a multi-head self-attention mechanism and a fully connected feed-forward network to capture vehicle features; S24, using a multi-head self-attention mechanism to enhance the modeling of complex interactive relationships, where a multi-head self-attention mechanism is used to aggregate information from different feature subspaces. This process is performed λ times, and the output head of each attention head is aggregated using a fully connected layer. i t Merge to form the final multi-head attention output φ S : Among them, W o is a learnable weight matrix; represents the total attention weight obtained in each calculation, and concat(·) represents the concatenation operation.

4. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 3 is characterized by: In step S23, the Transformer encoder includes multiple identical TEBlocks, each TEBlock includes a multi-head self-attention layer and a fully connected feedforward network, and each multi-head self-attention mechanism and fully connected feedforward network is connected to a normalized residual connection; In the first TEBlock, the feature vectors at each moment are first learned in the multi-head self-attention layer: in, are the query vector, key vector and value vector of the i-th vehicle at time t, Represents the characteristics of the vehicle, W q , W k , W v are three learnable weight matrices; right and Perform a scaled dot product, normalize the weights of the spatial edges between vehicle i and its surrounding vehicles j, and sum them up to get the attention weight of a single attention head of node i The calculation method is: Then the attention weight Input the fully connected feedforward layer and perform nonlinear processing; get the output a of each attention head i ′; and then the final attention output a is obtained after connection processing through the normalized residual module after the fully connected feedforward layer final : Among them, LayerNorm(·) represents the layer normalization operation.

5. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 3 is characterized by: In step S3, the following steps are included: S31. Use the multi-resolution map encoding module to divide the map image into several image blocks; and use VisionTransformer to process the image blocks to generate global and local feature embeddings of the map; S32, using a binary mask to represent a map of the drivable area, and adding pixel index and distance information; S33, add position encoding to the embedding vector to capture spatial position information, embedding the global g and local embedding Γ l The sequence is input into the Vision Transformer to generate map features φ representing different relationships between vehicles and maps m .

6. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 5 is characterized in that: In step S31, the map information in the traffic scene is processed, the image is segmented into a series of first-category image blocks, a feature map is generated according to the first-category image blocks, and the feature map is used to generate local and global embeddings; Among them, the first type of image blocks are transformed into dimension d through a linear layer m , we get the global embedding Γ of the entire scene g ; The first type of image blocks are further extracted into a second type of image blocks of size M×M, where the second type of image blocks are smaller than the first type of image blocks, and the second type of image blocks are linearly mapped to transform the dimension into d m , thus obtaining the local embedding Γ l ; In step S32, the drivable area map is first represented using a binary mask, and the pixel index and the Euclidean distance between each pixel and the center of the map are added in each channel; Secondly, through a sparse global graph, each edge is linearly interpolated to create a topological map in an automatic or semi-automatic way; In step S33, the positional encoding is added to the embedding vector: E′=PE+E Where E represents the processed embedding vector, and PE is the positional encoding; Embed the global g and local embedding Γ l The sequence is input into the Vision Transformer to generate a series of map features φ m ; φ m =ViT(concat(Γ g ,Γ l )) Where ViT(·) represents Vision Transformer; The loss is back-propagated through the single attention head that is closest to the true value in terms of the final displacement error, where the query vector is the interaction feature between vehicles, and the key vector and value vector are the feature vectors from the map encoder.

7. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 5 is characterized by: In step S4, the following steps are included: S41, matching the vehicle interaction features with the map features, processing the combined features through a fully connected layer to form a final feature representation; S42, use the Transformer decoder to calculate the predicted trajectory distribution, and further refine the prediction through multiple cross-attention modules, generate trajectory coordinates for each time step, and generate probability distribution through softmax for the prediction results; S43. Optimize the parameters using maximum likelihood estimation to maximize the marginal data log likelihood, and use the optimized parameters to perform trajectory prediction to obtain the final prediction result.

8. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 7 is characterized by: In step S41, the vehicle interaction features are matched with the map features, and the combined features are processed through the fully connected layer to form the final feature representation. The process is as follows: Each vehicle uses ω discrete patterns, each pattern is represented as a one-hot binary vector, the road features are input into a fully connected layer, and the maximum pooling is used to aggregate the information on the same lane; The lane features are processed by another fully connected layer, which concatenates the waypoint features, aggregate features, and lane features and passes them through a fully connected layer to obtain the final feature vector of the waypoint; The index of the vehicle is concatenated to the K-sized pattern representation of the vehicle. All vectors are then flattened to form a vector of size A×K and dimension 2 for all vehicles. The vector is then passed through a linear layer and multiplied with φ S Connect to form the intention embedding φ i , the intention embedding φ i Capture the vehicle's intention in the current traffic scenario based on the vehicle's historical status, interaction features, and map information.

9. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 8, characterized in that: In step S42, according to the context feature φ S and φ m Calculate the joint distribution Z S , each vehicle uses K learnable vectors P; The index of each vehicle is also concatenated to the vector P and projected to d using a linear layer. m dimension, input the sequence into the Transformer decoder block TDBlock, calculate all Z S The probability of ; Among them, TDBlock contains two cross attention modules; The predicted vehicle trajectory depends on three features, namely the historical state of the target vehicle, the interaction between vehicles, and the map information; the shape of the map feature is matched according to the interaction feature and the target vehicle feature along the mode axis, and then these three features are connected to form the final feature representation; The trajectory decoder outputs the trajectory coordinates for each time step, and the scores of all predicted trajectories are grouped and passed through a softmax layer to produce a probability distribution: P(Z s |φ)=Softmax(FFN(TDBlack d (ρ,φ))) Among them, φ represents φ S and φ m , ρ is a learnable vector, TDBlack d (·) denotes a TDBlock block, and FFN(·) denotes a fully connected layer.

10. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 9, characterized in that: In step S43, at each time step, the trajectory prediction of the target vehicle is conditioned on all other vehicles, and then the output sequence is passed to N feed-forward layers to generate the parameters of the binary Gaussian distribution. The maximum likelihood estimation is used to find the parameter θ that maximizes the marginal data log likelihood, and the trajectory prediction is performed using the optimized parameters to obtain the final prediction result.

Images