Vehicle trajectory prediction method based on interaction perception and map adaptation

Abstract

This invention relates to a vehicle trajectory prediction method based on interactive perception and map adaptation, belonging to the field of autonomous driving technology. The method includes collecting the vehicle's historical trajectory and geographic semantic information; using Transformer to model the interaction between dynamic objects and incorporating map information when decoding the state sequence; employing a hierarchical attention mechanism to consider the vehicle's future state, ensuring the consistency and accuracy of the prediction; furthermore, a multi-resolution map encoding module is constructed, which fully utilizes the Vision Transformer module to efficiently capture local and global scene context information, thereby closely associating map representation with vehicle state to accurately model the interaction process between the vehicle and the scene, thus generating more reasonable future trajectory prediction results. This invention demonstrates broad application potential, effectively improving computational performance and being suitable for more complex traffic environments.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology and relates to a vehicle trajectory prediction method based on interactive perception and map adaptation. Background Technology

[0002] Autonomous driving technology aims to enable vehicles to drive autonomously, typically including functional modules such as perception, localization, planning, decision-making, and control. The perception module collects environmental information through sensors such as cameras, LiDAR, and radar. The localization module combines GPS and inertial measurement units to provide the vehicle's precise location. The planning module generates feasible driving paths based on the current environment and target location. The control module executes the planning results, controlling actions such as steering, acceleration, and braking. In autonomous driving systems, trajectory prediction, as a key module connecting perception and decision-making, plays a crucial role. By performing high-level abstraction and understanding of the environmental information collected by sensors, the vehicle can accurately assess potential risks and make safe and efficient decisions, ensuring that autonomous vehicles can drive safely on congested city streets or highways.

[0003] Vehicle trajectory prediction methods are mainly divided into physics-based methods and data-driven methods. Physics-based methods typically use physical characteristics of the vehicle, such as speed, acceleration, and direction, combined with kinematic models for prediction. This method is simple and direct, with low computational cost, but it struggles to handle complex interactions and changing scenarios. With the rapid development of deep learning, data-driven methods have become mainstream. These methods utilize large amounts of historical trajectory data, learning the target's behavioral patterns through machine learning or deep learning models to achieve trajectory prediction. Common models include recurrent neural networks, long short-term memory networks, graph convolutional networks, and attention models. These methods can achieve more accurate trajectory prediction in multi-agent, multi-scenario interactions, and are particularly suitable for complex traffic environments.

[0004] However, in practice, predicting vehicle trajectories typically requires considering multiple factors, such as complex interactions between vehicles and dynamic road conditions. In real-world scenarios, a key challenge in vehicle trajectory prediction lies in accurately predicting the behavior of surrounding vehicles and understanding the scene context. Complex interaction relationships exist between vehicles, and accurately modeling these relationships is crucial for predicting vehicle trajectories. Furthermore, static and dynamic scenes within the road environment directly impact vehicle trajectories; combining scene information for prediction helps the system better understand the vehicle's movement patterns in different environments. Vehicle trajectory prediction is of great significance to the safety and efficiency of autonomous driving. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a vehicle trajectory prediction method based on interactive perception and map adaptation. This method collects and processes historical vehicle trajectories and map information through sensors, utilizes a Transformer encoder and a Vision Transformer model to model the interaction relationships between vehicles and the map information, and predicts the future trajectories of vehicles based on this information.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A vehicle trajectory prediction method based on interactive perception and map adaptation, the prediction method includes the following steps:

[0008] S1: Collect historical vehicle trajectory data and geographic semantic information, and preprocess the collected historical trajectory data and geographic semantic information;

[0009] S2: The Transformer encoder is used to model the interaction between vehicles, generate embedding vectors, and process the input sequence through a multi-head self-attention mechanism and residual connections to capture vehicle features;

[0010] S3: Employs a multi-resolution map encoding module and Vision Transformer to segment map image blocks and encode map features, and combines location encoding to process map information and generate global and local embeddings;

[0011] S4: Estimate future trajectories based on contextual information and joint distribution. Combine vehicle historical status, interaction features and map information, use Transformer decoder to calculate prediction probability distribution, and generate final trajectory prediction.

[0012] Furthermore, in step S1, historical trajectory data and geographic semantic information of the vehicle are collected through sensors and the vehicle system. The historical trajectory data includes vehicle location information, speed, and acceleration; the geographic semantic information includes the road network, terrain, buildings, and landmarks of the target area.

[0013] Historical trajectory data and geographic semantic information data are organized into sequence data. Based on the vehicle's position information in the global coordinate system and the positioning information of surrounding traffic participants in their respective local coordinate systems, the position information of surrounding traffic participants is uniformly mapped to the global coordinate system.

[0014] The collected historical trajectory data and geographic semantic information are then preprocessed, including noise removal, standardization, and interpolation.

[0015] Furthermore, step S2 includes the following steps:

[0016] S21. Model the interactions between vehicles using a Transformer encoder to generate historical trajectory embedding vectors for each vehicle; specifically, modeling the interactions between vehicles using a Transformer encoder yields historical trajectory sequences.

[0017]

[0018] Where A represents the total number of vehicles, τ is the time step, and the state of each vehicle is... Add index information I a d is then obtained through a linear layer transformation. m The embedding vector of dimension is reorganized into a historical trajectory embedding vector.

[0019]

[0020]

[0021] Among them, E 1 (t),E 2 (t),...,E A (t) represents the encoded information for each vehicle. This indicates a concatenation operation; Linear indicates a linear connection.

[0022] S22. Combine the historical trajectory embedding vector with the learned position code; whereby the learned position code refers to obtaining a set of position vectors through training that can provide additional temporal information about the vehicle state at each time step; the process of combining the historical trajectory embedding vector with the position code is to combine the historical embedding vector with the learned position code. The learned location code PE is concatenated with the time step to obtain a joint representation H containing temporal information and vehicle information, and then passed to a Transformer module;

[0023]

[0024]

[0025]

[0026]

[0027] Where pos represents the location information, and ξ represents the dimension. total h is the total dimension of the input embedding; t 1 ,h t 2 ,...,h t A Vehicle characteristics;

[0028] S23. In the Transformer module, the input sequence is processed by a multi-head self-attention mechanism and a fully connected feedforward network to capture vehicle features.

[0029] S24. A multi-head self-attention mechanism is employed to enhance the modeling of complex interaction relationships. This mechanism aggregates information from different feature subspaces, performing this process λ times. A fully connected layer is then used to process the output of each attention head. The results are merged to form the final multi-head attention output φ. S :

[0030]

[0031] Among them, W o It is a learnable weight matrix; This represents the total attention weights calculated each time, and concat(·) represents the concatenation operation.

[0032] Furthermore, in step S23, the Transformer encoder includes multiple identical TEBlocks, each TEBlock including a multi-head self-attention layer and a fully connected feedforward network, and each multi-head self-attention mechanism and fully connected feedforward network is followed by a normalized residual connection;

[0033] In the first TEBlock, the feature vectors at each time step are first learned in the multi-head self-attention layer:

[0034]

[0035] in, Let be the query vector, key vector, and value vector of the i-th vehicle at time t, respectively. W indicates the characteristics of the vehicle. q W k W v There are three learnable weight matrices;

[0036] right and A scaled dot product is performed to normalize the weights of the spatial edges between vehicle i and its surrounding vehicles j, and then the summation is used to obtain the attention weight of a single attention head at node i. The calculation method is as follows:

[0037]

[0038] Then adjust the attention weight The input is processed by a fully connected feedforward layer, which performs nonlinear processing to obtain the output 'a' of each attention head. iThen, after passing through the normalized residual module after the fully connected feedforward layer, the final attention output a is obtained. final :

[0039]

[0040] Among them, LayerNorm(·) represents the layer normalization operation.

[0041] Furthermore, step S3 includes the following steps:

[0042] S31. Use the multi-resolution map encoding module to segment the map image into several image patches; and use VisionTransformer to process the image patches to generate global and local feature embeddings of the map.

[0043] S32. Use a binary mask to represent the map of the drivable area, and add pixel index and distance information;

[0044] S33. Add positional encoding to the embedding vector to capture spatial positional information, and then add global embedding Γ. g and local embedding Γ l The sequence is input into the Vision Transformer to generate map features φ representing different relationships between vehicles and the map. m .

[0045] Furthermore, in step S31, the map information in the traffic scene is processed, the image is segmented into a series of first-class image blocks, and a feature map is generated based on the first-class image blocks. The feature map is used to generate local and global embeddings.

[0046] In this process, the first type of image patch is transformed in dimension d through a linear layer. m To obtain the global embedding Γ of the entire scene g ;

[0047] Further, extract a second type of image patch of size M×M from the first type of image patch, where the second type of image patch is smaller than the first type of image patch. Perform a linear mapping on the second type of image patch to transform its dimension to d. m Thus, the local embedding Γ is obtained. l ;

[0048] In step S32, a binary mask is first used to represent the drivable area map, and a pixel index and the Euclidean distance between each pixel and the center of the map are added to each channel.

[0049] Secondly, a topological map is created automatically or semi-automatically by linearly interpolating each edge through a sparse global graph.

[0050] In step S33, positional encoding is added to the embedding vector:

[0051] E′=PE+E

[0052] Where E represents the processed embedding vector, and PE is the positional encoding;

[0053] Global embedding Γ g and local embedding Γ l The sequence is input into the Vision Transformer to generate a series of map features φ m ;

[0054] φ m =ViT(concat(Γ) g ,Γ l ))

[0055] Where ViT(·) represents Vision Transformer;

[0056] The loss is backpropagated using a single attention head that is closest to the true value in terms of the final displacement error, where the query vector is the interaction feature between vehicles, and the key vector and value vector are feature vectors from the map encoder.

[0057] Furthermore, step S4 includes the following steps:

[0058] S41. Match vehicle interaction features with map features, process the combined features through a fully connected layer to form the final feature representation;

[0059] S42. Use the Transformer decoder to calculate the predicted trajectory distribution, and further refine the prediction through multiple cross-attention modules. Generate trajectory coordinates for each time step, and generate a probability distribution from the prediction results using softmax.

[0060] S43. Optimize the parameters using maximum likelihood estimation to maximize the log-likelihood of the marginal data. Use the optimized parameters to predict the trajectory and obtain the final prediction result.

[0061] Furthermore, in step S41, the vehicle interaction features are matched with map features, and the combined features are processed through a fully connected layer to form the final feature representation. The process is as follows:

[0062] Each vehicle uses ω discrete patterns, each pattern represented as a one-hot binary vector. Road features are input into a fully connected layer, and max pooling is used to aggregate information from the same lane.

[0063] Lane features are processed by another fully connected layer, which concatenates waypoint features, aggregate features and lane features, and then passes them through a fully connected layer to obtain the final feature vector of the waypoints;

[0064] The vehicle index is concatenated to the vehicle's pattern representation of size K. Then, all vectors are flattened to form a vector of size A×K with dimension 2 for all vehicles. This vector is then passed through a linear layer and connected to φ. S The connection forms an intention to embed φ i Intending to embed φ i Based on vehicle history, interaction features, and map information, the system captures the vehicle's intent in the current traffic scenario.

[0065] Furthermore, in step S42, based on the context feature φ S and φ m The joint distribution Z is calculated. S Each vehicle uses K learnable vectors P;

[0066] The index of each vehicle is also concatenated with vector P and projected onto d using a linear layer. m Dimension, input the sequence into the Transformer decoder block TDBlock, and compute all Z... S The probability of the above; where TDBlock contains two cross-attention modules;

[0067] The predicted vehicle trajectory depends on three features: the target vehicle's historical state, the interactions between vehicles, and map information. The shape of the map features is matched along the pattern axis based on the interaction features and the target vehicle features. Then, these three features are connected to form the final feature representation.

[0068] The trajectory decoder outputs the trajectory coordinates at each time step. The scores of all predicted trajectories are grouped and passed through a softmax layer to generate a probability distribution.

[0069] P(Z s |φ)=Softmax(FFN(TDBlack d (ρ,φ)))

[0070]

[0071] Where φ represents φ S and φ m ρ is a learnable vector, TDBlack d (·) represents a TDBlock block, and FFN(·) represents a fully connected layer.

[0072] Furthermore, in step S43, at each time step, the trajectory prediction of the target vehicle is conditioned on all other vehicles, and then the output sequence is passed to N feedforward layers to generate parameters of a binary Gaussian distribution. Maximum likelihood estimation is used to find the parameter θ that maximizes the log-likelihood of the marginal data. The optimized parameters are used to perform trajectory prediction to obtain the final prediction result.

[0073] The beneficial effects of this invention are as follows:

[0074] This invention uses Transformer to model the interaction relationships between dynamic objects in traffic scenes, effectively capturing complex interactions between vehicles and improving the overall accuracy of trajectory prediction. Map information is incorporated when decoding state sequences, and a hierarchical attention mechanism is used to dynamically adjust the weights and effects of past vehicle trajectories and scene encoding, making the prediction results more reasonable and consistent with real-world scenarios. Through a multi-resolution map encoding module, this method can efficiently capture local and global scene context information. Utilizing the Vision Transformer module, which tightly integrates map information with vehicle states, it models the complex relationships between vehicles and the scene at different scales.

[0075] The future trajectory predictions generated by this invention not only exhibit high accuracy in the short term but also maintain stability and consistency in long-term predictions, making vehicle trajectory prediction more reliable. Through innovative methods and module design, this invention significantly improves the accuracy and rationality of vehicle trajectory prediction, is applicable to complex traffic scenarios, and possesses good flexibility and scalability, demonstrating broad application prospects.

[0076] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0077] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0078] Figure 1 This is an overall flowchart of the trajectory prediction method of the present invention;

[0079] Figure 2 This is a schematic diagram of the vehicle prediction coding module of the present invention;

[0080] Figure 3 This is a schematic diagram of the map encoding module of the present invention;

[0081] Figure 4This is a schematic diagram of the vehicle prediction decoding module of the present invention. Detailed Implementation

[0082] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0083] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0084] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0085] Please see Figures 1-4 This is a vehicle trajectory prediction method based on interactive perception and map adaptation.

[0086] Example

[0087] This embodiment discloses a specific implementation of a vehicle trajectory prediction method based on interaction perception and map adaptation. This method employs the Transformer approach to model the interaction relationships between dynamic objects in a scene and incorporates map information when decoding the state sequence. This fusion process is achieved through a hierarchical attention mechanism, where the vehicle's state dynamically adjusts the weights and contributions of past trajectories and scene encoding in the final prediction result in an autoregressive manner. Furthermore, a multi-resolution map encoding module is constructed. This module fully utilizes the Vision Transformer module to efficiently capture local and global scene context information, thereby closely associating map representation with vehicle state to accurately model the interaction process between the vehicle and the scene, thus generating more reasonable future trajectory prediction results. Figure 1 As shown, the method specifically includes the following steps:

[0088] S1: Collect historical vehicle trajectory data and geographic semantic information, and preprocess the data to ensure continuity and accuracy;

[0089] S2: The Transformer encoder is used to model the interaction between vehicles, generate embedding vectors, and process the input sequence through a multi-head self-attention mechanism and residual connections to capture the complex interaction features between vehicles.

[0090] S3: Employs a multi-resolution map encoding module and Vision Transformer to segment map image blocks and encode map features, and combines location encoding to process map information and generate global and local embeddings;

[0091] S4: Estimate future trajectories based on contextual information and joint distribution. Combine vehicle historical status, interaction features and map information, use Transformer decoder to calculate prediction probability distribution, and generate final trajectory prediction.

[0092] In step S1 of this embodiment, historical trajectory data of the vehicle is comprehensively collected through sensors and the vehicle-mounted system. This data covers key indicators such as vehicle location, speed, and acceleration, as well as geographic semantic information such as road networks, terrain, buildings, and landmarks in the target area. This data is then organized into a sequence of data. Based on the precise position of the vehicle in the global coordinate system and the positioning information of surrounding traffic participants in their respective local coordinate systems, the position information of the surrounding traffic participants is uniformly mapped to the global coordinate system.

[0093] In addition, step S1 includes preprocessing the collected historical trajectory data and geographic semantic information data, including noise removal, standardization, and interpolation, to ensure the continuity and accuracy of the data.

[0094] In step S2 of this embodiment, the trajectory prediction coding module is used to obtain vehicle trajectory features, such as... Figure 2 As shown, the specific steps include:

[0095] S21. Use the Transformer encoder to model the interaction between vehicles and generate the historical trajectory embedding vector for each vehicle.

[0096] Specifically, historical trajectory sequences are obtained by modeling the interactions between vehicles using a Transformer encoder. Where A represents the total number of vehicles, and τ is the time step and the state of each vehicle. Add index information I a d is then obtained through a linear layer transformation. m The embedding vector of dimension is reorganized into a historical trajectory embedding vector. The calculation method is as follows:

[0097]

[0098]

[0099] Among them, E 1 (t),E 2 (t),...,E A (t) represents the encoded information for each vehicle. This indicates a splicing operation, and Linear indicates a linear connection; the sequence covers the trajectories of all vehicles within the observation range, and the variable number of vehicles can be flexibly handled through index allocation.

[0100] S22. Combine the historical trajectory embedding vector with the learned position encoding;

[0101] Specifically, the learned position code refers to a set of position vectors obtained through training, which can provide additional temporal information about the vehicle state at each time step; the process of combining the historical trajectory embedding vector with the position code is to concatenate the historical embedding S′ with the learned position code PE according to the time steps, thereby obtaining a joint representation H with temporal information and vehicle information, and then pass it to a Transformer module.

[0102]

[0103]

[0104]

[0105]

[0106] Where pos represents the location information, v represents the dimension, and ξ represents the position information. total Given the total dimension of the input embedding, Features of the vehicle.

[0107] S23. In the Transformer module, the input sequence is processed by a multi-head self-attention mechanism and a fully connected feedforward network to capture the complex interaction features between vehicles.

[0108] Specifically, the Transformer encoder consists of multiple identical TEBlocks stacked together. Each TEBlock contains a multi-head self-attention mechanism and a fully connected feedforward network. Furthermore, each TEBlock's multi-head self-attention mechanism and fully connected feedforward network are followed by a normalized residual connection to efficiently process and encode the input sequence. In the first TEBlock, the encoded input sequence is first processed through the multi-head self-attention layer, and then the processing result is passed to the fully connected feedforward neural network. The feature vectors at each time step are first learned in the multi-head self-attention layer. Describe the characteristics of a vehicle; and A scaled dot product is performed to normalize the weights of the spatial edges between vehicle i and its surrounding vehicles j, and then the summation is used to obtain the attention weight of a single attention head at node i. The calculation method is as follows: The three vectors above are the query vectors for the i-th vehicle at time t. key vector value vector W q W k W v These are three learnable weight matrices. The attention weights... The input is processed by a fully connected feedforward layer, which performs nonlinear processing to obtain the output 'a' of each attention head. i Then, after passing through the normalized residual module after the fully connected feedforward layer for connection processing, the final attention output is obtained:

[0109]

[0110] Among them, LayerNorm(·) represents the layer normalization operation.

[0111] S24. Employ a multi-head self-attention mechanism to enhance the modeling of complex interaction relationships;

[0112] Specifically, to enhance the modeling of complex interaction relationships from multiple perspectives, a multi-head self-attention mechanism is employed to aggregate information from different feature subspaces. This process is performed λ times, and a fully connected layer is used to process the output of each attention head. The inputs of each TEBlock are merged to form the final multi-head attention output. Each TEBlock employs residual connections, linking the inputs of each layer to its output, and merging them before normalization at the application layer, enabling the capture of complex interactions between vehicles. S , φ S Characteristics representing interactions between vehicles:

[0113]

[0114] Among them, W o It is a learnable weight matrix; This represents the total attention weights calculated each time, and concat(·) represents the concatenation operation.

[0115] In step S3 of this embodiment, a multi-resolution map encoding module and a Vision Transformer are used to segment map image patches and encode map features. Combined with location encoding, map information is processed to generate global and local embeddings, such as... Figure 3 As shown, the specific steps include:

[0116] S31. Use the multi-resolution map encoding module to segment the map image into several image patches; and use VisionTransformer to process the image patches to generate global and local feature embeddings of the map.

[0117] Specifically, when processing map information using the Vision Transformer, the map image is segmented into a series of image patches. These patches are then processed by the Vision Transformer's encoder to capture global and local features of the map. First, the map information in the traffic scene is processed by segmenting the image into a series of first-class image patches, generating feature maps. These feature maps are used to generate local and global embeddings. Finally, these image patches are transformed to dimension d using a linear layer. m This creates a global embedding Γ for the entire scene. g Then, extract second-class image patches of size M×M, where M is smaller than the size of the feature map. Perform linear mapping on these image patches to transform the dimensions to d. m Thus, the local embedding Γ is obtained. l .

[0118] S32. Use a binary mask to represent the map of the drivable area, and add pixel index and distance information;

[0119] Specifically, for map information, simple semantics are used to help the model avoid unrealistic predictions, such as for drivable areas. This invention uses a binary mask to represent the map of drivable areas, adding a pixel index and the Euclidean distance between each pixel and the map center in each channel to further enrich the map information, allowing the model to directly obtain the spatial information of each pixel. By linearly interpolating each edge through a sparse global graph, a dense topological map is created. Such topological maps can also be created automatically or semi-automatically. Utilizing relatively simple and low-cost topological maps to achieve effective trajectory prediction and navigation functions makes our method more flexible and scalable in practical applications.

[0120] S33. In order to capture spatial location information, location encoding is added to the embedding vector. The sequences of global embedding and local embedding are input into the Vision Transformer to generate map features that represent the different relationships between vehicles and the map.

[0121] Furthermore, to process sequences sequentially, a positional encoding E′ = PE + E is added to the embedding vector so that the model can capture information from different locations in the image. The global embedding Γ is then applied. g and local embedding Γ l The sequence is input into the VisionTransformer to generate a series of map features φ m , φ m =ViT(concat(Γ) g ,Γ l The output is pattern features, with each pattern having different features corresponding to different relationships between the target vehicle and the map. To ensure the diversity of these patterns, different parts of the map are considered, and the loss is backpropagated using a single attention head that is closest to the true value in terms of the final displacement error. Here, the query vector is the interaction feature between vehicles, while the key and value vectors are feature vectors from the map encoder.

[0122] In step S4 of this embodiment, the future trajectory is estimated based on contextual information and joint distribution. Combining vehicle historical state, interaction features, and map information, a Transformer decoder is used to calculate the predicted probability distribution and generate the final trajectory prediction, as shown below. Figure 4 As shown, the specific steps include:

[0123] S41. Match vehicle interaction features with map features, process the combined features through a fully connected layer to form the final feature representation;

[0124] The goal of the prediction is to estimate the joint distribution of future trajectories for all vehicles based on contextual information, which is the observed state φ.S and map φ m To estimate the joint distribution, each vehicle uses ω discrete patterns, each represented as a one-hot binary vector. Road features are input into a fully connected layer, and max pooling is used to aggregate information from the same lane. Lane features are processed by another fully connected layer. Waypoint features, aggregated features, and lane features are concatenated and passed through a fully connected layer to obtain the final feature vector of the waypoints. Similar to interaction modeling, vehicle indices are concatenated to the vehicle's pattern representation of size K. Then, all vectors are flattened to form a vector of size A×K with dimension 2 for all vehicles. These vectors are then passed through a linear layer and coupled with φ... S Forming the intention to embed φ before connection i This intent embedding is based on the vehicle's historical state, interaction features, and map information, capturing the vehicle's intent in the current traffic scenario.

[0125] S42. Use the Transformer decoder to calculate the predicted trajectory distribution, and further refine the prediction through multiple cross-attention modules. Generate trajectory coordinates for each time step, and generate a probability distribution from the prediction results using softmax.

[0126] Specifically, based on contextual features φ S and φ m The joint distribution Z is calculated. S Each vehicle uses K learnable vectors V. The index of each vehicle is also concatenated with this vector and projected onto d using a linear layer. m The dimension is calculated by inputting this sequence into the Transformer decoder block TDBlock to compute all Z dimensions. S The TDBlock contains two cross-attention modules to calculate the probability of a given trajectory. This approach allows the model to learn different driving patterns in an unsupervised manner, representing various driving behaviors and decisions such as going straight, turning left, turning right, accelerating, and decelerating. The predicted vehicle trajectory depends on three features: the target vehicle's historical state, vehicle interactions, and map information. The shape of the map features is matched along the pattern axis based on the interaction features and target vehicle features, and then these three features are concatenated to form the final feature representation. The trajectory decoder outputs the trajectory coordinates at each time step, and the scores of all predicted trajectories are grouped and passed through a softmax layer to generate a probability distribution.

[0127] P(Z s |φ)=Softmax(FFN(TDBlack d (V,φ)))

[0128]

[0129] Where φ represents φ S and φ m ρ is a learnable vector, TDBlack d (·) represents a TDBlock block, and FFN(·) represents a fully connected layer.

[0130] S43. Optimize parameters using maximum likelihood estimation to maximize the log-likelihood of marginal data, ensuring model accuracy and stability.

[0131] Specifically, each possible future trajectory of the vehicle depends on a pattern that better aligns with the target of the entire trajectory intent in the latent space. Since layer normalization delays convergence, the second decoder block does not employ layer normalization. At each time step, the trajectory prediction for the target vehicle is conditioned on all other vehicles. The output sequence is then passed to N feedforward layers to generate parameters of a binary Gaussian distribution. Maximum likelihood estimation is used to find the parameter θ that maximizes the log-likelihood of the marginal data. The optimized parameters are then used for trajectory prediction to obtain the final prediction result.

[0132]

[0133]

[0134] in, Indicates the predicted trajectory, It is the average value. It is the standard deviation;

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A vehicle trajectory prediction method based on interactive perception and map adaptation, characterized in that: The prediction method includes the following steps: S1: Collect historical vehicle trajectory data and geographic semantic information, and preprocess the collected historical trajectory data and geographic semantic information; S2: The Transformer encoder is used to model the interaction between vehicles, generate embedding vectors, and process the input sequence through a multi-head self-attention mechanism and residual connections to capture vehicle features; S3: Employs a multi-resolution map encoding module and Vision Transformer to segment map image blocks and encode map features, and combines location encoding to process map information and generate global and local embeddings; S4: Estimate future trajectories based on contextual information and joint distribution. Combine vehicle historical status, interaction features and map information, use Transformer decoder to calculate prediction probability distribution, and generate final trajectory prediction.

2. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 1, characterized in that: In step S1, historical trajectory data and geographic semantic information of the vehicle are collected through sensors and vehicle systems. The historical trajectory data includes vehicle location information, speed, and acceleration; the geographic semantic information includes the road network, terrain, buildings, and landmarks of the target area. Historical trajectory data and geographic semantic information data are organized into sequence data. Based on the vehicle's position information in the global coordinate system and the positioning information of surrounding traffic participants in their respective local coordinate systems, the position information of surrounding traffic participants is uniformly mapped to the global coordinate system. The collected historical trajectory data and geographic semantic information are then preprocessed, including noise removal, standardization, and interpolation.

3. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 1, characterized in that: Step S2 includes the following steps: S21. Model the interactions between vehicles using a Transformer encoder to generate historical trajectory embedding vectors for each vehicle; specifically, modeling the interactions between vehicles using a Transformer encoder yields historical trajectory sequences. Where A represents the total number of vehicles, τ is the time step, and the state of each vehicle is... Add index information I a d is then obtained through a linear layer transformation. m The embedding vector of dimension is reorganized into a historical trajectory embedding vector. Among them, E 1 (t),E 2 (t),...,E A (t) represents the encoded information for each vehicle. This indicates a concatenation operation; Linear indicates a linear connection. S22. Combine the historical trajectory embedding vector with the learned position code; whereby the learned position code refers to obtaining a set of position vectors through training that can provide additional temporal information about the vehicle state at each time step; the process of combining the historical trajectory embedding vector with the position code is to combine the historical embedding vector with the learned position code. The learned location code PE is concatenated with the time step to obtain a joint representation H containing temporal information and vehicle information, and then passed to a Transformer module; Where pos represents the location information, v represents the dimension, and ξ represents the position information. total The total dimension of the input embedding; Vehicle characteristics; S23. In the Transformer module, the input sequence is processed by a multi-head self-attention mechanism and a fully connected feedforward network to capture vehicle features. S24. A multi-head self-attention mechanism is employed to enhance the modeling of complex interaction relationships. This mechanism aggregates information from different feature subspaces, performing this process λ times. A fully connected layer is used to process the output of each attention head. i t The results are merged to form the final multi-head attention output φ. S : Among them, W o It is a learnable weight matrix; This represents the total attention weights calculated each time, and concat(·) represents the concatenation operation.

4. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 3, characterized in that: In step S23, the Transformer encoder includes multiple identical TEBlocks, each TEBlock including a multi-head self-attention layer and a fully connected feedforward network, and each multi-head self-attention mechanism and fully connected feedforward network is followed by a normalized residual connection; In the first TEBlock, the feature vectors at each time step are first learned in the multi-head self-attention layer: in, Let be the query vector, key vector, and value vector of the i-th vehicle at time t, respectively. W indicates the characteristics of the vehicle. q W k W v There are three learnable weight matrices; right and A scaled dot product is performed to normalize the weights of the spatial edges between vehicle i and its surrounding vehicles j, and then the summation is used to obtain the attention weight of a single attention head at node i. The calculation method is as follows: Then adjust the attention weight The input is processed by a fully connected feedforward layer, which performs nonlinear processing to obtain the output 'a' of each attention head. i Then, after passing through the normalized residual module after the fully connected feedforward layer, the final attention output a is obtained. final : Among them, LayerNorm(·) represents the layer normalization operation.

5. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 3, characterized in that: Step S3 includes the following steps: S31. Use the multi-resolution map encoding module to segment the map image into several image patches; and use VisionTransformer to process the image patches to generate global and local feature embeddings of the map. S32. Use a binary mask to represent the map of the drivable area, and add pixel index and distance information; S33. Add positional encoding to the embedding vector to capture spatial positional information, and then add global embedding Γ. g and local embedding Γ l The sequence is input into the Vision Transformer to generate map features φ representing different relationships between vehicles and the map. m .

6. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 5, characterized in that: In step S31, the map information in the traffic scene is processed, the image is segmented into a series of first-class image patches, and a feature map is generated based on the first-class image patches. The feature map is used to generate local and global embeddings. In this process, the first type of image patch is transformed in dimension d through a linear layer. m To obtain the global embedding Γ of the entire scene g ; Further, extract a second type of image patch of size M×M from the first type of image patch, where the second type of image patch is smaller than the first type of image patch. Perform a linear mapping on the second type of image patch to transform its dimension to d. m Thus, the local embedding Γ is obtained. l ; In step S32, a binary mask is first used to represent the drivable area map, and a pixel index and the Euclidean distance between each pixel and the center of the map are added to each channel. Secondly, a topological map is created automatically or semi-automatically by linearly interpolating each edge through a sparse global graph. In step S33, positional encoding is added to the embedding vector: E′=PE+E Where E represents the processed embedding vector, and PE is the positional encoding; Global embedding Γ g and local embedding Γ l The sequence is input into the Vision Transformer to generate a series of map features φ m ; φ m =ViT(concat(Γ g ,Γ l )) Where ViT(·) represents Vision Transformer; The loss is backpropagated using a single attention head that is closest to the true value in terms of the final displacement error, where the query vector is the interaction feature between vehicles, and the key vector and value vector are feature vectors from the map encoder.

7. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 5, characterized in that: Step S4 includes the following steps: S41. Match vehicle interaction features with map features, process the combined features through a fully connected layer to form the final feature representation; S42. Use the Transformer decoder to calculate the predicted trajectory distribution, and further refine the prediction through multiple cross-attention modules. Generate trajectory coordinates for each time step, and generate a probability distribution from the prediction results using softmax. S43. Optimize the parameters using maximum likelihood estimation to maximize the log-likelihood of the marginal data. Use the optimized parameters to predict the trajectory and obtain the final prediction result.

8. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 7, characterized in that: In step S41, vehicle interaction features are matched with map features, and the combined features are processed through a fully connected layer to form the final feature representation. The process is as follows: Each vehicle uses ω discrete patterns, each pattern represented as a one-hot binary vector. Road features are input into a fully connected layer, and max pooling is used to aggregate information from the same lane. Lane features are processed by another fully connected layer, which concatenates waypoint features, aggregate features and lane features, and then passes them through a fully connected layer to obtain the final feature vector of the waypoints; The vehicle index is concatenated to the vehicle's pattern representation of size K. Then, all vectors are flattened to form a vector of size A×K with dimension 2 for all vehicles. This vector is then passed through a linear layer and connected to φ. S The connection forms an intention to embed φ i Intending to embed φ i Based on vehicle history, interaction features, and map information, the system captures the vehicle's intent in the current traffic scenario.

9. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 8, characterized in that: In step S42, based on the context feature φ S and φ m The joint distribution Z is calculated. S Each vehicle uses K learnable vectors P; The index of each vehicle is also concatenated with vector P and projected onto d using a linear layer. m Dimension, input the sequence into the Transformer decoder block TDBlock, and compute all Z... S The probability of the above; where TDBlock contains two cross-attention modules; The predicted vehicle trajectory depends on three features: the target vehicle's historical state, the interactions between vehicles, and map information. The shape of the map features is matched along the pattern axis based on the interaction features and the target vehicle features. Then, these three features are connected to form the final feature representation. The trajectory decoder outputs the trajectory coordinates at each time step. The scores of all predicted trajectories are grouped and passed through a softmax layer to generate a probability distribution. P(Z s |φ)=Softmax(FFN(TDBlack d (ρ,φ))) Where φ represents φ S and φ m ρ is a learnable vector, TDBlack d (·) represents a TDBlock block, and FFN(·) represents a fully connected layer.

10. The vehicle trajectory prediction method based on interactive perception and map adaptation according to claim 9, characterized in that: In step S43, at each time step, the trajectory prediction of the target vehicle is conditioned on all other vehicles. The output sequence is then passed to N feedforward layers to generate parameters of a binary Gaussian distribution. Maximum likelihood estimation is used to find the parameter θ that maximizes the log-likelihood of the marginal data. The optimized parameters are then used for trajectory prediction to obtain the final prediction result.

Images