Vehicle multimodal trajectory prediction method and system considering spatiotemporal interaction information

Abstract

This invention provides a method and system for multimodal trajectory prediction of vehicles considering spatiotemporal interaction information, relating to the field of trajectory prediction for autonomous vehicles. The method includes: collecting scene information during vehicle operation; selecting a spatiotemporal attention model as a base model, inputting the scene information into the base model for encoding to obtain encoded information fused with the spatiotemporal attention mechanism and social grid information of surrounding vehicles; selecting an autonomous driving intention prediction model, fusing the encoded information and social grid information and inputting them into the autonomous driving intention prediction model to obtain the predicted probability of the vehicle's own intention in the future time domain; constructing a decoder; inputting the concatenated intention prediction probability and encoded information into the decoder to decode and obtain the predicted trajectory under the corresponding intention. This invention can realize multimodal trajectory prediction of intelligent driving vehicles in future moments, including lateral and longitudinal motivations, providing support for subsequent path planning and decision-making.

Description

Technical Field

[0001] This invention relates to the field of autonomous vehicle trajectory prediction, and in particular to a method and system for predicting vehicle multimodal trajectories that takes into account spatiotemporal interaction information. Background Technology

[0002] In the fields of transportation and autonomous driving, trajectory prediction is a crucial task, aiming to infer the potential location and trajectory of traffic participants (such as vehicles, pedestrians, and cyclists) at future moments based on historical observations and environmental information. High-accuracy trajectory prediction is an important prerequisite for achieving safe and efficient decision-making (decision planning, collision avoidance, speed control, etc.).

[0003] Traditional prediction methods based on kinematics or rules can achieve accurate short-term modeling by using common uniform or accelerated models, but accuracy is difficult to guarantee in complex scenarios and long-term prediction tasks.

[0004] Deep learning-driven prediction methods can capture complex nonlinear relationships in data. By building deep networks and training them with massive amounts of data, they can achieve predictions in multi-dimensional feature scenarios. Sequence models based on RNNs (such as LSTM and GRU) can better capture the dynamic changes of time series and perform multi-step predictions by combining scene information. In recent years, Transformers and Graph Neural Networks (GNNs) have also been widely used to more flexibly model scene interactions and temporal features.

[0005] However, in the real world, the same historical information may lead to multiple future trajectories (such as going straight or changing lanes), requiring the explicit expression of different prediction branches. Due to the uncertainty of traffic scenarios in real life, trajectory prediction emphasizes the prediction distribution rather than the prediction result of a single trajectory.

[0006] To address the shortcomings of existing trajectory prediction methods in modeling multimodal and uncertainties, which often fails to fully represent the various potential future movement patterns of traffic participants, deep neural networks have become the mainstream approach for high-precision trajectory prediction. This approach combines deep neural networks with modeling of scene encoding, intent prediction, social interaction, and multimodal uncertainty. It provides safer and more efficient support for intelligent decision-making in autonomous driving and complex traffic scenarios. Summary of the Invention

[0007] Purpose of the invention: To propose a multimodal trajectory prediction method for vehicles that considers spatiotemporal interaction information, and to propose a system that can automatically execute the above prediction method. The aim is to realize the multimodal trajectory prediction of intelligent vehicles in future moments, including lateral and longitudinal motivations, based on the historical information of the traffic scene in which the intelligent vehicle is located, so as to provide support for the subsequent path planning and decision-making of intelligent vehicles.

[0008] In a first aspect, the present invention proposes a method for predicting vehicle multimodal trajectories considering spatiotemporal interaction information, comprising the following steps:

[0009] Collect scene information while the vehicle is in motion; the scene information includes the vehicle's own trajectory information and the trajectory information of surrounding vehicles within a predetermined range before and after adjacent lanes when traveling in the same direction.

[0010] An open-source spatiotemporal attention model is selected as the base model. The scene information is imported into the base model for encoding to obtain the encoded information H that integrates the spatiotemporal attention mechanism. enc And social network information of surrounding vehicles h nbrs ;

[0011] We selected an open-source autonomous driving intent prediction model and fused it with encoded information H. enc and social grid information h nbrs And input it into the autonomous driving intention prediction model to obtain the intention prediction probability P of the vehicle in the future time domain;

[0012] Construct a decoder; combine the concatenated intent prediction probability P and encoded information H enc The input is fed into the decoder, and the predicted trajectory under the corresponding intention is obtained.

[0013] In a further embodiment of the first aspect, the scene information I includes the trajectory information I of the vehicle itself. ego And the trajectory information of surrounding vehicles within a predetermined range before and after adjacent lanes traveling in the same direction. nbrs :

[0014] I = [I ego I nbrs ]

[0015] Trajectory information of its own vehicle I ego ={x,y,v x ,v y ,a x ,a y ,lane};

[0016] Where x represents the vehicle's horizontal coordinate information, y represents the vehicle's vertical coordinate information, and v x ,v y These are the velocity components of the vehicle in the lateral and longitudinal directions, respectively, ax ,a y These represent the acceleration components of the vehicle in the horizontal and vertical directions, respectively, and lane indicates the lane number.

[0017] In a further embodiment of the first aspect, the probability P of predicting the intention of the vehicle in the future time domain is P = {P lon ,P lat};

[0018] Among them, P lon P represents the probability of longitudinal driving intention, including the probability of accelerating, maintaining speed, and decelerating; lat This represents the probability of lateral driving intention, including the probability of changing lanes to the left, changing lanes to the right, and going straight.

[0019] In a further embodiment of the first aspect, the scene information is imported into the base model for encoding, specifically including:

[0020] The trajectory information of its own vehicle I ego The embedded state e of the vehicle is obtained by embedding representation through the encoder. ego :

[0021] e ego =MLP(I ego W emb )

[0022] Track information of surrounding vehicles nbrs The embedding state e of surrounding vehicles is obtained by embedding representation using an encoder. nbrs :

[0023] e nbrs =MLP(I nbrs W emb )

[0024] In the formula, MLP(*) is a multilayer perceptron, W emb These are the weight coefficients of the embedding layer;

[0025] Embedding its own vehicle into state e ego Embedded state of surrounding vehicles e nbrs The trajectory features are encoded by inputting them into the LSTM layer to obtain their respective hidden layer feature vectors θ:

[0026] θ ego =LSTM(e ego W emb )

[0027] θ nbrs =LSTM(e nbrs W emb )

[0028] In the formula, LSTM(*) represents encoding through an LSTM layer; θ egi θ represents the hidden layer feature vector of the vehicle after LSTM encoding; nbrs This represents the hidden layer feature vector of the surrounding vehicles after LSTM encoding.

[0029] In a further embodiment of the first aspect, the scene information is imported into the base model for encoding to obtain encoded information H that integrates the spatiotemporal attention mechanism. enc :

[0030]

[0031] In the formula, h represents the output of the time attention mechanism. The output of the spatial attention mechanism is represented by α, β, and γ, which are weight parameters for lesson learning and used to control the contribution of each part.

[0032] In a further embodiment of the first aspect, the scene information is imported into the base model for encoding to obtain the social grid information h of the surrounding vehicles. nbrs Specifically, it includes:

[0033] The mask matrix is ​​used to analyze the hidden feature vectors θ of surrounding vehicles after layer normalization. layernorm Sparse extraction is performed to obtain the social network information h of surrounding vehicles. nbrs :

[0034] h nbrs =θ layernorm ⊙mask

[0035] Among them, the normalized hidden layer feature vector of surrounding vehicles Let μ be the hidden layer feature vector of the surrounding vehicles at the i-th time step, and σ be the mean of the hidden vector. 2 Let represent the variance, and ∈ be a constant.

[0036] In a further embodiment of the first aspect, obtaining the predicted probability P of the vehicle's intention in the future time domain specifically includes:

[0037] Fusion of encoded information H through attention mechanism enc and social grid information h nbrs :

[0038] H fused =Attention(H enc ,h nbrs )

[0039] Where H fused This represents the fused features, and Attention(*) is the attention mechanism.

[0040] The fused feature H is processed through a fully connected layer. fused Perform feature extraction:

[0041] H trans =ReLU(H fused W trans +b trans )

[0042] Where H trans W represents the extracted features. trans It is the weight matrix of the fully connected layer, b trans It is the bias vector, and ReLU(*) is the activation function;

[0043] Predicting the high-level intention category P of traffic participants high :

[0044] P high =Softmax(H trans W high +b high )

[0045] W high For the weight matrix of high-level intent classification, b high It is the bias vector;

[0046] Based on high-level intent category P hig The probability P of the longitudinal driving intention is obtained by refining the details. lon Probability P of lateral driving intention lat :

[0047] P lat ={p lc p rc p kl} = Softmax(H lat W lat +b kat )

[0048] P lon ={p acc p keep p dec} = Softmax(H lon W lon +b lon )

[0049] Among them, W lat W lon The weight matrix for low-level intent classification, b lat 、b lon This is the bias vector.

[0050] In a further embodiment of the first aspect, the decoder has six branches, each corresponding to an intent;

[0051] After receiving shared features, the six branches independently decode and generate the corresponding future trajectories. The structure of each branch is as follows:

[0052]

[0053] In the formula, These represent the hidden state and the memory state of branch k at time step t, respectively. LSTM k This represents the LSTM network with the k-th branch. The predicted output coordinates of the k-branch at time step t. For the output mapping parameters;

[0054] After recursively generating at time step T, the multimodal trajectory prediction results under six intentions are obtained through integration.

[0055]

[0056] In the formula, These are respectively: left lane change intention, right lane change intention, left acceleration lane change intention, right acceleration lane change intention, straight driving at constant speed intention, and deceleration intention.

[0057] In a second aspect, the present invention provides a vehicle multimodal trajectory prediction system, the prediction system comprising at least one processing module and a memory storing computer program instructions; the at least one processing module, when executing the computer program instructions, implements the vehicle multimodal trajectory prediction method considering spatiotemporal interaction information as described in the first aspect.

[0058] A third aspect of the invention provides a computer-readable storage medium storing at least one executable instruction that, when executed on an electronic device, causes the electronic device to perform the vehicle multimodal trajectory prediction method considering spatiotemporal interaction information as described in the first aspect.

[0059] Compared with the prior art, the present invention has at least the following beneficial effects:

[0060] (1) By separating the base model and the sub-model, the overall trajectory prediction task is divided into two independent modules: the base model focuses on encoding historical data within the scene, extracting spatiotemporal interaction information, and abstracting and representing the global picture; while the sub-model generates future candidate trajectories based on the output of the base model. This modular design facilitates independent optimization of the system, reduces the complexity of model operation, and enables the system to have better scalability and flexibility. The independent design between modules facilitates debugging and optimization of each link and makes it easy to deploy in large-scale real-time scenarios.

[0061] (2) A multi-level coarse-grained and fine-grained intent classification and recognition system is introduced: coarse-grained determination is made through high-level intent classification (lane keeping / lane changing), and then finer-grained intent classification is achieved by jointly predicting lateral and longitudinal intents, which helps guide the discretization of future trajectories. In addition, a model training loss function for multi-level classification is proposed to improve the semantic interpretability and practical usability of intent prediction results. Attached Figure Description

[0062] Figure 1 This is an overall architecture diagram of the vehicle multimodal trajectory prediction method in this embodiment of the invention. Detailed Implementation

[0063] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0064] This embodiment discloses a vehicle multimodal trajectory prediction method considering spatiotemporal interaction information, the steps of which are as follows:

[0065] S1. Collect scene information while the vehicle is driving, including the vehicle's own trajectory information and the trajectory information of surrounding vehicles within 100 meters before and after it in the same direction of travel in the adjacent lane, that is:

[0066] I = [I ego I nbrs ]

[0067] I ego ={x,y,v x ,v y ,a x ,a y ,lane}

[0068] In the formula, I ego I nbrs These are the trajectory information of the target vehicle and surrounding vehicles, respectively, in I... ego For example, x represents the vehicle's horizontal coordinate information, y represents the vehicle's vertical coordinate information, and v x ,v y These are the velocity components of the vehicle in the lateral and longitudinal directions, respectively, a x ,a y These represent the acceleration components of the vehicle in the horizontal and vertical directions, respectively, and lane indicates the lane number.

[0069] S2. Based on the data collected in S1, the collected scene information is encoded using the base model to obtain the encoded information H of the fusion spatiotemporal attention mechanism. enc And social network information h representing surrounding vehicles nbrs The base model here uses an open-source spatio-temporal attention model, such as STAR (Spatio-Temporal Attention Network), ST-GCN (Spatial-Temporal Graph Convolutional Networks), or Trajectory Transformer.

[0070] S3, Encoded information H obtained based on S2 encoding enc Social grid information h nbrs The input to the autonomous driving intent prediction model is pooled to obtain the predicted probability of the vehicle driving in the future time domain, P = {P lon ,P lat}, where the probability of longitudinal driving intention P lon Including p acc p keep p dec And the probability P of lateral driving intention lat Including p lc p rc p kl Where p acc p keep p dec Let p represent the probabilities of accelerating, maintaining speed, and decelerating, respectively. lc p rc p kl These represent the probabilities of changing lanes to the left, right, and going straight, respectively. The autonomous driving intent prediction model adopts an open-source model, which can be CAR-Net (Context-Aware Road Network), IntentNet, PRECOG (PREdiction Conditioned On Goals), or MultiPath.

[0071] S4. Obtain the intention prediction probability from S3 and the scene encoding information H obtained from S2. enc The data are concatenated and then input into each sub-model to decode and obtain the predicted trajectory under the corresponding intent.

[0072] In a preferred embodiment, in S1, information about the vehicle and its surrounding environment is collected by installing a positioning module, speed sensor, acceleration sensor, lidar, camera and radar sensor on the vehicle. This includes vehicle driving information, lane information, the status information of vehicles in the lane where the vehicle is located, and the relative positions of vehicles within 100 meters before and after the adjacent lane.

[0073] In a preferred embodiment, in S2, the base model is a spatiotemporal attention mechanism model, and the encoding process of the base model includes:

[0074] S2.1. Perform feature encoding on the input data, including:

[0075] S2.1.1. For information about the vehicle itself and neighboring vehicles, e is obtained by embedding representation using their respective encoders. ego ,e nbrs :

[0076] e = MLP(I,W emb )

[0077] In the formula, e represents the embedded state, MLP is a multilayer perceptron, and W emb These are the weight coefficients for the embedding layer.

[0078] S2.1.2. Input the embedded states of different types of traffic participants obtained in S2.1.1 into the LSTM layer for trajectory feature encoding, and perform layer normalization. Taking the vehicle itself as an example, the process is as follows:

[0079] θ ego =LSTM(e ego W emb )

[0080]

[0081] In the formula, LSTM(·) represents encoding through an LSTM layer, and W emb θ is the weighting coefficient. ego This represents the hidden layer feature vector of the vehicle after LSTM encoding. Let μ be the hidden layer feature vector at the i-th time step, and σ be the mean of the hidden vector. 2 Represents the variance, where ∈ is a very small constant used to prevent division by zero errors. h i This represents the vector information at the i-th time step after layer normalization.

[0082] S2.1.3. Sparse extraction of neighbor vehicle encoding information from the social grid. This invention defines a social grid centered on the vehicle itself, with a size of 3*41 and a single unit length of 5 meters, to represent spatial feature information. Considering the sparsity of the social grid, the state information of vehicles that are not empty within the grid position is extracted. The expression is as follows:

[0083]

[0084] h nbrs =θ layernorm ⊙mask

[0085] In the formula, mask is a mask matrix used to represent information about the presence of vehicles in the social grid; 1 indicates that there is a vehicle at that location at the current time, and 0 indicates that there is no vehicle at the current time. θ layernorm Neighbor vehicle information after layer normalization, h nbrs This is the sparsed neighbor vehicle encoding information.

[0086] S2.2 Input the coding information of the vehicle itself and neighboring vehicles into the spatial attention mechanism, focus on the spatial interaction between the vehicle itself and neighboring vehicles in each time step, and dynamically adjust the attention to different neighboring vehicles to quantify the behavioral interaction between vehicles.

[0087] S2.3. The spatial distribution feature information is input into the time attention mechanism to integrate these spatiotemporal features across the entire time series, capture temporal dependencies and global behavioral patterns, and form a global feature representation.

[0088] The aforementioned vehicle multimodal trajectory prediction method considering spatiotemporal interaction information, in S2.2, includes:

[0089] S2.2.1 Normalize the feature encoding of traffic participants in the social grid to ensure that each participant receives a consistent weight scale in the subsequent spatial attention mechanism.

[0090]

[0091] Among them, h i Let μ be the encoding information of the i-th grid among the neighboring vehicles obtained in S2.1.3, and let μ and σ represent the mean and standard deviation of the social grid encoding information, respectively.

[0092] S2.2.2, Vehicle's own code and normalized neighbor vehicle code information h ego h nbrs,norm The query, key, and value Q, K, V are obtained through projection, and the attention H is calculated using a multi-head attention mechanism. The formula is as follows:

[0093] Q = h nbrs,norm W Q

[0094] K = h ego W K

[0095] V = h ego W V

[0096] H=MultiHead(Q,K,V)={head1,head2,…head n}W O

[0097]

[0098] Among them W Q W K W V Let head represent the trainable weight matrix of the projection, respectively. i d represents the attention calculation result for each head. k W represents the dimension of the vector for each head. O It is a trainable weight matrix used to perform a linear transformation on the output of a multi-head system.

[0099] S2.2.3. Input the spatial feature information H extracted by the spatial attention mechanism into the dual-gated linear unit (D_GLU):

[0100]

[0101] Where W1 and W2 are trainable weight matrices, and b1 and b2 are bias vectors. σ is the sigmoid activation function used for the first gating, and φ is the tanh activation function used for the second gating operation. The first gating function transforms the input feature H into a gating vector between 0 and 1 using the sigmoid function, determining which feature information needs to be preserved or suppressed. The second gating function further transforms the features through non-linear activation, increasing the non-linear expressive power of the features to capture the complex patterns of the model. Then, the outputs of the two gating functions are multiplied element-wise with the original feature H, and residual connections are made.

[0102] As a preferred embodiment, S2.3 includes:

[0103] S2.3.1, All spatial feature information of the historical time domain The input is fed into a multi-head self-attention mechanism to capture the contribution relationship at each time step, resulting in a variable h containing time feature information.

[0104]

[0105] h = MultiHead(Q) t ,K t V t )

[0106] in These represent the trainable weight matrices of the projection.

[0107] S2.3.2 Input the temporal feature information h extracted by the temporal attention mechanism into the dual-gated linear unit (D_GLU):

[0108]

[0109] Where W3 and W4 are trainable weight matrices, and b3 and b4 are bias vectors.

[0110] S2.3.3, Weighted residual concatenation of temporal and spatial feature information to obtain H enc :

[0111]

[0112] Where h is the output of the time attention mechanism, The output of the spatial attention mechanism is represented by α, β, and γ, which are weight parameters for lesson learning and used to control the contribution of each part.

[0113] In a preferred embodiment, S3 specifically includes:

[0114] S3.1, convert the encoded information H obtained in S2 enc With social grid information h nbrs Feature fusion through attention mechanisms:

[0115] H fused =Attention(H enc ,h nbrs )

[0116] S3.2, The fused feature H is processed through a fully connected layer. fused Perform feature extraction:

[0117] H trans =ReLU(H fused W trans +b trans )

[0118] Among them, W trans It is the weight matrix of the fully connected layer, b trans It is the bias vector. ReLU is the activation function.

[0119] S3.3 Extracted Feature H transConstruct a hierarchical multi-task intent classification module to achieve refined intent prediction in layers:

[0120] S3.3.1 First, predict the high-level intention category P of traffic participants. high (Keep in lane / Change lanes):

[0121] P high =Softmax(H trans W high +b high )

[0122] Among them W high For the weight matrix of high-level intent classification, b high This is the bias vector.

[0123] S3.3.2. Based on high-level intent, further refine the prediction of specific horizontal and vertical intents. Introduce the idea of ​​multi-task learning, and achieve joint prediction of horizontal and vertical intents through shared and dedicated feature extraction layers:

[0124] H share =ReLU(H trans W share +b share )

[0125] H lat =ReLU(H share W lat +b lat )

[0126] H lon =ReLU(H share W lon +b lon )

[0127] Among them, W share W lat W lo These are the weight matrices for the shared feature extraction layer and the horizontal / vertical feature extraction layer, respectively. share 、b lat 、b lo This is the corresponding bias vector.

[0128] S3.3.3. Classify horizontal and vertical intentions, i.e., low-level intentions, respectively to obtain probability values:

[0129] P lat ={p lc p rc p kl} = Softmax(H lat W lat +b lat )

[0130] P lon ={p acc p keep p dec} = Softmax(H lon W lon +b lon )

[0131] Among them, W lat W lon The weight matrix for low-level intent classification, b lat 、b lon This is the bias vector.

[0132] S3.4 Calculate the intended loss In the model training process, a hierarchical loss function is introduced. It not only optimizes the classification accuracy of high-level intents but also improves the fine-grained classification accuracy of low-level intents; it introduces feature orthogonalization loss. To avoid feature interference between tasks and ensure the orthogonality of shared features and task-specific features in the feature space, the effectiveness of multi-task learning can be improved.

[0133]

[0134] in, The cross-entropy loss corresponds to high-level intent, lateral intent, and vertical intent, and the calculation formula for cross-entropy loss is shown in the following equation; λ high , λ low , λ ortho These correspond to the weighting coefficients of high-level intent, low-level intent, and orthogonalization loss.

[0135]

[0136] Where C represents the total number of categories, y = (y1, y2, ..., y c () is the one-hot encoding of the real tag. It is the output predicted probability distribution.

[0137] In a preferred embodiment, S4 specifically includes:

[0138] S4.1, the encoded information H of S2 is fused. enc And the intent probability P obtained from S3 is used to obtain the initial features H of the decoder. cond

[0139] S4.2, H cond The input decoder is designed with 6 branches corresponding to 6 intentions, generating future trajectories for each of the 6 intentions.

[0140] The aforementioned vehicle trajectory fusion prediction method based on deep learning and kinematic models, in S4.2, includes:

[0141] S4.2.1 Initial features are fed into the shared layer for feature extraction:

[0142] H share_out =MLP(H cond )

[0143] In the formula H share_out MLP stands for Multilayer Perceptron, which represents the shared features of the shared layer.

[0144] S4.2.2 After receiving the shared features, the six branches independently decode and generate the corresponding future trajectories. The structure of each branch is as follows:

[0145]

[0146] In the formula, These represent the hidden state and the memory state of branch k at time step t, respectively. LSTM k This represents the LSTM network with the k-th branch. The predicted output coordinates of the k-branch at time step t. These are the mapping parameters for the output.

[0147] S4.2.3 After recursively generating for T time steps, the multimodal trajectory prediction results under 6 intentions are obtained through integration.

[0148] Using the logic and process of the vehicle multimodal trajectory prediction method disclosed in the above embodiments, a vehicle multimodal trajectory prediction system can be formed. The prediction system includes at least one processing module and a memory storing computer program instructions. When the at least one processing module executes the computer program instructions, it implements the vehicle multimodal trajectory prediction method considering spatiotemporal interaction information disclosed in the above embodiments, which will not be elaborated here.

[0149] Furthermore, the vehicle multimodal trajectory prediction method disclosed in the above embodiments can, in practical applications, have its operational logic programmed into a program product, which is then written into a storage medium and run on an electronic device. More specific examples of the computer-readable storage medium mentioned in this embodiment may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable signal medium may also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0150] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A method for predicting vehicle multimodal trajectories considering spatiotemporal interaction information, characterized in that, Includes the following steps: Collect scene information while the vehicle is in motion; the scene information includes the vehicle's own trajectory information and the trajectory information of surrounding vehicles within a predetermined range before and after adjacent lanes when traveling in the same direction. The scene information I includes the vehicle's own trajectory information. And the trajectory information of surrounding vehicles within a predetermined range before and after adjacent lanes traveling in the same direction. : Trajectory information of its own vehicle ; Where x represents the vehicle's horizontal coordinate information, and y represents the vehicle's vertical coordinate information. These represent the velocity components of the vehicle in the horizontal and vertical directions, respectively. These represent the acceleration components of the vehicle in the lateral and longitudinal directions, respectively. Indicates the lane number; An open-source spatiotemporal attention model is selected as the base model. The scene information is imported into the base model for encoding to obtain the encoded information that integrates the spatiotemporal attention mechanism. And social network information of surrounding vehicles The scene information is imported into the base model for encoding, specifically including: The trajectory information of its own vehicle The embedding state of the vehicle is obtained by embedding representation using an encoder. : Track information of surrounding vehicles The embedding state of surrounding vehicles is obtained by embedding representation using an encoder. : In the formula, It is a multilayer perceptron. These are the weight coefficients of the embedding layer; Embedding its own vehicle Embedded state with surrounding vehicles The trajectory features are encoded by inputting them into the LSTM layer to obtain their respective hidden layer feature vectors. : In the formula, This indicates that encoding is performed through an LSTM layer; This represents the hidden layer feature vector of the vehicle after LSTM encoding. This represents the hidden layer feature vector of the surrounding vehicles after LSTM encoding. We selected an open-source autonomous driving intent prediction model and fused it with encoded information. and social grid information The information is then input into the autonomous driving intent prediction model to obtain the intent prediction probability P of the vehicle in the future time domain; the intent prediction probability P of the vehicle in the future time domain. ; in, This represents the probability of a longitudinal driving intention, including the probability of accelerating, maintaining speed, and decelerating; This represents the probability of lateral driving intention, including the probability of changing lanes to the left, changing lanes to the right, and going straight; Construct a decoder; combine the concatenated intent prediction probability P and the encoded information. The input is fed into the decoder, and the predicted trajectory under the corresponding intention is obtained.

2. The vehicle multimodal trajectory prediction method considering spatiotemporal interaction information according to claim 1, characterized in that, The scene information is imported into the base model for encoding to obtain the encoded information fused with the spatiotemporal attention mechanism. : In the formula, h represents the output of the time attention mechanism. For the output of the spatial attention mechanism, The weighting parameters for the lesson are used to control the contribution of each part.

3. The vehicle multimodal trajectory prediction method considering spatiotemporal interaction information according to claim 1, characterized in that, The scene information is imported into the base model for encoding to obtain the social grid information of surrounding vehicles. Specifically, it includes: The mask matrix is ​​used to analyze the hidden layer feature vectors of surrounding vehicles after layer normalization. Sparse extraction is performed to obtain the social network information of surrounding vehicles. : Among them, the normalized hidden layer feature vector of surrounding vehicles ; Let be the hidden layer feature vector of the surrounding vehicles at the i-th time step. To hide the vector mean, Let ϵ represent the variance, and ϵ be a constant.

4. The vehicle multimodal trajectory prediction method considering spatiotemporal interaction information according to claim 1, 2, or 3, characterized in that, The process of obtaining the predicted probability P of the vehicle's intention in the future time domain specifically includes: Fusion of encoded information through attention mechanism and social grid information : In the formula, Indicates the features after fusion. For attention mechanisms; The fused features are processed through a fully connected layer. Perform feature extraction: In the formula, Indicates the extracted features. It is the weight matrix of the fully connected layer. It is a bias vector. For activation functions; Predicting high-level intention categories of traffic participants : The weight matrix for high-level intent classification. It is the bias vector; Based on high-level intent categories The probability of longitudinal driving intention is obtained by refining the details. Probability of lateral driving intention : in, The weight matrix for low-level intent classification. This is the bias vector.

5. The vehicle multimodal trajectory prediction method considering spatiotemporal interaction information according to claim 1, characterized in that, The decoder has six branches, each corresponding to an intent; After receiving shared features, the six branches independently decode and generate the corresponding future trajectories. The structure of each branch is as follows: In the formula, They are respectively Branch at time step The hidden state and the memory state, Indicates the first A branched LSTM network for Branch at time step The predicted output coordinates; For the output mapping parameters; After recursively generating at time step T, the multimodal trajectory prediction results under six intentions are obtained through integration. : In the formula, These are respectively: left lane change intention, right lane change intention, left acceleration lane change intention, right acceleration lane change intention, straight driving at constant speed intention, and deceleration intention.

6. A vehicle multimodal trajectory prediction system, characterized in that, include: At least one processing module, And memory that stores computer program instructions; At least one processing module implements the vehicle multimodal trajectory prediction method considering spatiotemporal interaction information as described in any one of claims 1 to 5 when executing the computer program instructions.

7. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on an electronic device, causes the electronic device to perform the vehicle multimodal trajectory prediction method considering spatiotemporal interaction information as described in any one of claims 1 to 5.

Images