An interactive vehicle trajectory prediction and driving intention recognition method based on multi-task learning

Abstract

The application relates to an interactive vehicle trajectory prediction and driving intention recognition method based on multi-task learning. The method comprises the following steps: processing vehicle trajectory dataset data, determining target data corresponding to a target vehicle and surrounding traffic participants according to vehicle identifiers contained in the data; a multi-task learning method is invented, a CTG-MTLN model based on multi-task learning is established by using the processed data to realize the functions of interactive vehicle trajectory prediction and driving intention recognition, and the model has the characteristics of high precision and high efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation, in particular to a method for interactive vehicle trajectory prediction and driving intention recognition based on multi-task learning. BACKGROUND

[0002] With the rapid growth of the number of cars, complex traffic scenes are increasing, and accurate prediction of interactive vehicle trajectory and recognition of driving intention will help improve the decision quality of early warning and intervention, thereby effectively avoiding the occurrence of collision accidents. In addition, the method of interactive vehicle trajectory prediction and driving intention recognition can also be applied to the realization of human-like trajectory planning of vehicles in automatic driving technology. However, the complex behavior of vehicles, the limitations of structured roads, and the social interaction between agents will all pose challenges to vehicle trajectory prediction and driving intention recognition. At the same time, in complex traffic scenes, incomplete observation and object occlusion will cause high noise of sensors, which will affect the accuracy of interactive vehicle trajectory prediction and driving intention recognition. Since the recurrent neural network (RNN) and its variants can effectively capture the time correlation, they are widely used in interactive vehicle trajectory prediction and driving intention recognition. However, the dimensionality reduction process of this method will transmit the perception features from the encoder to the cell, which will not only cause the loss of key perception information, but also may even lead to suboptimal prediction. This problem is particularly prominent in interactive vehicle trajectory prediction and driving intention recognition in high-density traffic flow state. In addition, traditional interactive vehicle trajectory prediction and driving intention recognition usually adopts modular design, which needs to consume a large amount of computing resources to realize the functions of interactive vehicle trajectory prediction and driving intention recognition by respectively constructing interactive vehicle trajectory prediction module and driving intention recognition module. SUMMARY

[0003] Therefore, the present application aims to provide a method for interactive vehicle trajectory prediction and driving intention recognition based on multi-task learning, to solve the problems of inherent error accumulation of the autoregressive model used in interactive vehicle trajectory prediction and driving intention recognition, and the need for a large amount of computing resources to realize the functions of interactive vehicle trajectory prediction and driving intention recognition based on modular design.

[0004] To achieve the above-mentioned purpose, the present application adopts the following technical solution: a method for interactive vehicle trajectory prediction and driving intention recognition based on multi-task learning, comprising the following steps:

[0005] Step 1: processing interactive vehicle trajectory prediction and driving intention recognition dataset data;

[0006] Step 2: determining the definition of the trajectory prediction problem;

[0007] Step 3: Use the multi-task learning method to design the CTG-MTLN (Convolutional Transformer Gated-recurrent-unit Multi-task Transfer Learning Network) model for interactive vehicle trajectory prediction and driving intention recognition.

[0008] In a preferred embodiment, the step 1 specifically includes: first, loading data based on the target vehicle ID, and classifying trajectory data according to the lane change operation type; then, searching for surrounding traffic agent IDs and matching them with the corresponding trajectories; after processing, obtaining the position, speed, acceleration and vehicle type parameter information relative to the target vehicle; calculating the safety factor and lane environment information of the target vehicle;

[0009] In addition, after processing the trajectory data of all agents, n+m seconds of historical trajectory data are extracted, where n seconds are used for understanding and m seconds are used for prediction; model training is divided into two stages: the first stage focuses on trajectory prediction, and the entire data set is shuffled using a fixed random seed regardless of data type; the second stage focuses on driving intention recognition, and the fixed order of lane-keeping data in the interactive vehicle trajectory and driving intention data set is maintained; finally, the data of the two stages is divided into training set, validation set and test set.

[0010] In a preferred embodiment, the step 2 specifically includes: interactive vehicle trajectory prediction and driving intention recognition respectively estimate the future motion state of the interactive vehicle and the future operation intention of the target vehicle driver; interactive vehicle trajectory prediction and driving intention data contain attributes, safety risk indicators and road structure information describing each observed traffic agent;

[0011] The pre-processed observable feature matrix is selected as the input V = [v1, v2, … v t , …v T ]; at the same time, a predictor is designed to output the future interactive vehicle trajectory and driving intention where t = {1, 2, …, T} is the time stamp; l = {1, 2, …, L} is the time stamp of future prediction, L represents the output / prediction range;

[0012] The interactive vehicle trajectory prediction and driving intention recognition parameter matrix v t = (x target , x Other , x RI , x RR , x label} is composed of five parameter matrices, including the target vehicle feature set x target , the surrounding traffic participant feature set x Other, a risk indicator feature set x of the target vehicle RI , a spatial relationship feature set x of the vehicle and the road RR , and a driving intention label feature set x label ; x target and x Other include kinematic parameters such as vehicle type, position, speed and acceleration; x target also include additional kinematic parameters specific to the target vehicle, such as yaw rate, acceleration rate and heading angle; surrounding traffic participants use x Other = {x p , x rp , x lp , x ra , x la , x f , x rf , x lf} to represent, where x p represents the front traffic participant, x rp represents the front right traffic participant, x lp represents the front left traffic participant, x ra represents the right adjacent traffic participant, x la represents the left adjacent traffic participant, x f represents the rear traffic participant, x rf represents the rear right traffic participant, x lf represents the rear left traffic participant;

[0013] a target vehicle feature set x RI contains risk indicators related to front and rear vehicles, a road feature set x RR then includes the relative lane position, lane type, lane number, lane width of the target vehicle, and vehicle size information.

[0014] In a preferred embodiment, the attributes of the traffic subjects include 2D bounding box, 2D position, speed and acceleration; the safety risk indicators include time to collision TTC and headway THW; the road structure information includes lane number, current lane position.

[0015] In a preferred embodiment, the CTG-MTLN model has a four-layer architecture, including an embedding layer, an extended embedding layer, an encoder layer, and a decoder layer; the embedding layer maps the sequence data to a high-dimensional space so that the encoder layer can better extract features; the extended embedding layer extends the trajectory data within a time length L and maps it to a high-dimensional space to enhance the feature extraction capability of the model; the encoder layer is responsible for analyzing the interaction between vehicles and converting the extracted features into hidden states as a summary and understanding of the traffic agent trajectory; the decoder layer extracts these understandings from the sequence input and generates future trajectories in combination with driving intentions.

[0016] In a preferred embodiment, the embedding layer maps the observed trajectory sequence V = [v1, v2, …v t , …v T ] containing agent information to a high-dimensional space to generate an output sequence O em = {h1, h2, …h t , …h T} of hidden features, where each h t is a hidden feature vector of dimension d model ; the output sequence O em is calculated layer by layer from top to bottom:

[0017] O em = E token +E spat +E pos (1)

[0018] As shown in equation (1), the embedding layer is composed of token embedding E token , spatial embedding E spat , and position embedding E pos , which are described as follows:

[0019] E token = Conv1d(V Tra ) (2)

[0020] E spat = W spat2 ·(W spat1 ·X RR +b spat1 )+b spat2 (3)

[0021]

[0022] where W spat1 and W spat2 are trainable weights of linear layers;

[0023] Equations (2) and (3) show that the former uses a one-dimensional convolutional layer, while the latter uses two linear transformation layers; both map V Tra and X RR to high-dimensional representations for subsequent processing or feature extraction; Equation (4) describes the relative position encoding function of the model, where pos represents the position, d model represents the dimension of the hidden layer, and i represents the dimension; the position information of each dimension corresponds to a sine function.

[0024] In a preferred embodiment, the extended embedding layer extends the observed trajectory sequence V and the spatial feature representation X token in time before calculating the encoding vectors of E spat and E RR to generate the extended observed trajectory sequence V' and the extended spatial feature representation X' RR , whose time length is T+L, and then the characteristics of the extended embedding layer, which is different from the embedding layer, are as follows:

[0025] z t = σ(W z · [h t-1 , v t ]) (5)

[0026] r t = σ(W r · [h t-1 , v t ]) (6)

[0027]

[0028]

[0029] The above equations (5) to (8) describe the basic structure of the gated recurrent unit GRU, where h t-1 represents the hidden state at time step t-1, represents the candidate hidden state at time step t, z t is the update gate, which controls the update of the activation, r t is the reset gate, which determines whether to discard h t-1 when calculating the candidate activation; W z , W r , and are trainable weights of the functions σ(·) and tanh(·);

[0030] The following equations (9) to (11) describe how the bidirectional structure of BiGRU obtains a more comprehensive understanding of the context information on the sequence data and predicts the hidden layer output yt at time step T+1 based on the data of the previous time length T.

[0031]

[0032]

[0033]

[0034] where b y is the bias term; the hidden state h t or the arrow → or ← above the function GRU(·) represents the forward or backward computation process, respectively; the hidden layer output y t of BiGRU is mapped to v′ t+1 by a linear layer.

[0035]

[0036] An additional gelu activation layer is introduced; where, and are the weight and bias term of the linear layer, respectively; based on the composite structure described in equations (9) to (12), the observed trajectory sequence V and the spatial feature representation X RR are extended to the time length T+L in an autoregressive manner; the observed trajectory sequence V′ of the time length T+L and the extended spatial feature representation X′ RR are converted into a high-dimensional output sequence O′ em = {h′1, h′2, … h′ t , … h′ T+L}, whose expression is as follows:

[0037] O′ em = E′ token +E′ spat +E′ pos (13)

[0038] E token = Conv1d(V′) (14)

[0039] E′ spat = W′ spat2 ·(W′ spat1 ·X′ RR +b′ spat1 )+b′ spat2 (15)

[0040]

[0041] where W′ spat1 , W′ spat2 , b′ spat1 , b′ spat2are trainable weights and bias terms of the linear layer.

[0042] In a preferred embodiment, the encoder layer processes the hidden feature matrix O em = {h1, h2,... h t ,... h T}, the encoder output matrix O en = {01, 02,... 0 t ,... 0 T}, which encapsulates the potential features in the interaction between different agents and long-term dependencies in the observable input sequence; the output sequence O en is calculated layer by layer from top to bottom; the multi-head self-attention Multi-Head Attention module is described as follows:

[0043]

[0044] wherein the query matrix Q, the key matrix K and the value matrix V are all from the linear transformation of the input matrix; the constant h represents the number of attention heads in the block; the multi-head self-attention mechanism and the feed forward network are as follows:

[0045] H1 = lnorm (O em + Attention (O em , O em , O em )) (18)

[0046] H2 = lnorm (relu (H1W1) W'1 + H1) (19)

[0047] H3 = lnorm (H2 + Attention (H2, H2, H2)) (20)

[0048] H4 = lnorm (relu (H3W2) W'2 + H3) (21)

[0049] wherein W1, W'1, W2, W'2 are trainable weights of the lnorm(·) and relu(·) functions; each sub-layer contains a residual connection, followed by layer normalization; the residual connection serves as a direct skip connection; the layer normalization is used to normalize the output of all neurons within the layer; the relu activation function in formula (19) introduces nonlinearity; the following formula (22) describes the hidden features obtained by the multi-head attention mechanism through convolution, elu activation, max pooling and layer normalization to extract and enhance relevant features:

[0050] O en = lnorm (MaxPool1d (elu (Conv1d (H4)))) (22).

[0051] In a preferred embodiment, the decoder processes the extended output sequence V′={v′1,v′2,…,v′ t ,…,v′ T+L} and the encoder's output sequence O en ={O1, O2, ... O t , ...O T}, to generate future trajectory sequences and the driver's operating intentions The proposed decoder generates the complete output sequence by performing a single top-down computation; the decoder's input sequence V′={v′1,v′2,…,v′ t ,…,v′ T+L The sequence is generated in an autoregressive manner, as described in formulas (9) to (12), forming the initial predicted interactive vehicle trajectory sequence; the top-down calculation process is as follows:

[0052] H5=lnorm(O′ em +Attention(O′ em O′ em O′ em )) (twenty three)

[0053] H6=lnorm(relu(H5W3)W′3+H5) (24)

[0054] H7=lnorm(H6+Attention(H6,O en O en (25)

[0055] H8=lnorm(relu(H7W4)W′4+H7) (26)

[0056]

[0057] H9 = lnorm(H8)(28)

[0058] H 10 =W′6relu((W6dropout(H9)+b6)+H9)+b′6 (29)

[0059] H 11 =dropout(H 10 (30)

[0060]

[0061] wherein, W3, W'3, W4, W'4, W5, W6, W'6, W7, W'7 are trainable weights of the fully connected layer and the linear connected layer; b5, b6, b'6, b7, b'7 are trainable biases of the fully connected layer and the linear connected layer; dropout(·) is a regularization function;

[0062] The above formulas (23) and (24) describe the multi-head self-attention layer in the decoder, which is the same as formulas (18) and (19) in the encoder; formulas (25) and (26) describe the multi-head cross-attention layer, in which the key and value vectors come from the encoder output O en , while the query vector is a linear mapping of the hidden features further extracted from O' em by the multi-head self-attention mechanism; the cross-attention layer plays a key role in establishing the similarity features between the initial predicted interactive vehicle trajectory V' and the historical interactive vehicle trajectory V; formula (27) describes the process of network predicting the interactive vehicle trajectory , which is the result obtained by the output linear connected layer; formulas (28) and (29) describe how the cross-attention mechanism extracts and enhances the hidden features of the driving intention related features by dropout, relu activation, linear connected layer, residual connection and layer normalization; finally, formulas (30) and (31) describe how to output the hidden features by the feature classification layer to form the driving intention sequence; the last layer outputs the driving intention sequence by the linear connected layer

[0063] Compared with the prior art, the present application has the following beneficial effects: the interactive vehicle trajectory prediction and driving intention recognition method based on multi-task learning performs robust spatial feature extraction in a lane environment, adopts a transfer learning framework, and enables sharing of a model to extract long-term dependencies in time series data and interactive features between agents. This method enables the CTG-MTLN model to simultaneously complete the multi-task goals of interactive vehicle trajectory prediction and driving intention recognition, making it more accurate and efficient. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 A flowchart of an interactive vehicle trajectory prediction and driving intention recognition method based on multi-task learning provided by the preferred embodiment of the present application is shown.

[0065] Figure 2 A CTG-MTLN model structure diagram for realizing interactive vehicle trajectory prediction and driving intention recognition based on multi-task learning provided by the preferred embodiment of the present application is shown.

[0066] Figure 3A model embedding layer structure diagram in a CTG-MTLN model for realizing interactive vehicle trajectory prediction and driving intention recognition based on multi-task learning is provided for a preferred embodiment of the present application.

[0067] Figure 4 A model expansion embedding layer structure diagram in a CTG-MTLN model for realizing interactive vehicle trajectory prediction and driving intention recognition based on multi-task learning is provided for a preferred embodiment of the present application.

[0068] Figure 5 A model coding layer structure diagram in a CTG-MTLN model for realizing interactive vehicle trajectory prediction and driving intention recognition based on multi-task learning is provided for a preferred embodiment of the present application.

[0069] Figure 6 A model decoding layer structure diagram in a CTG-MTLN model for realizing interactive vehicle trajectory prediction and driving intention recognition based on multi-task learning is provided for a preferred embodiment of the present application. DETAILED DESCRIPTION

[0070] The present application will be further described below in conjunction with the accompanying drawings and embodiments.

[0071] It should be noted that the following detailed description is illustrative only, and is intended to provide further description in order to provide a further understanding of the present application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0072] It should be noted that the terms used herein are merely for the purpose of describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present application; as used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and furthermore, it should be understood that when the terms "comprise" and / or "include" are used in the specification, there is a presence of a feature, step, operation, device, component and / or combinations thereof.

[0073] For the purpose of more clearly illustrating one or more embodiments of the present specification, technical solutions and advantages thereof, the technical solutions of one or more embodiments of the present specification will be described in detail below in conjunction with specific embodiments and related drawings. Obviously, the described embodiments are only a part of the present specification, rather than all embodiments. Based on the embodiments in the present specification, other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of one or more embodiments of the present specification.

[0074] In conjunction with the accompanying drawings, the technical solutions provided by the embodiments of the present specification will be described in detail below.

[0075] In the prior art, due to the inherent error accumulation problem of the autoregressive model used in interactive vehicle trajectory prediction and driving intention recognition, and the problem of consuming a large amount of computing resources to realize the functions of interactive vehicle trajectory prediction and driving intention recognition based on modular design, the accuracy and computing efficiency of the prediction and recognition results are both low.

[0076] In view of the deficiencies in the prior art, the following embodiments are proposed:

[0077] A method for interactive vehicle trajectory prediction and driving intention recognition based on multi-task learning is provided, which combines the advantages of Figures 1-6 The specific description is as follows.

[0078] Figure 1 A flowchart of a method for interactive vehicle trajectory prediction and driving intention recognition based on multi-task learning is shown. From the software point of view, the execution subject of the flowchart can be a program or application client deployed on an application server. From the hardware point of view, the execution subject of the flowchart can be a cloud control platform, etc. The present embodiment does not particularly limit this.

[0079] As Figure 1 shown, the flowchart can include the following steps:

[0080] Step 1: Process interactive vehicle trajectory prediction and driving intention recognition dataset data.

[0081] First, load the data based on the target vehicle ID, and classify the trajectory data according to the lane change operation type. Then, search for the surrounding traffic agent ID and match it with the corresponding trajectory. After processing, various parameters such as position, speed, acceleration, and vehicle type relative to the target vehicle are obtained. Calculate the safety factors and lane environment information of the target vehicle according to the available data.

[0082] In addition, after processing the trajectory data of all agents, extract the historical trajectory data for n+m seconds (where n seconds are used for understanding and m seconds are used for prediction). Model training is divided into two stages. The first stage focuses on trajectory prediction, and the entire dataset is shuffled using a fixed random seed regardless of data type. The second stage focuses on driving intention recognition, and the lane keeping data in the interactive vehicle trajectory and driving intention dataset is kept in a fixed order. To solve the class imbalance problem, these data are sampled at a certain sampling rate, while the left and right lane change data remain unchanged. Finally, the data of the two stages are divided into training set, validation set and test set in a certain proportion.

[0083] Step 2: Determine the problem definition of trajectory prediction.

[0084] Interactive vehicle trajectory prediction and driving intention recognition refer to estimating the future motion state of interactive vehicles and the future operational intentions of the target vehicle's driver, respectively, based on their previously observed trajectories. Interactive vehicle trajectory prediction and driving intention data not only include various attributes describing each observed traffic subject (such as 2D bounding boxes, 2D positions, speeds, accelerations, etc.), but also safety risk indicators (such as time to collision (TTC), headway (THW), etc.) and road structure information (such as the number of lanes, current lane position, etc.).

[0085] To simultaneously address the interactive vehicle trajectory prediction task in regression and the driver intent recognition task in classification, a preprocessed observable feature matrix was chosen as the input V = [v1, v2, ... v2]. t , ...v T Simultaneously, a predictor was designed to output the future trajectory of the interactive vehicle. and driving intention Where t = {1, 2, ..., T} is the timestamp; l = {1, 2, ..., L} is the timestamp of the future prediction, and L represents the output / prediction range.

[0086] Interactive vehicle trajectory prediction and driving intention recognition parameter matrix v t ={x target x Other x RI x RR x label It consists of five parameter matrices, including the target vehicle feature set x. target Feature set of surrounding traffic participants x Other Risk indicator feature set x of the target vehicle RI The spatial relationship feature set of vehicles and roads x RR and driving intention label feature set x label x target and x Other Both include kinematic parameters such as vehicle type, position, speed, and acceleration. The difference lies in x. target It also includes additional kinematic parameters specific to the target vehicle, such as yaw rate, acceleration rate, and heading angle. The target vehicle is represented by x. target Indicated by x. Surrounding traffic participants are represented by x. Other ={x p x rp x lp x ra x la x f x rf x lf} represents, where x p Indicates the traffic participant ahead, xrp represents a front right side traffic participant, x lp represents a front left side traffic participant, x ra represents a right side adjacent traffic participant, x la represents a left side adjacent traffic participant, x f represents a rear traffic participant, x rf represents a rear right side traffic participant, x lf represents a rear left side traffic participant.

[0087] In addition, the target vehicle feature set x RI contains risk indicators related to front and rear vehicles, such as time to collision (TTC) and headway time (THW). The road feature set x RR includes the relative lane position, lane type, lane number, lane width, and vehicle size of the target vehicle.

[0088] Step 3: Design the CTG-MTLN model using a multi-task learning method.

[0089] As shown in Figure 2 , the designed CTG-MTLN model has a four-layer architecture, including an embedding layer, an extended embedding layer, an encoder layer, and a decoder layer. In this architecture, the embedding layer maps the sequence data to a high-dimensional space to facilitate better feature extraction by the encoder layer. The extended embedding layer extends the trajectory data within a time length of L and maps it to a high-dimensional space to enhance the model's feature extraction capability. The encoder layer is responsible for analyzing the interactions between vehicles and converting the extracted features into hidden states as a summary and understanding of the traffic agent trajectories. The decoder layer extracts these understandings from the sequence input and generates future trajectories in combination with driving intentions. The specific details of the embedding layer, extended embedding layer, encoder layer, and decoder layer are as follows:

[0090] Embedding layer: The embedding layer maps the observation trajectory sequence V = [v1, v2, … v t , … v T ] containing agent information to a high-dimensional space, generating an output sequence of hidden features O em = {h1, h2, … h t , … h T}, where each h t is a d model -dimensional hidden feature vector. As shown in Figure 3 , the output sequence O em is calculated layer by layer from top to bottom.

[0091] O em = E token +E spat +E pos #(1)

[0092] As shown in equation (1), the embedding layer is composed of token embedding E token , spatial embedding E spat , and position embedding E pos , which are described as follows:

[0093] E token = Conv1d(V Tra ) # (2)

[0094] E spat = W spat2 · (W spat1 · X RR + b spat1 ) + b spat2 # (3)

[0095]

[0096] where W spat1 and W spat2 are trainable weights of linear layers; the function Conv1d(·) will be introduced later.

[0097] Equations (2) and (3) respectively show that the former uses a one-dimensional convolution layer, while the latter uses two linear transformation layers. Both of them map V Tra and X RR to high-dimensional representations for subsequent processing or feature extraction. Equation (4) describes the relative position encoding function of the model, where pos represents the position, d model represents the dimension of the hidden layer, and i represents the dimension. The position information of each dimension corresponds to a sine function.

[0098] Extended embedding layer: Before calculating the encoding vectors of E token and E spat , the observed trajectory sequence V and the spatial feature representation X RR are extended in time to generate the extended observed trajectory sequence V' and the extended spatial feature representation X' RR , whose time length is T+L, as shown in Figure 4 Then, the characteristics of the extended embedding layer, which is different from the embedding layer, are introduced as follows:

[0099] z t = σ (W z · [h t-1 , v t ]) # (5)

[0100] r t = σ (W r · [h t-1 , vt ])#(6)

[0101]

[0102]

[0103] The above equations (5) to (8) describe the basic structure of a gated recurrent unit (GRU), where h t-1 denotes the hidden state at time step t-1, denotes the candidate hidden state at time step t, z t is the update gate, which controls the activation update, r t is the reset gate, which decides whether to discard h t-1 ; W z , W r , and are trainable weights for the functions σ(·) and tanh(·).

[0104] The following equations (9) to (11) describe how the bidirectional structure of BiGRU obtains a more comprehensive understanding of the context information on the sequence data and predicts the hidden layer output yt at time step T+1 based on the data of the previous time length T.

[0105]

[0106]

[0107]

[0108] where b y is the bias term; the hidden state h t or the arrow → or ← above the function GRU(·) represents the forward or backward computation process, respectively. The hidden layer output y t of BiGRU is mapped to v′ t+1 by a linear layer, as follows:

[0109]

[0110] To alleviate the impact of gradient vanishing, an additional gelu activation layer is introduced; where, and are the weight and bias term of the linear layer, respectively. Based on the composite structure described in equations (9) to (12), the observed trajectory sequence V and the spatial feature representation X RR are extended to a time length of T+L in an autoregressive manner. In order to further process or extract features, the observed trajectory sequence V′ of time length T+L and the extended spatial feature representation X′ RR are converted into a high-dimensional output sequence O′em ={h′1, h′2, ..., h′} t ,…h′ T+L}, its expression is as follows:

[0111] O′ em =E′ token +E′ spat +E′ pos #(13)

[0112] E token =Conv1d(V′)#(14)

[0113] E′ spat =W′ spat2 ·(W′ spat1 ·X′ RR +b′ spat1 )+b′ spat2 #(15)

[0114]

[0115] Among them, W′ spat1 , W′ spat2 , b′ spat1 , b′ spat2 These are the trainable weights and biases of the linear layer.

[0116] Encoder layer: By processing the hidden feature matrix O em ={h1, h2, ... h t , ...h T The encoder output matrix O en ={O1, O2, ... O t , ...O T This matrix encapsulates the interactions between different agents in the observable input sequence, as well as the latent features in long-term dependencies. (See reference...) Figure 5 Output sequence O en It is calculated layer by layer from top to bottom. The multi-head attention module can be described as:

[0117]

[0118] In this mechanism, the query matrix Q, key matrix K, and value matrix V are all derived from linear transformations of the input matrices; the constant h represents the number of attention heads in the block. The attention weights are calculated using scaling dot products and softmax normalization operations. Furthermore, it includes... This stabilizes the variance of the dot product, making it less sensitive to the number of attention heads, thereby improving model performance. The multi-head self-attention mechanism and feedforward network are shown below:

[0119] H1 = lnorm(O em +Attention(O em O em O em ))#(18)

[0120] H2=lnorm(relu(H1W1)W′1+H1)#(19)

[0121] H3=lnorm(H2+Attention(H2, H2, H2))#(20)

[0122] H4=lnorm(relu(H3W2)W′2+H3)#(21)

[0123] Where W1, W′1, W2, and W′2 are the trainable weights of the lnorm(·) and relu(·) functions. Each sublayer contains a residual connection, followed by layer normalization. The residual connection, as a direct skip connection, effectively solves the gradient vanishing problem and promotes model convergence. Layer normalization is used to normalize the output of all neurons within the layer, thereby improving the generalization performance of the model. The relu activation function in formula (19) introduces nonlinearity, which helps the network learn features related to complex interactions between traffic agents and long-term dependencies in trajectory sequences. The following formula (22) describes the hidden features obtained by the multi-head attention mechanism through convolution, elu activation, max pooling, and layer normalization to extract and enhance relevant features:

[0124] O en =lnorm(MaxPool1d(elu(Conv1d(H4))))#(22)

[0125] Convolutional operations are used to reduce the dimensionality of the data, helping the model better understand the hidden features in the observed trajectory sequence. Furthermore, ensembled max-pooling layers further reduce the spatial dimensionality of the input data, thereby capturing the most critical features. The presence of ELU activation and layer normalization further enhances the model's nonlinearity and generalization ability.

[0126] Decoder: The decoder processes the extended output sequence V′={v′1,v′2,…,v′ t ,…,v′ T+L} and the encoder's output sequence O en ={O1, O2, ... O t , ...O T}, to generate future trajectory sequences and the driver's operating intentions The proposed decoder can generate a complete output sequence by performing a single top-down computation, similar to the encoder. The decoder's input sequence is V′={v′1,v′2,…,v′ t ,…,v′ T+L The sequence is generated in an autoregressive manner, as described in formulas (9) to (12), forming the initial predicted interactive vehicle trajectory sequence. To fuse O′... em and O en The hidden features in the sequence of predicted interactive vehicle trajectories are widely used in the decoder layer through cross-attention mechanisms, achieving global optimization of the hidden features in the initial prediction sequence. For example... Figure 6 As shown, the calculation process from top to bottom is as follows:

[0127] H5=lnorm(O′ em +Attention(O′ em O′ em O′ em ))#(twenty three)

[0128] H6=lnorm(relu(H5W3)W′3+H5)#(24)

[0129] H7=lnorm(H6+Attention(H6,O en O en ))#(25)

[0130] H8=lnorm(relu(H7W4)W′4+H7)#(26)

[0131]

[0132] H9 = lnorm(H8)#(28)

[0133] H 10 =W′6relu((W6dropout(H9)+b6)+H9)+b′6#(29)

[0134] H 11 =dropout(H 10 )#(30)

[0135]

[0136] Where W3, W′3, W4, W′4, W5, W6, W′6, W7, W′7 are the trainable weights of the fully connected layer and the linear connected layer; b5, b6, b′6, b7, b′7 are the trainable biases of the fully connected layer and the linear connected layer; dropout(·) is the regularization function.

[0137] The above equations (23) and (24) describe the multi-head self-attention layer in the decoder, which is the same as equations (18) and (19) in the encoder. Equations (25) and (26) describe the multi-head cross-attention layer, where the key and value vectors come from the linear mapping of the encoder output O en , while the query vector is a linear mapping of the hidden features further extracted from O' em by the multi-head self-attention mechanism. The cross-attention layer plays a key role in establishing the similarity features between the initial predicted interacting vehicle trajectory V' and the historical interacting vehicle trajectory V. This approach helps to optimize the hidden layer features globally in the output sequence, thus enhancing the model's understanding of the trajectory and its long-term dependencies. Equation (27) describes the process of the network predicting the interacting vehicle trajectory , which is the result obtained by the output linear connection layer. Equations (28) and (29) describe how the cross-attention mechanism extracts and enhances the hidden features related to driving intent through dropout, relu activation, linear connection layer, residual connection, and layer normalization. Dropout can prevent the network from over-relying on any single neuron, promoting better generalization ability. Finally, equations (30) and (31) describe how to output the hidden features through the feature classification layer to form the driving intent sequence. These layers use dropout and contain residual connections and relu activation to extract and enhance features related to driving intent. This approach effectively enhances the classification ability and generalization performance of the network. The last layer outputs the driving intent sequence

Claims

1. A method for interactive vehicle trajectory prediction and driving intention recognition based on multi-task learning, characterized in that, The method comprises the following steps: Step 1: processing interactive vehicle trajectory prediction and driving intention recognition dataset data; Step 2: determining the problem definition of trajectory prediction; Step 3: designing a CTG-MTLN model for interactive vehicle trajectory prediction and driving intention recognition using a multi-task learning method; The step 1 specifically comprises: firstly, loading data based on the target vehicle ID, and classifying the trajectory data according to the lane change operation type; then, searching for surrounding traffic agent IDs and matching them with the corresponding trajectories; after processing, obtaining the position, speed, acceleration and vehicle type parameter information relative to the target vehicle; calculating the safety factor of the target vehicle and the lane environment information; In addition, after processing the trajectory data of all agents, the following is extracted seconds of historical trajectory data, wherein seconds are used for understanding, seconds are used for prediction; model training is divided into two stages: the first stage focuses on trajectory prediction, the entire dataset is shuffled, and a fixed random seed is used regardless of data type; the second stage focuses on driving intent recognition, keeping the fixed order of lane-keeping data in the interactive vehicle trajectory and driving intent dataset; finally, the data of the two stages is divided into training set, validation set and test set; The step 2 specifically comprises: interactive vehicle trajectory prediction and driving intention recognition respectively estimate the future motion state of the interactive vehicle and the future operation intention of the target vehicle driver; interactive vehicle trajectory prediction and driving intention data contain attribute information, safety risk indicators and road structure information describing each observed traffic agent; selecting a pre-processed observable feature matrix as input ; simultaneously, designing a predictor to output future interacting vehicle trajectories and driving intents wherein is a timestamp; is a future predicted timestamp, represents an output / prediction horizon; Interaction vehicle trajectory prediction and driving intent recognition parameter matrix Composed of five parameter matrices, including target vehicle feature set , surrounding traffic participant feature set , risk indicator feature set of target vehicle , spatial relationship feature set of vehicle and road , and driving intent label feature set ; and Both include vehicle type, position, speed and acceleration kinematic parameters; Also includes additional kinematic parameters specific to the target vehicle, including yaw rate, acceleration rate and heading angle; surrounding traffic participants are represented by , where represents the front traffic participant, represents the right front traffic participant, represents the left front traffic participant, represents the right adjacent traffic participant, represents the left adjacent traffic participant, represents the rear traffic participant, represents the right rear traffic participant, represents the left rear traffic participant; Target vehicle feature set including risk indicators related to vehicles in front and behind, road feature set including target vehicle's relative lane position, lane type, number of lanes, lane width, and vehicle size information; The CTG-MTLN model has a four-layer architecture, including an embedding layer, an extended embedding layer, an encoder layer, and a decoder layer; the embedding layer maps the sequence data to a high-dimensional space so that the encoder layer can better perform feature extraction; the extended embedding layer extends the trajectory data in the range of and maps it to a high-dimensional space to enhance the feature extraction capability of the model; the encoder layer is responsible for analyzing the interaction between vehicles and converting the extracted features into hidden states as a summary and understanding of the traffic agent trajectories; the decoder layer extracts these understandings from the sequence input and generates future trajectories in combination with driving intentions.

2. The method of claim 1, wherein, The attribute information of the traffic agent includes a 2D bounding box, a 2D position, a speed and an acceleration; the safety risk indicators include a time to collision TTC and a headway THW; and the road structure information includes the number of lanes and the current lane position.

3. The method of claim 1, wherein, The embedding layer will contain a sequence of observation trajectories of agent information mapped to a high-dimensional space, generating a sequence of output of hidden features where each is a hidden feature vector of dimension Output sequence Computed layer by layer from top to bottom: As shown in formula (1), the embedding layer is composed of a token embedding , a spatial embedding , and a position embedding , which are described as follows: wherein, and are trainable weights of the linear layer; Equations (2) and (3) show that the former uses a one-dimensional convolutional layer, while the latter uses two linear transformation layers; both of them map and to high-dimensional representations for subsequent processing or feature extraction; Equation (4) describes the relative position encoding function of the model, where represents the position, represents the dimension of the hidden layer, represents the dimension; the position information of each dimension corresponds to a sine function.

4. The method of claim 1, wherein, The extended embedding layer in computation and Before the encoding vector, the observed trajectory sequence V and spatial feature representation are... Extend the time horizon to generate an extended sequence of observation trajectories. and extended spatial feature representation Their time length is Then, the characteristics of the extended embedding layer, which differs from the embedding layer, are as follows: The above equations (5) to (8) describe the basic structure of a gated recurrent unit GRU, wherein denotes the hidden state at time step , denotes the candidate hidden state at time step , is the update gate, which controls the activation update, is the reset gate, which decides whether to discard ; , , and are trainable weights of the functions and ; The following equations (9) to (11) describe how the bidirectional structure of BiGRU obtains a more comprehensive understanding of the context information on the sequence data and predicts the hidden layer output at time step based on the data of the previous time length . ; where, is the bias term; hidden state or function the arrow above or denote the forward or backward computation process, respectively; hidden layer output of BiGRU mapped through a linear layer to as follows: Introduce an additional Activation layer; where, and These are the weights and bias terms of the linear layer, respectively; based on the composite structure described in formulas (9) to (12), the observed trajectory sequence and spatial feature representation Extended to a time length of 1000 by autoregression The duration is Observation trajectory sequence and extended spatial feature representation Convert to high-dimensional output sequence Its expression is as follows: wherein, , , , are trainable weights and bias terms of the linear layer.

5. The method of claim 1, wherein, The encoder layer encodes the hidden feature matrix , the encoder output matrix , which encapsulates the latent features in the interaction between different agents in the observable input sequence and long-term dependencies; Output sequence is computed layer by layer from top to bottom; the Multi-Head Attention module is described as: wherein the query matrix Q, the key matrix K, and the value matrix V are all from linear transformations of the input matrix; the constant denotes the number of attention heads in the block; the multi-head self-attention mechanism and the feed forward network are as follows: wherein, , , , is and trainable weights of the function; each sub-layer contains a residual connection followed by layer normalization; the residual connection serves as a direct skip connection; the layer normalization is used to normalize the output of all neurons within a layer; the relu activation function in equation (19) introduces nonlinearity; the following equation (22) describes the hidden features obtained by the multi-head attention mechanism extracting and enhancing relevant features through convolution, elu activation, max-pooling, and layer normalization: 。 6. The method of claim 1, wherein, The decoder processes the extended output sequence of embedded layers and the output sequence of the encoder to generate the future trajectory sequence and the operation intention of the driver The proposed decoder generates the complete output sequence by performing one top-down computation; the input sequence of the decoder is generated in an autoregressive manner, as described by equations (9) to (12), forming an initial predicted interaction vehicle trajectory sequence; the top-down computation process is as follows: wherein, , , , , , , , , are trainable weights of the fully connected and linearly connected layers; , , , , are trainable biases of the fully connected and linearly connected layers; is a regularization function; The above equations (23) and (24) describe the multi-head self-attention layer in the decoder, which is the same as equations (18) and (19) in the encoder; equations (25) and (26) describe the multi-head cross-attention layer, in which the key and value vectors are linear mappings of the encoder output , while the query vector is a linear mapping of the hidden features further extracted from by the multi-head self-attention mechanism; the cross-attention layer plays a key role in establishing the similarity features between the initial predicted interacting vehicle trajectory and the historical interacting vehicle trajectory ; equation (27) describes the process of the network predicting the interacting vehicle trajectory , which is the result obtained by the output linear connection layer; equations (28) and (29) describe how the cross-attention mechanism extracts and enhances the hidden features related to driving intent by dropout, relu activation, linear connection layer, residual connection and layer normalization; finally, equations (30) and (31) describe how to output the hidden features by the feature classification layer to form the driving intent sequence; the last layer outputs the driving intent sequence by the linear connection layer.

Images