An interactively enhanced trajectory prediction method and system for L3 high-speed autonomous driving

Abstract

The present invention relates to a method and system for interactively enhanced trajectory prediction for Level 3 high-speed autonomous driving. The method comprises: obtaining predicted historical trajectories of a host vehicle and surrounding vehicles; constructing a trajectory prediction model comprising an input module, a target encoding module, an interactive encoding module, and a decoding module. The input module encodes the trajectories and adds position information to obtain trajectory features; the target encoding module processes the predicted trajectory features of the host vehicle to obtain target encoding features; the interactive encoding module processes the predicted trajectory features of the host vehicle and surrounding vehicles to obtain interactive encoding features; and the decoding module decodes the predicted trajectory features based on the target encoding features and the interactive encoding features to obtain predicted future trajectories. The trajectory prediction model is then trained and used for interactively enhanced trajectory prediction in Level 3 high-speed autonomous driving scenarios. Compared with existing technologies, the present invention has the advantages of being applicable to all high-speed scenarios, high computational efficiency, and the ability to predict even when target objects are missing.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an interactively enhanced trajectory prediction method and system for L3 high-speed autonomous driving. Background Art

[0002] The Society of Automotive Engineers (SAE) categorizes autonomous driving into five levels, with L2 representing partial autonomy and L3 representing conditional autonomy. In recent years, an increasing number of vehicles have achieved L2 autonomy, capable of performing functions such as autonomous parking and adaptive cruise control. To overcome the bottlenecks of L3 autonomous driving, many companies are developing safe and robust autonomous driving algorithms, focusing on highway scenarios. This requires accurate prediction of surrounding traffic.

[0003] However, accurate trajectory prediction in high-speed scenarios presents several challenges. First, the trajectory of the vehicle being predicted is significantly affected by surrounding traffic, which must be considered during trajectory prediction. Second, providing high-precision maps for large, high-speed scenarios is prohibitively expensive, necessitating the development of trajectory prediction algorithms that don't rely on them. Finally, despite continuous advancements in hardware and software algorithms, some frames of perception output still lack objects. Summary of the Invention

[0004] The purpose of the present invention is to provide an interactively enhanced trajectory prediction method and system for L3 high-speed autonomous driving, which can well capture the interactive information between surrounding traffic vehicles and the predicted vehicle, thereby improving the accuracy of trajectory prediction; at the same time, the prediction model is designed based on the attention mechanism, and parallel computing is adopted to improve the prediction speed; and missing data can be adaptively processed to improve the robustness of trajectory prediction.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] An interactively enhanced trajectory prediction method for L3 high-speed autonomous driving, comprising the following steps:

[0007] In L3 high-speed autonomous driving scenarios, obtain the historical trajectories of the main vehicle and surrounding vehicles;

[0008] Constructing a trajectory prediction model, which includes an input module, a target encoding module, an interaction encoding module, and a decoding module. The input module encodes the trajectory and adds position information to obtain trajectory features. The target encoding module processes the predicted trajectory features of the main vehicle based on a multi-head attention mechanism to obtain target encoding features. The interaction encoding module processes the predicted trajectory features of the main vehicle and surrounding vehicles based on a multi-head attention mechanism to obtain interaction encoding features. The decoding module decodes the target encoding features and the interaction encoding features to obtain a predicted future trajectory.

[0009] The trajectory prediction model is trained using historical trajectories, and interactive enhanced trajectory prediction is performed in L3 high-speed autonomous driving scenarios based on the trained trajectory prediction model.

[0010] The input module uses a linear layer network to encode the trajectory and uses a position encoding layer to add position information to the temporal trajectory, which is expressed as:

[0011]

[0012]

[0013]

[0014]

[0015] Among them, AV represents the main vehicle to be predicted, SV represents the surrounding vehicles, and W linear represents the weight matrix of the linear layer network, Represents the historical trajectory of length obs, that is Where (x t ,y t ) represents the coordinates of the vehicle at time t, D is The length of the vector, d∈{1,2,…,D}.

[0016] The target encoding module feeds the predicted trajectory features of the main vehicle into the multi-head attention mechanism, and uses the residual module and batch normalization module to reduce its error, and then feeds it into the feedforward network and normalization layer to obtain the target encoding features. The multi-head attention mechanism is expressed as:

[0017] MultiHeadQ,K,V)=Concat(head1,…,head h )W O

[0018] Where head = Attention(QW i Q ,KW i K,VW i V )

[0019] Where Q, K, and V represent the query matrix, keyword matrix, and value matrix, respectively. In the target encoding module, Q, K, and V are all outputs of the input module.

[0020] In the interactive encoding module, the predicted trajectory features of the main vehicle are sent to the batch normalization module, and the trajectory features of the surrounding vehicles are connected and sent to the linear layer. The output features are input into the multi-head attention mechanism, where the predicted trajectory features of the main vehicle serve as the query matrix Q of the attention mechanism, and the trajectory features of the surrounding vehicles serve as the keyword matrix and value matrices K and V of the attention mechanism. The trajectory features of the surrounding vehicles are summed with the results of the multi-head attention mechanism using a residual network, and then sent to the feedforward network, normalization layer and a target encoding module to obtain interactive encoding features.

[0021] The decoding module directly uses the predicted future trajectory features of the main vehicle during training. When applied, the all-zero vector of the same dimension as the predicted future trajectory of the main vehicle is fed into the multi-head attention mechanism to obtain the future trajectory encoding features. The future trajectory encoding features are used as the query matrix Q of the next multi-head attention mechanism. The weighted sum of the features from the target encoding module and the interaction encoding module is used as the keyword matrix and value matrix K and V of the attention mechanism, where the weighted sum is expressed as em=O AE +λ×O SE , where O AE represents the features from the target encoding module, O SE Represents the features from the interactive encoding module, and λ represents the weighting coefficient; then a feedforward network and layer normalization are used, and then a linear layer is passed to obtain the predicted future trajectory of the main vehicle.

[0022] An interactively enhanced trajectory prediction system for L3 high-speed autonomous driving, comprising:

[0023] Trajectory acquisition unit, used to obtain the historical trajectory of the host vehicle and surrounding vehicles in L3 high-speed autonomous driving scenarios;

[0024] A trajectory prediction model construction unit, wherein the trajectory prediction model includes an input module, a target encoding module, an interaction encoding module, and a decoding module. The input module encodes the trajectory and adds position information to obtain trajectory features. The target encoding module processes the predicted trajectory features of the main vehicle based on a multi-head attention mechanism to obtain target encoding features. The interaction encoding module processes the predicted trajectory features of the main vehicle and surrounding vehicles based on a multi-head attention mechanism to obtain interaction encoding features. The decoding module decodes the target encoding features and the interaction encoding features to obtain a predicted future trajectory.

[0025] A trajectory prediction model training unit, used to train the trajectory prediction model using historical trajectories;

[0026] The trajectory prediction unit is used to perform interactive enhanced trajectory prediction in L3 high-speed autonomous driving scenarios based on the trained trajectory prediction model.

[0027] The input module uses a linear layer network to encode the trajectory and uses a position encoding layer to add position information to the temporal trajectory, which is expressed as:

[0028]

[0029]

[0030]

[0031]

[0032] Among them, AV represents the main vehicle to be predicted, SV represents the surrounding vehicles, and W linear represents the weight matrix of the linear layer network, Represents the historical trajectory of length obs, that is Where (x t ,y t ) represents the coordinates of the vehicle at time t, D is The length of the vector, d∈{1,2,…,D}.

[0033] The target encoding module feeds the predicted trajectory features of the main vehicle into the multi-head attention mechanism, and uses the residual module and batch normalization module to reduce its error, and then feeds it into the feedforward network and normalization layer to obtain the target encoding features. The multi-head attention mechanism is expressed as:

[0034] MultiHead(Q,K,V)=Concat(head1,…,head h )W O

[0035] Where head = Attention(QW i Q ,KW i K ,VW i V )

[0036] Where Q, K, and V represent the query matrix, keyword matrix, and value matrix, respectively. In the target encoding module, Q, K, and V are all outputs of the input module.

[0037] In the interactive encoding module, the predicted trajectory features of the main vehicle are sent to the batch normalization module, and the trajectory features of the surrounding vehicles are connected and sent to the linear layer. The output features are input into the multi-head attention mechanism, where the predicted trajectory features of the main vehicle serve as the query matrix Q of the attention mechanism, and the trajectory features of the surrounding vehicles serve as the keyword matrix and value matrices K and V of the attention mechanism. The trajectory features of the surrounding vehicles are summed with the results of the multi-head attention mechanism using a residual network, and then sent to the feedforward network, normalization layer and a target encoding module to obtain interactive encoding features.

[0038] The decoding module directly uses the predicted future trajectory features of the main vehicle during training. When applied, the all-zero vector of the same dimension as the predicted future trajectory of the main vehicle is fed into the multi-head attention mechanism to obtain the future trajectory encoding features. The future trajectory encoding features are used as the query matrix Q of the next multi-head attention mechanism. The weighted sum of the features from the target encoding module and the interaction encoding module is used as the keyword matrix and value matrix K and V of the attention mechanism, where the weighted sum is expressed as em=O AE +λ×O SE , where O AE represents the features from the target encoding module, O SE Represents the features from the interactive encoding module, and λ represents the weighting coefficient; then a feedforward network and layer normalization are used, and then a linear layer is passed to obtain the predicted future trajectory of the main vehicle.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) The present invention uses vehicle coordinates as input and does not use high-precision maps, and can be applied to all high-speed scenarios.

[0041] (2) All modules of the present invention are based on the attention mechanism and use parallel computing to perform calculations, which has high computational efficiency and is suitable for multi-batch long-term predictions.

[0042] (3) The present invention adds a position encoding layer to the input module, so that when the autonomous driving perception algorithm has a missing target object, the missing position can be skipped without affecting the operation of the overall prediction algorithm, thereby improving the robustness of autonomous driving.

[0043] (4) The present invention takes into account the interactive effects of surrounding vehicles on the trajectory, thereby improving the trajectory prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of the method of the present invention.

[0045] Figure 2 It is a structural diagram of the target coding module of the present invention.

[0046] Figure 3 It is a structural diagram of the interactive coding module of the present invention.

[0047] Figure 4 A schematic diagram of the structure of the decoding module of the present invention DETAILED DESCRIPTION

[0048] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0049] Example 1

[0050] This embodiment provides an interactive enhanced trajectory prediction method for L3 high-speed autonomous driving, such as Figure 1 As shown, the following steps are included:

[0051] S1: In an L3 high-speed autonomous driving scenario, obtain the historical trajectories of the host vehicle and surrounding vehicles.

[0052] S2: Building a trajectory prediction model

[0053] Because high-precision maps are unavailable in high-speed scenarios, this paper constructs a trajectory prediction model based on Transformer. This model uses the predicted historical trajectories of the host vehicle and surrounding vehicles as input and outputs the predicted future trajectory of the host vehicle. In this embodiment, the trajectory prediction model includes an input module, a target encoding module, an interactive encoding module, and a decoding module.

[0054] (A) Input module

[0055] like Figure 1 As shown, AV represents the main vehicle to be predicted, and SV represents the surrounding vehicles. Represents the historical trajectory of length obs, that is Where (x t ,y t ) represents the coordinates of the vehicle at time t, and Represent the predicted future trajectory and the true future trajectory respectively, where Similarly To input For example, the input module first uses a linear layer network to encode the input, and then inputs the position encoding layer to add position information to the time series trajectory. The input module can be expressed as the following formula:

[0056]

[0057]

[0058]

[0059]

[0060] Where D is The length of the vector, d∈{1,2,…,D}.

[0061] (B) Target encoding module

[0062] like Figure 2 As shown in the figure, the target encoding module feeds the predicted trajectory features of the main vehicle into the multi-head attention mechanism, and uses the residual module and batch normalization module to reduce its error, and then feeds it into the feedforward network and normalization layer to obtain the target encoding features. The multi-head attention mechanism is expressed as:

[0063] MultiHeadQ,K,V)=Concat(head1,…,head h )W O

[0064] Where head = Attention(QW i Q ,KW i K ,VW i V )

[0065] Here, Q, K, and V represent the query matrix, keyword matrix, and value matrix, respectively. In the target encoding module, Q, K, and V are the outputs of the input module. In this example, h = 8. The features are then input into a linear network and combined with a residual module and batch normalization module to obtain the target encoding features. To reduce the model weight, the target encoding module has 1 layer.

[0066] (C) Interaction Encoding Module

[0067] like Figure 3 As shown in the figure, the interaction encoding module uses the predicted historical trajectory of the main vehicle and the historical trajectory of surrounding vehicles as input to extract interaction features between the main vehicle and surrounding vehicles. The predicted trajectory features of the main vehicle are fed into the batch normalization module, and the trajectory features of the surrounding vehicles are concatenated and fed into the linear layer. The output features are input into the multi-head attention mechanism, where the predicted trajectory features of the main vehicle serve as the query matrix Q of the attention mechanism, and the trajectory features of the surrounding vehicles serve as the keyword matrix K and value matrices V of the attention mechanism. A residual network is used to sum the trajectory features of the surrounding vehicles with the results of the multi-head attention mechanism. The sum is then fed into a feedforward network, a normalization layer, and a target encoding module to obtain the interaction encoding features.

[0068] (D) Decoding module

[0069] like Figure 4 As shown in the figure, during training, the decoding module takes the predicted future trajectory of the main vehicle as input, obtains the future trajectory features of the main vehicle through the input module, and then sends them to the encoding layer. Specifically, after extracting features through the multi-head attention mechanism, residual module, and batch normalization module, the predicted future trajectory features of the main vehicle are directly used during training. During application, the all-zero vector with the same dimension as the predicted future trajectory of the main vehicle is sent to the multi-head attention mechanism to obtain the future trajectory encoding features. The future trajectory encoding features are used as the query matrix Q of the next multi-head attention mechanism. The features from the target encoding module and the interaction encoding module are weighted and summed as the keyword matrix and value matrix K and V of the attention mechanism. This weighted sum can be expressed as em=O AE +λ×O SE , where O AE represents the features from the target encoding module, O SE represents the features from the interactive encoding module, and λ represents the weighting coefficient. In this example, λ = 0.1. A feedforward network and a residual network are then used to sum the features of the main vehicle's future trajectory with the results of the multi-head attention mechanism, which are then fed into the batch normalization module. Finally, a linear layer is applied to obtain the predicted future trajectory of the main vehicle. During prediction, the decoding module fills the future trajectory of the main vehicle with zeros, and all other parameters remain the same as during training. To reduce model weight, the decoding module has one layer.

[0070] S3: Use historical trajectories to train the trajectory prediction model, and then perform interactive enhanced trajectory prediction in L3 high-speed autonomous driving scenarios based on the trained trajectory prediction model.

[0071] This embodiment discloses a Transformer-based interactively enhanced trajectory prediction method suitable for L3 high-speed autonomous driving. It innovatively applies the Transformer architecture to the field of trajectory prediction, enabling accurate and efficient trajectory prediction. In particular, the three main modules of this method are all based on a multi-head attention mechanism, where the target encoding module is used to process the historical trajectory of the main vehicle, the interactive encoding module is used to extract interactive information about surrounding vehicles, and the decoding module is used to predict future trajectories. This method is designed specifically for L3 high-speed autonomous driving scenarios and can improve the safety and robustness of L3 autonomous driving.

[0072] Example 2

[0073] This embodiment provides an interactively enhanced trajectory prediction system for L3 high-speed autonomous driving, configured to implement the method described in Example 1, including:

[0074] (1) Trajectory acquisition unit, used to obtain the historical trajectories of the main vehicle and surrounding vehicles in the L3 high-speed autonomous driving scenario.

[0075] (2) A trajectory prediction model construction unit, wherein the trajectory prediction model includes an input module, a target encoding module, an interaction encoding module, and a decoding module, wherein the input module encodes the trajectory and adds position information to obtain trajectory features, the target encoding module processes the trajectory features of the predicted main vehicle based on a multi-head attention mechanism to obtain target encoding features, the interaction encoding module processes the trajectory features of the predicted main vehicle and surrounding vehicles based on a multi-head attention mechanism to obtain interaction encoding features, and the decoding module decodes based on the target encoding features and the interaction encoding features to obtain a predicted future trajectory.

[0076] (3) Trajectory prediction model training unit, used to train the trajectory prediction model using historical trajectories.

[0077] (4) A trajectory prediction unit, which is used to perform interactive enhanced trajectory prediction in L3 high-speed autonomous driving scenarios based on the trained trajectory prediction model.

[0078] The specific implementation process of the system is shown in Example 1 and will not be repeated here.

[0079] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. An interactively enhanced trajectory prediction method for L3 high-speed autonomous driving, characterized by: The following steps are involved: In L3 high-speed autonomous driving scenarios, obtain the historical trajectories of the main vehicle and surrounding vehicles; A trajectory prediction model is constructed, which includes an input module, a target encoding module, an interaction encoding module and a decoding module. The input module encodes the trajectory and adds position information to obtain trajectory features. The target encoding module processes the trajectory features of the predicted main vehicle based on the multi-head attention mechanism to obtain target encoding features. The interaction encoding module processes the trajectory features of the predicted main vehicle and surrounding vehicles based on the multi-head attention mechanism to obtain interaction encoding features. The decoding module decodes based on the target encoding features and the interaction encoding features to obtain the predicted future trajectory. The target encoding module feeds the predicted trajectory features of the main vehicle into the multi-head attention mechanism and uses the residual The difference module and batch normalization module reduce the error, and then send it to the feedforward network and normalization layer to obtain the target coding feature; in the interactive coding module, the predicted trajectory features of the main vehicle are sent to the batch normalization module, and the trajectory features of the surrounding vehicles are connected and sent to the linear layer. The output features are input into the multi-head attention mechanism, where the predicted trajectory features of the main vehicle are used as the query matrix Q of the attention mechanism, and the trajectory features of the surrounding vehicles are used as the keyword matrix and value matrices K and V of the attention mechanism. The residual network is used to add the trajectory features of the surrounding vehicles and the results of the multi-head attention mechanism, and then sent to the feedforward network, normalization layer and a target coding module to obtain the interactive coding feature; The trajectory prediction model is trained using historical trajectories, and interactive enhanced trajectory prediction is performed in L3 high-speed autonomous driving scenarios based on the trained trajectory prediction model.

2. The interactive enhanced trajectory prediction method for L3 high-speed autonomous driving according to claim 1, characterized in that: The input module uses a linear layer network to encode the trajectory and uses a position encoding layer to add position information to the temporal trajectory, which is expressed as: Among them, AV represents the main vehicle to be predicted, SV represents the surrounding vehicles, and W linear represents the weight matrix of the linear layer network, Represents the historical trajectory of length obs, that is Where (x t ,y t ) represents the coordinates of the vehicle at time t, D is The length of the vector, d∈{1,2,…,D}.

3. The interactive enhanced trajectory prediction method for L3 high-speed autonomous driving according to claim 1, characterized in that: The decoding module directly uses the predicted future trajectory features of the main vehicle during training. When applied, the all-zero vector of the same dimension as the predicted future trajectory of the main vehicle is fed into the multi-head attention mechanism to obtain the future trajectory encoding features. The future trajectory encoding features are used as the query matrix Q of the next multi-head attention mechanism. The weighted sum of the features from the target encoding module and the interaction encoding module is used as the keyword matrix and value matrix K and V of the attention mechanism, where the weighted sum is expressed as em=O AE +λ×O SE , where O AE represents the features from the target encoding module, O SE Represents the features from the interactive encoding module, and λ represents the weighting coefficient; then a feedforward network and layer normalization are used, and then a linear layer is passed to obtain the predicted future trajectory of the main vehicle.

4. An interactively enhanced trajectory prediction system for L3 high-speed autonomous driving, characterized by: include: Trajectory acquisition unit, used to obtain the historical trajectory of the host vehicle and surrounding vehicles in L3 high-speed autonomous driving scenarios; The trajectory prediction model construction unit includes an input module, a target encoding module, an interaction encoding module and a decoding module, wherein the input module encodes the trajectory and adds position information to obtain trajectory features, the target encoding module processes the trajectory features of the predicted main vehicle based on the multi-head attention mechanism to obtain target encoding features, the interaction encoding module processes the trajectory features of the predicted main vehicle and surrounding vehicles based on the multi-head attention mechanism to obtain interaction encoding features, and the decoding module decodes based on the target encoding features and the interaction encoding features to obtain the predicted future trajectory; the target encoding module feeds the predicted trajectory features of the main vehicle into the multi-head attention mechanism and uses The residual module and batch normalization module reduce the error, and then send it to the feedforward network and normalization layer to obtain the target encoding feature. In the interactive encoding module, the predicted trajectory features of the main vehicle are sent to the batch normalization module, and the trajectory features of the surrounding vehicles are connected and sent to the linear layer. The output features are input into the multi-head attention mechanism, where the predicted trajectory features of the main vehicle serve as the query matrix Q of the attention mechanism, and the trajectory features of the surrounding vehicles serve as the keyword matrix and value matrices K and V of the attention mechanism. The residual network is used to add the trajectory features of the surrounding vehicles and the results of the multi-head attention mechanism, and then sent to the feedforward network, normalization layer and a target encoding module to obtain the interactive encoding feature. A trajectory prediction model training unit, used to train the trajectory prediction model using historical trajectories; The trajectory prediction unit is used to perform interactive enhanced trajectory prediction in L3 high-speed autonomous driving scenarios based on the trained trajectory prediction model.

5. The interactive enhanced trajectory prediction system for L3 high-speed autonomous driving according to claim 4, characterized in that: The input module uses a linear layer network to encode the trajectory and uses a position encoding layer to add position information to the temporal trajectory, which is expressed as: Among them, AV represents the main vehicle to be predicted, SV represents the surrounding vehicles, and W linear represents the weight matrix of the linear layer network, Represents the historical trajectory of length obs, that is Where (x t ,y t ) represents the coordinates of the vehicle at time t, D is The length of the vector, d∈{1,2,…,D}.

6. The interactive enhanced trajectory prediction system for L3 high-speed autonomous driving according to claim 4, characterized in that: The decoding module directly uses the predicted future trajectory features of the main vehicle during training. When applied, the all-zero vector of the same dimension as the predicted future trajectory of the main vehicle is fed into the multi-head attention mechanism to obtain the future trajectory encoding features. The future trajectory encoding features are used as the query matrix Q of the next multi-head attention mechanism. The weighted sum of the features from the target encoding module and the interaction encoding module is used as the keyword matrix and value matrix K and V of the attention mechanism, where the weighted sum is expressed as em=O AE +λ×O SE , where O AE represents the features from the target encoding module, O SE Represents the features from the interactive encoding module, and λ represents the weighting coefficient; then a feedforward network and layer normalization are used, and then a linear layer is passed to obtain the predicted future trajectory of the main vehicle.

Images