Transform-based automatic driving long time sequence track prediction method and system

Abstract

The invention discloses an automatic driving long time sequence track prediction method and system based on Transform, and relates to the technical field of automatic driving. The method comprises the following steps: performing automatic driving long-time-sequence trajectory prediction according to a historical trajectory and map information by using a Transform structure; specifically, position codes are added to historical track points, feature extraction is carried out on map information to obtain map embedded features, and fusion analysis is carried out on the historical track points, the position codes and the map embedded features from the angle of space-time intersection; feature extraction is carried out from the perspective of space-time interaction by using fusion information, and interaction relationships among the traffic participants are established step by step; and performing time sequence joint prediction on the implicit behavior intention features of the traffic participants in a recursive manner based on a multi-head attention mechanism to obtain a predicted trajectory. Through automatic driving scene high-precision modeling based on space-time interaction, traffic scene features are accurately expressed, the calculation amount is remarkably reduced, and the long-time-sequence track prediction precision of a complex scene is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a Transformer-based autonomous driving long-time series trajectory prediction method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Predicting the trajectories of traffic participants is a fundamental and crucial task in autonomous driving technology, and its accuracy significantly impacts the safety of autonomous vehicles. To address this issue, researchers have focused on trajectory prediction methods based on graph neural networks (GNNs). These methods rely on an explicit graph structure, where nodes represent individual traffic participants (agents) and edges represent their interactions (e.g., distance, relative position, potential interactions between vehicles, etc.).

[0004] To better handle the relational interactions in traffic processes, some current graph neural network-based trajectory prediction methods use the following encoder processing flow: first, they process the interactions between all traffic participants (e.g., vehicle-to-vehicle interactions) at each historical moment, and integrate the feature results of all historical moments together (historical moment dimension) to further capture cross-temporal dependencies; second, they process the interactions between all traffic participants and the map (e.g., vehicle-to-map interactions).

[0005] It can be seen that the above methods independently handle interactions with other traffic participants, their own historical trajectories, and their interactions with the map. However, in real life, the future trajectory of a single vehicle is influenced by its historical trajectory, the map, and other traffic participants. The above methods cannot accurately represent the interactions between traffic participants. Furthermore, some graph convolution methods require multiple graph convolution operations to ensure effective models, making them unsuitable for complex traffic scenarios such as congested intersections.

[0006] To sum up, how to reasonably express autonomous driving scenarios and efficiently explore the interactive relationships between traffic participants is a technical problem that needs to be urgently solved by existing technologies. Summary of the Invention

[0007] In response to the shortcomings of the existing technology, the purpose of the present invention is to provide a Transformer-based long-term trajectory prediction method and system for autonomous driving. Through high-precision modeling of autonomous driving scenarios with spatiotemporal interaction, the characteristics of traffic scenarios are accurately expressed, the amount of calculation is significantly reduced, and the accuracy of long-term trajectory prediction in complex scenarios is improved.

[0008] In order to achieve the above object, the present invention is implemented through the following technical solutions: The first aspect of the present invention provides a long-term trajectory prediction method for autonomous driving based on Transformer, comprising the following steps: Obtain the historical trajectories and map information of each traffic participant, and use the Transformer structure to perform long-term trajectory prediction for autonomous driving based on the historical trajectories and map information. The steps include: Perform scene expression based on the historical trajectories and map information. Among them, extract the relative position information in the historical trajectories to obtain historical trajectory points, add position encoding to the historical trajectory points, extract feature maps from the map information to obtain map embedding features, and perform fusion analysis on the historical trajectory points, position encoding, and map embedding features from the perspective of spatio-temporal intersection to obtain fusion information; Extract features from the perspective of spatio-temporal interaction using the fusion information, and gradually establish the interaction relationships between traffic participants to obtain the implicit behavior intention features of each traffic participant; Based on the multi-head attention mechanism, perform temporal joint prediction on the implicit behavior intention features of traffic participants in a recursive manner to obtain the predicted trajectory.

[0009] Further, the map information includes lane centerlines, intersections, traffic signals, and traffic signs.

[0010] Further, the specific steps for feature extraction from the map information are as follows: Unify the position information in the map information with the coordinate system of the historical trajectories; Extract the geometric features, topological structure features, and semantic features in the map information and splice them to obtain map embedding features.

[0011] Further, the specific steps for fusion analysis of historical trajectory points, position encoding, and map embedding features from the perspective of spatio-temporal intersection are as follows: Preprocess the historical trajectory points, position encoding, and map embedding features; Splice the preprocessed data and perform dimension alignment to obtain fusion information.

[0012] Further, the specific steps for extracting features from the perspective of spatio-temporal interaction using the fusion information and gradually establishing the interaction relationships between traffic participants are as follows: Perform self-attention calculation on the fusion information of each traffic participant to obtain its own intention features; Establish the interaction relationships between traffic participants in the local area through the cross-attention mechanism to obtain the features of traffic participants in the local area; Through the interaction of the features of traffic participants in each local area, obtain the implicit behavior intention features of traffic participants globally.

[0013] Further, the specific steps for recursively performing temporal joint prediction on the implicit behavioral intention features of traffic participants based on the multi-head attention mechanism are as follows: Use the multi-head attention mechanism to predict the implicit behavioral intention features of traffic participants to obtain potential trajectory features; Use the MLP multi-modal predictor to predict the future trajectory of traffic participants based on the potential trajectory features.

[0014] The second aspect of the present invention provides an autonomous driving long-term trajectory prediction system based on Transformer, including: A data processing module, configured to obtain the historical trajectories and map information of each traffic participant, and perform autonomous driving long-term trajectory prediction based on the historical trajectories and map information using the Transformer structure. The Transformer structure includes: A scene expression module, configured to obtain the historical trajectories and map information of each traffic participant, and perform scene expression based on the historical trajectories and map information. Among them, the relative position information in the historical trajectories is extracted to obtain historical trajectory points, position encoding is added to the historical trajectory points, feature extraction is performed on the map information to obtain map embedding features, and the historical trajectory points, position encoding, and map embedding features are fused and analyzed from the perspective of spatio-temporal intersection to obtain fusion information; A spatio-temporal intersection scene feature extraction module, configured to perform feature extraction from the perspective of spatio-temporal interaction using the fusion information, and gradually establish the interaction relationship between each traffic participant to obtain the implicit behavioral intention features of each traffic participant; A temporal joint multi-modal trajectory prediction module, configured to perform temporal joint prediction on the implicit behavioral intention features of traffic participants recursively based on the multi-head attention mechanism to obtain the predicted trajectory.

[0015] The third aspect of the present invention provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded and executed by a processor to perform the steps in the autonomous driving long-term trajectory prediction method based on Transformer described in the first aspect of the present invention.

[0016] The fourth aspect of the present invention provides a computer device, which includes: A processor, suitable for executing a computer program; A computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by the processor, it implements the autonomous driving long-term trajectory prediction method based on Transformer described in the first aspect of the present invention.

[0017] The fifth aspect of the present invention provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, so that the computer device executes the steps in the Transformer-based long-term trajectory prediction method for autonomous driving as described in the first aspect of the present invention. The above technical solution(s) has / have the following beneficial effects: The present invention discloses a Transformer-based long-term trajectory prediction method and system for autonomous driving, effectively solving the problems of insufficient accuracy and large computational amount in traffic participant trajectory prediction in the prior art. Through high-precision modeling of autonomous driving scenarios with spatio-temporal interaction, the traffic scene features are accurately expressed, enabling the historical trajectories, map information of traffic participants, and their interaction relationships with other traffic participants to be comprehensively and deeply considered, overcoming the defect of independent processing of interaction relationships in traditional graph neural network methods, and thus being able to more realistically reflect the combined influence of various factors in the traffic scene on the future trajectories of vehicles, significantly improving the accuracy of long-term trajectory prediction in complex scenarios. At the same time, the invention has also made a major breakthrough in computational efficiency, greatly reducing the computational amount. Even in complex scenarios with a large number of traffic participants, such as congested intersections, it can efficiently perform trajectory prediction, providing strong support for the application of autonomous driving technology in complex traffic environments, and having broad application prospects and important practical significance.

[0018] The present invention extracts features from historical trajectories and map information, and conducts fusion analysis on historical trajectory points, position encoding, and map embedding features from the perspective of spatio-temporal intersection, realizing their deep fusion in the spatio-temporal dimension, enabling the movement trajectories of traffic participants to be closely combined with the road environment, providing richer context information for subsequent trajectory prediction, and significantly improving the prediction accuracy. The spatio-temporal intersection fusion method adopted by the present invention can fully capture the movement characteristics of traffic participants at different time points and their interaction relationships with the surrounding environment, enabling the model to better understand the dynamic changes of the traffic scene, and thus providing more accurate input features for long-term trajectory prediction.

[0019] The feature extraction of the fused information in the present invention can not only effectively capture the complex interaction relationships among traffic participants, but also process long-time series data, significantly improving the model's ability to model complex traffic scenarios. At the same time, the computational complexity is greatly reduced, and the prediction efficiency is improved. In the trajectory prediction stage, the present invention adopts a multi-head attention mechanism to perform temporal joint prediction on the implicit behavioral intention features of traffic participants, and uses an MLP multi-modal predictor to predict the future trajectories of traffic participants based on potential trajectory features. It can not only make full use of the multi-dimensional features of traffic participants, but also effectively handle the multi-modal problems in trajectory prediction, improving the accuracy and reliability of the prediction results.

[0020] Advantages of additional aspects of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The specification drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0022] Figure 1 It is a flowchart of the method for long-time series trajectory prediction for autonomous driving based on Transformer in the first embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0024] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof; Embodiment 1: The first embodiment of the present invention provides a method for long-time series trajectory prediction for autonomous driving based on Transformer, as Figure 1As shown in the figure, the Transformer structure is used to perform long-term trajectory prediction for autonomous driving based on historical trajectories and map information, including scene representation, spatio-temporal interaction feature extraction, and multi-modal joint trajectory prediction. The scene representation part realizes the extraction and characterization of historical trajectories and map information. The spatio-temporal interaction feature extraction part establishes the interaction relationships of traffic participants and obtains the implicit behavioral intentions of each traffic participant; the multi-modal joint trajectory prediction part predicts future trajectories step by step in a recursive manner.

[0025] Specifically, it includes the following steps: Step 1: Obtain the historical trajectories and map information of each traffic participant, and perform scene representation based on the historical trajectories and map information to realize the extraction and characterization of historical trajectories and map information.

[0026] Among them, the relative position information in the historical trajectory is extracted to obtain historical trajectory points, position encoding is added to the historical trajectory points, feature extraction is performed on the map information to obtain map embedding features, and the historical trajectory points, position encoding, and map embedding features are fused and analyzed from the perspective of spatio-temporal intersection to obtain fusion information.

[0027] In this embodiment, the traffic participants are pedestrians or vehicles driven by pedestrians, and the vehicle can be a motor vehicle or a non-motor vehicle.

[0028] Step 1.1: Obtain the historical trajectories and map information of each traffic participant, and add position encoding to the historical trajectory points.

[0029] Step 1.1.1: Obtain the historical trajectories of each traffic participant.

[0030] The historical trajectories and corresponding heading angles and other information are obtained through the acquisition device, and the relative position information in the historical trajectory of each traffic participant is extracted to obtain historical trajectory points.

[0031] Step 1.1.2: Add position encoding to the historical trajectory points.

[0032] In this embodiment, the historical trajectory points are the actual coordinate sequences of the vehicle in the past period of time, which are the original position data and only describe "where the vehicle is", and are static coordinates. The position encoding is a dynamic spatio-temporal feature, which refers to mapping the original coordinates to a high-dimensional vector through mathematical transformations such as sine or cosine functions, and is used to represent the relative relationships between positions, such as time sequence and distance interval, etc., to help the model understand spatio-temporal dependencies and enhance the model's ability to model time sequence and relative distance.

[0033] In this embodiment, the historical trajectories are obtained through real-time acquisition by in-vehicle GPS, lidar, or visual positioning system, etc., so as to further obtain a sequence of historical trajectory points. By H = (x t，y t ） Perform independent encoding to obtain position encoding .

[0034] Specifically, for each traffic participant i, transfer its historical trajectory and heading angle to a certain reference frame. This can place the trajectories of different vehicles at different times in the same coordinate system, facilitating the model to capture the relative motion relationship. The coordinate transformation formula is: .

[0035] In the formula: and are the relative position (coordinate) and heading angle of traffic participant i at the t-th moment respectively. and are the absolute position (coordinate) and heading angle of traffic participant i at the t-th moment respectively. and are the absolute position (coordinate) and heading angle of the ego vehicle e at the t-th moment respectively.

[0036] After that, assign a unique learnable position encoding to each traffic participant, and then obtain the position information of traffic participant i: . The position information of traffic participant i corresponding to t = -2, t = -1, and t = 0 is respectively: , , .

[0037] In the formula, and are the position information and position encoding information of traffic participant i at the t-th moment respectively.

[0038] Step 1.1.3: Obtain map information.

[0039] The map information includes map elements such as lane centerlines, intersections, traffic signals, and traffic signs.

[0040] Step 1.2: Extract features from the map information.

[0041] Step 1.2.1: Unify the coordinate system of the position information in the map information with the historical trajectory.

[0042] Specifically, taking the positions of the autonomous vehicle (AV) at the first N moments (t = -n + 1, -n + 2,... -2, -1, 0) as the historical trajectory, with the position at t = 0 as the coordinate origin, subtract all position information of the map, such as lane lines, intersection positions, etc., from the position of the ego vehicle (AV) to obtain relative position information. Here, n represents the sequence index.

[0043] Step 1.2.4: Extract the geometric features, topological structure features, and semantic features in the map information and splice them to obtain the map embedding features.

[0044] In a specific implementation, first extract geometric features such as lanes and intersections from the map. For lanes, represent the lane as a polyline, then sample a fixed number of points and splice them into a unified dimensional vector. For example, each lane uses 20 sampled points, and each point is expressed by two-dimensional coordinates (x, y) to form a vector representation with a dimension of 40. For intersections, express information such as the contour, central position, and connected lane directions of the intersection as a unified dimensional vector. Secondly, extract topological structure features. Represent the connection relationship between lanes (such as upstream and downstream relationships) as a vector in the form of an adjacency matrix or topological embedding (such as Node2Vec). Implement the vector representation of the connection structure between intersections and lanes through graph embedding technology (such as Graph Embedding). Thirdly, extract semantic features. Use one-hot encoding or an Embedding layer for lane types such as straight, left-turn, and right-turn lanes to obtain semantic vectors of a fixed length; represent the positions and types (traffic lights, speed limit signs) of traffic lights and traffic signs with semantic vectors of a fixed dimension. Finally, after representing the above geometric, topological, and semantic features as vectors respectively, splice them and then send them into a 4-layer CNN network to output unified map embedding features. Among them, after the map embedding features are vectorized, they pass through convolution and activation functions, but no pooling operation is performed, and the [L, N] format is retained, where L is the number of lanes in the scene and N is the number of dimensions. This can facilitate the subsequent fine-grained interaction between the vehicle and each lane information, enabling the feature extraction part of the Transformer structure to explicitly perceive which lanes each traffic participant interacts with most closely.

[0045] Step 1.3: Perform fusion analysis on the encoded historical trajectory position information and the map embedding features from the perspective of spatio-temporal intersection.

[0046] Specifically, take the map embedding features and the encoded historical trajectory position information for row-wise concatenation, that is, take each traffic participant and the corresponding map embedding features as features of the same type for concatenation, and uniformly place them on the long dimension to obtain the fusion information , arrange them in chronological order, and then use the feature extraction part of the Transformer structure to process this long dimension.

[0047] In this embodiment, for a certain frame time t, first splice the encoded historical trajectory information of the i-th traffic participant to obtain , and then cascade it with the global map feature to obtain the fusion information . Send the fusion information into the Transformer in chronological order for trajectory prediction. The resulting scene representation has the advantage of comprehensiveness. Its historical trajectory points can provide direct observations of vehicle dynamic behavior, and the position encoding can enhance the model's understanding of temporal dependencies. The use of map embeddings introduces environmental prior knowledge such as lane topologies and traffic rules.

[0048] Since the map is static, this embodiment uses a CNN to extract global features. Since the trajectory is time-varying, a relative coordinate combined with position encoding method is adopted to maintain temporal consistency. The purpose of this embodiment is to unify the structured vector, the map embedding features extracted by the CNN, and the temporal features into one scene. This embodiment uses relative coordinates to eliminate the differences in absolute positions, enabling the model to focus more on interaction relationships rather than just position relationships. In short, the goal of the scene representation part of this embodiment is how to align and fuse the high-precision map with the historical trajectories of multiple vehicles to generate a unified feature representation suitable for downstream temporal models.

[0049] Step 2: Use the fusion information to extract features from the perspective of spatio-temporal interaction, gradually establish the interaction relationships between traffic participants, and obtain the implicit behavior intention features of each traffic participant. In this embodiment, first extract the intention features of the traffic participants themselves, then expand to extract local features, and finally expand to extract global features, gradually establishing the interaction relationships between traffic participants.

[0050] Step 2.1: Perform self-attention calculation on the fusion information of each traffic participant to obtain the self-intention features.

[0051] First, take the fusion information of each traffic participant at each historical moment as the input, and initially extract the driving intention features of itself through an 8-layer Transformer architecture based on the self-attention mechanism.

[0052] The self-attention calculation formula is: .

[0053] In the formula, is the fusion information of each traffic participant, is the learnable parameter, is the attention dimension, is the dimension of the input data, and Q, V, and K represent Query, Value, and Key respectively.

[0054] Step 2.2: Establish the interaction relationship among traffic participants in the local area through the cross-attention mechanism to obtain the traffic participant features in the local area.

[0055] Secondly, taking each traffic participant as the center and with a radius of R, a local area is set up ( ) In the local area, the initially extracted own driving intention feature is used as Query (Q), and the initially extracted driving intention features of other traffic participants are used as Value (V) and Key (K). Through the Transformer architecture based on the cross-attention mechanism, the features of each traffic participant in the local area are obtained ( ) to establish the interaction relationship among traffic participants in the local area and further enhance the implicit behavior intention features of each traffic participant. Among them, as Figure 1 shown, and represent different local areas, 、 、 are used to represent the features of different traffic participants in the same local area.

[0056] Step 2.3: Obtain the implicit behavior intention features of each traffic participant globally through the interaction of the traffic participant features in each local area.

[0057] Finally, taking the features of traffic participants in the local area as Query (Q), and the features of traffic participants in other local areas as Value (V) and Key (K), the features of each traffic participant globally are obtained.

[0058] Step 3: Based on the multi-head attention mechanism, perform temporal joint prediction on the implicit behavior intention features of traffic participants recursively to obtain the predicted trajectory.

[0059] Step 3.1: Use the multi-head attention mechanism to predict the implicit behavior intention features of traffic participants to obtain the potential trajectory features.

[0060] In this embodiment, the implicit behavior intention features of traffic participants are input into a 4-layer Transformer structure with a multi-head attention mechanism to obtain the potential trajectory features at t = 1.

[0061] Step 3.2: Use the MLP multi-modal predictor to predict the future trajectory of traffic participants according to the potential trajectory features.

[0062] Secondly, input the potential trajectory features at t = 1 into a two-layer MLP-based multi-modal predictor to predict the trajectories of each traffic participant at t = 1. Again, concatenate the implicit behavior intention features of each traffic participant with the potential trajectory features at t = 1 as the implicit behavior intention features of the traffic participants at t = 2, and repeat the above network architecture to obtain the trajectories of each traffic participant predicted at t = 2. Finally, loop the above operation until all trajectories are predicted.

[0063] In this embodiment, a total loss function is constructed considering multi-objective loss, and the total loss includes multi-modal optimal matching loss, modal diversity loss, kinematic smoothing loss, and interaction safety loss. The total loss function is as follows: .

[0064] In the formula, ~ are weights.

[0065] 1) Multi-modal optimal matching loss : Among all modalities, only optimize the trajectory that is closest to the ground truth to ensure that at least one modality fits the real data well and avoid the blurring of multi-modal predictions due to averaging.

[0066] .

[0067] In the formula, M is the total number of predicted modalities (e.g., generating 6 candidate trajectories). T is the number of predicted time steps (e.g., predicting the next 5 seconds, with 2 points per second, then (T = 10)). is the predicted position of the m-th modality at time t (representing the predicted position information here: two-dimensional coordinates). is the position of the ground truth trajectory at time t (representing the ground truth position information here: two-dimensional coordinates).

[0068] 2) Modal diversity loss : Encourage the generation of diverse candidate trajectories to cover different driving intentions (such as turning left, turning right, going straight) and improve the coverage ability of the prediction.

[0069] .

[0070] In the formula, is the minimum distance threshold between modalities (e.g., set to 2 meters).

[0071] 3) Kinematic smoothing loss : Constrain the second-order continuity of the trajectory to avoid sudden trajectories that do not conform to vehicle dynamics such as sudden acceleration and sudden braking.

[0072] 。

[0073] 4) Interaction safety loss : Penalize the potential collision risk of the trajectories of different participants within the same modality to enhance the safety of prediction.

[0074] 。

[0075] In the formula, represents the safety distance threshold. represents the predicted position of traffic participant i at time t in the m-th modality. represents the predicted position of traffic participant j at time t in the m-th modality, where i and j represent two different traffic participants.

[0076] Example Two: Example two of the present invention provides an autonomous driving long-term trajectory prediction system based on Transformer, including: A data processing module, configured to obtain the historical trajectories and map information of each traffic participant, and use the Transformer structure to perform autonomous driving long-term trajectory prediction based on the historical trajectories and map information. The Transformer structure includes: A scene expression module, configured to obtain the historical trajectories and map information of each traffic participant, and perform scene expression according to the historical trajectories and map information. Among them, the relative position information in the historical trajectories is extracted to obtain historical trajectory points, position encodings are added to the historical trajectory points, feature extraction is performed on the map information to obtain map embedding features, and the historical trajectory points, position encodings, and map embedding features are fused and analyzed from the perspective of spatio-temporal intersection to obtain fusion information; A spatio-temporal intersection scene feature extraction module, configured to perform feature extraction from the perspective of spatio-temporal interaction using the fusion information, and gradually establish the interaction relationships between each traffic participant to obtain the implicit behavior intention features of each traffic participant; A temporal joint multi-modal trajectory prediction module, configured to perform temporal joint prediction on the implicit behavior intention features of traffic participants in a recursive manner based on the multi-head attention mechanism to obtain the predicted trajectories.

[0077] Example Three: Example three of the present invention provides a computer-readable storage medium, which stores a computer program, and the computer program is adapted to be loaded and executed by a processor to perform the steps in the autonomous driving long-term trajectory prediction method based on Transformer as described in Example One of the present invention.

[0078] Example Four: Example four of the present invention provides a computer device, which includes: A processor, adapted to execute a computer program; A computer-readable storage medium storing a computer program, which when executed by the processor, implements the steps in the Transformer-based long-term trajectory prediction method for autonomous driving according to Embodiment 1 of the present invention.

[0079] Embodiment 5: Embodiment 5 of the present invention provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to execute the steps in the Transformer-based long-term trajectory prediction method for autonomous driving according to Embodiment 1 of the present invention.

[0080] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in this application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application. In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data processing device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive (SSD)). The above description is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A long-term trajectory prediction method for autonomous driving based on Transformer, characterized in that Including the following steps: Obtain the historical trajectories and map information of each traffic participant, and use the Transformer structure to perform long-term time-series trajectory prediction for autonomous driving based on the historical trajectories and map information. The steps include: Perform scene expression based on the historical trajectories and map information. Among them, extract the relative position information in the historical trajectories to obtain historical trajectory points, add position encoding to the historical trajectory points, extract feature of the map information to obtain map embedding features, and perform fusion analysis on the historical trajectory points, position encoding, and map embedding features from the perspective of spatio-temporal intersection to obtain fusion information; Extract features from the perspective of spatio-temporal interaction using the fusion information, and gradually establish the interaction relationships between each traffic participant to obtain the implicit behavior intention features of each traffic participant; Based on the multi-head attention mechanism, perform time-series joint prediction on the implicit behavior intention features of traffic participants in a recursive manner to obtain the predicted trajectories.

2. The method for predicting long-term trajectories of autonomous driving based on Transformer according to claim 1, wherein, The map information includes lane centerlines, intersections, traffic signals, and traffic signs.

3. The Transformer-based long-term trajectory prediction method for autonomous driving according to claim 1, wherein, The specific steps for extracting features from the map information are: Unify the position information in the map information with the coordinate system of the historical trajectories; Extract the geometric features, topological structure features, and semantic features in the map information and splice them to obtain map embedding features.

4. The Transformer-based long-term trajectory prediction method for autonomous driving according to claim 1, wherein The specific steps for performing fusion analysis on the historical trajectory points, position encoding, and map embedding features from the perspective of spatio-temporal intersection are: Preprocess the historical trajectory points, position encoding, and map embedding features; Splice the preprocessed data and align the dimensions.

5. The method for long-term trajectory prediction of autonomous driving based on Transformer according to claim 1, wherein The specific steps for extracting features from the perspective of spatio-temporal interaction using the fusion information and gradually establishing the interaction relationships between each traffic participant are: Perform self-attention calculation on the fusion information of each traffic participant to obtain its own intention features; Establish the interaction relationships between traffic participants in the local area through the cross-attention mechanism to obtain the features of traffic participants in the local area; Obtain the implicit behavior intention features of traffic participants globally through the interaction of the features of traffic participants in each local area.

6. The Transformer-based long-term trajectory prediction method for autonomous driving according to claim 1, wherein The specific steps for performing time-series joint prediction on the implicit behavior intention features of traffic participants based on the multi-head attention mechanism in a recursive manner are: Use the multi-head attention mechanism to predict the implicit behavior intention features of traffic participants to obtain potential trajectory features; Use the MLP multi-modal predictor to predict the future trajectories of traffic participants based on the potential trajectory features.

7. A long-term trajectory prediction system for autonomous driving based on Transformer, characterized in that, Including: A data processing module configured to obtain the historical trajectories and map information of each traffic participant, and use the Transformer structure to perform long-term time-series trajectory prediction for autonomous driving based on the historical trajectories and map information. The Transformer structure includes: A scene expression module configured to obtain the historical trajectories and map information of each traffic participant, perform scene expression based on the historical trajectories and map information. Among them, extract the relative position information in the historical trajectories to obtain historical trajectory points, add position encoding to the historical trajectory points, extract feature of the map information to obtain map embedding features, and perform fusion analysis on the historical trajectory points, position encoding, and map embedding features from the perspective of spatio-temporal intersection to obtain fusion information; The spatio-temporal cross-scenario feature extraction module is configured to extract features from the perspective of spatio-temporal interaction by using the fusion information, establish the interaction relationships between traffic participants step by step, and obtain the implicit behavior intention features of each traffic participant. The temporal joint multi-modal trajectory prediction module is configured to perform temporal joint prediction on the implicit behavior intention features of traffic participants in a recursive manner based on the multi-head attention mechanism to obtain the predicted trajectories.

8. A computer program product, characterized in that, The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the Transformer-based long-term autonomous driving trajectory prediction method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor to implement the Transformer-based long-term autonomous driving trajectory prediction method according to any one of claims 1-6.

10. A computer device, characterized in that a processor suitable for executing a computer program; a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, it implements the Transformer-based long-term autonomous driving trajectory prediction method according to any one of claims 1-6.

Images