Transformer-based long-term trajectory prediction method and system for autonomous driving

Abstract

The present invention discloses a Transformer-based long-term trajectory prediction method and system for autonomous driving, which relates to the field of autonomous driving technology. The method comprises the steps of: using the Transformer structure to predict the long-term trajectory of autonomous driving based on historical trajectories and map information. Specifically, adding position codes to historical trajectory points, extracting features from the map information to obtain map embedding features, and performing fusion analysis on the historical trajectory points, position codes and map embedding features from the perspective of time-space intersection; using the fused information to extract features from the perspective of time-space interaction, and establishing the interaction relationship between each traffic participant step by step; based on the multi-head attention mechanism, recursively performing time-series joint prediction on the implicit behavioral intention features of traffic participants to obtain a predicted trajectory. The present invention accurately expresses the characteristics of traffic scenes through high-precision modeling of time-space interactive autonomous driving scenes, significantly reduces the amount of calculation, and improves the accuracy of long-term trajectory prediction in complex scenes.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a Transformer-based autonomous driving long-time series trajectory prediction method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Predicting the trajectories of traffic participants is a fundamental and crucial task in autonomous driving technology, and its accuracy significantly impacts the safety of autonomous vehicles. To address this issue, researchers have focused on trajectory prediction methods based on graph neural networks (GNNs). These methods rely on an explicit graph structure, where nodes represent individual traffic participants (agents) and edges represent their interactions (e.g., distance, relative position, potential interactions between vehicles, etc.).

[0004] To better handle the relational interactions in traffic processes, some current graph neural network-based trajectory prediction methods use the following encoder processing flow: first, they process the interactions between all traffic participants (e.g., vehicle-to-vehicle interactions) at each historical moment, and integrate the feature results of all historical moments together (historical moment dimension) to further capture cross-temporal dependencies; second, they process the interactions between all traffic participants and the map (e.g., vehicle-to-map interactions).

[0005] It can be seen that the above methods independently handle interactions with other traffic participants, their own historical trajectories, and their interactions with the map. However, in real life, the future trajectory of a single vehicle is influenced by its historical trajectory, the map, and other traffic participants. The above methods cannot accurately represent the interactions between traffic participants. Furthermore, some graph convolution methods require multiple graph convolution operations to ensure effective models, making them unsuitable for complex traffic scenarios such as congested intersections.

[0006] To sum up, how to reasonably express autonomous driving scenarios and efficiently explore the interactive relationships between traffic participants is a technical problem that needs to be urgently solved by existing technologies. Summary of the Invention

[0007] In response to the shortcomings of the existing technology, the purpose of the present invention is to provide a Transformer-based long-term trajectory prediction method and system for autonomous driving. Through high-precision modeling of autonomous driving scenarios with spatiotemporal interaction, the characteristics of traffic scenarios are accurately expressed, the amount of calculation is significantly reduced, and the accuracy of long-term trajectory prediction in complex scenarios is improved.

[0008] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0009] A first aspect of the present invention provides a Transformer-based method for predicting long-time-series trajectories for autonomous driving, comprising the following steps:

[0010] Obtain the historical trajectory and map information of each traffic participant, and use the Transformer structure to perform long-term trajectory prediction for autonomous driving based on the historical trajectory and map information. The steps include:

[0011] Scenario expression is performed based on historical trajectory and map information. The relative position information in the historical trajectory is extracted to obtain historical trajectory points, and location codes are added to the historical trajectory points. Feature extraction is performed on the map information to obtain map embedding features. The historical trajectory points, location codes, and map embedding features are fused and analyzed from a spatiotemporal perspective to obtain fused information.

[0012] By using the fusion information to extract features from the perspective of spatiotemporal interaction, the interaction relationship between traffic participants is established step by step, and the implicit behavioral intention characteristics of each traffic participant are obtained;

[0013] Based on the multi-head attention mechanism, the implicit behavioral intention features of traffic participants are recursively predicted in a time series manner to obtain the predicted trajectory.

[0014] Furthermore, the map information includes lane centerlines, intersections, traffic signals, and traffic signs.

[0015] Furthermore, the specific steps for feature extraction of map information are as follows:

[0016] Unify the location information in the map information with the coordinate system of the historical trajectory;

[0017] The geometric features, topological structure features and semantic features in the map information are extracted and spliced ​​to obtain map embedding features.

[0018] Furthermore, the specific steps for integrating and analyzing historical trajectory points, location codes, and map embedding features from a spatiotemporal perspective are as follows:

[0019] Preprocess historical trajectory points, location encoding, and map embedding features;

[0020] The preprocessed data is spliced ​​and dimensionally aligned to obtain fusion information.

[0021] Furthermore, the specific steps for extracting features from the perspective of spatiotemporal interaction using fused information and gradually establishing the interaction relationship between traffic participants are as follows:

[0022] Perform self-attention calculation on the fused information of each traffic participant to obtain its own intention features;

[0023] The interactive relationship between traffic participants in the local area is established through the cross-attention mechanism to obtain the characteristics of traffic participants in the local area;

[0024] Through the interaction of the characteristics of traffic participants in each local area, the implicit behavioral intention characteristics of each traffic participant in the world are obtained.

[0025] Furthermore, the specific steps of recursively predicting the implicit behavioral intention features of traffic participants in a time series based on the multi-head attention mechanism are as follows:

[0026] The multi-head attention mechanism is used to predict the implicit behavioral intention characteristics of traffic participants and obtain the latent trajectory characteristics;

[0027] The MLP multimodal predictor is used to predict the future trajectories of traffic participants based on their latent trajectory features.

[0028] A second aspect of the present invention provides a Transformer-based autonomous driving long-time series trajectory prediction system, comprising:

[0029] The data processing module is configured to obtain the historical trajectory and map information of each traffic participant and use the Transformer structure to perform long-term trajectory prediction for autonomous driving based on the historical trajectory and map information. The Transformer structure includes:

[0030] A scenario expression module is configured to obtain the historical trajectory and map information of each traffic participant and perform scenario expression based on the historical trajectory and map information. The module extracts relative position information from the historical trajectory to obtain historical trajectory points, adds position codes to the historical trajectory points, extracts features from the map information to obtain map embedding features, and fuses and analyzes the historical trajectory points, position codes, and map embedding features from a spatiotemporal perspective to obtain fused information.

[0031] The spatiotemporal cross-scene feature extraction module is configured to use the fused information to extract features from the perspective of spatiotemporal interaction, gradually establish the interaction relationship between each traffic participant, and obtain the implicit behavioral intention characteristics of each traffic participant;

[0032] The temporal joint multimodal trajectory prediction module is configured to perform temporal joint prediction on the implicit behavioral intention features of traffic participants in a recursive manner based on the multi-head attention mechanism to obtain the predicted trajectory.

[0033] The third aspect of the present invention provides a computer-readable storage medium storing a computer program, which is suitable for being loaded by a processor and executing the steps in the Transformer-based autonomous driving long-time series trajectory prediction method as described in the first aspect of the present invention.

[0034] A fourth aspect of the present invention provides a computer device, comprising:

[0035] a processor adapted to execute a computer program;

[0036] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the Transformer-based autonomous driving long-time series trajectory prediction method as described in the first aspect of the present invention is implemented.

[0037] In a fifth aspect, the present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the Transformer-based long-time series trajectory prediction method for autonomous driving described in the first aspect of the present invention. One or more of the above technical solutions have the following beneficial effects:

[0038] The present invention discloses a Transformer-based long-term trajectory prediction method and system for autonomous driving, which effectively solves the problems of insufficient accuracy and large amount of computation in the existing technology for predicting the trajectory of traffic participants. Through high-precision modeling of autonomous driving scenes with spatiotemporal interaction, the characteristics of the traffic scene are accurately expressed, so that the historical trajectory, map information and interactive relationships of traffic participants with other traffic participants can be comprehensively and deeply considered, overcoming the defects of independent processing of interactive relationships in traditional graph neural network-based methods, thereby being able to more realistically reflect the joint influence of various factors in the traffic scene on the future trajectory of the vehicle, and significantly improving the accuracy of long-term trajectory prediction in complex scenes. At the same time, the invention has also made a major breakthrough in computational efficiency, greatly reducing the amount of computation. Even in complex scenes with a large number of traffic participants, such as congested intersections, trajectory prediction can be performed efficiently, providing strong support for the application of autonomous driving technology in complex traffic environments, and has broad application prospects and important practical significance.

[0039] The present invention extracts features from historical trajectories and map information, and fuses and analyzes historical trajectory points, location codes, and map embedding features from a spatiotemporal perspective, achieving a deep fusion of the two in the spatiotemporal dimension. This allows the movement trajectories of traffic participants to be closely integrated with the road environment, providing richer contextual information for subsequent trajectory predictions and significantly improving prediction accuracy. The spatiotemporal fusion approach adopted by the present invention can fully capture the movement characteristics of traffic participants at different time points and their interaction with the surrounding environment, enabling the model to better understand the dynamic changes in traffic scenes, thereby providing more accurate input features for long-term trajectory predictions.

[0040] The present invention's feature extraction of fused information not only effectively captures the complex interactions between traffic participants but also processes long time series data, significantly improving the model's ability to model complex traffic scenarios while significantly reducing computational complexity and increasing prediction efficiency. During the trajectory prediction phase, the present invention employs a multi-head attention mechanism to perform a time-series joint prediction of the implicit behavioral intention features of traffic participants and utilizes an MLP multimodal predictor to predict the future trajectory of traffic participants based on their underlying trajectory features. This not only fully utilizes the multidimensional characteristics of traffic participants but also effectively addresses the multimodal issues involved in trajectory prediction, improving the accuracy and reliability of the prediction results.

[0041] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0043] Figure 1 This is a flowchart of the Transformer-based long-time trajectory prediction method for autonomous driving in Example 1 of the present invention. DETAILED DESCRIPTION

[0044] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0045] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations;

[0046] Example 1:

[0047] The first embodiment of the present invention provides a Transformer-based long-term trajectory prediction method for autonomous driving. Figure 1 As shown in the figure, the Transformer architecture is used to predict long-term trajectories for autonomous driving based on historical trajectories and map information. This approach involves scene representation, spatiotemporal interaction feature extraction, and multimodal joint trajectory prediction. The scene representation component extracts and represents historical trajectories and map information. The spatiotemporal interaction feature extraction component establishes the interaction relationships between traffic participants and obtains their implicit behavioral intentions. The multimodal joint trajectory prediction component recursively predicts future trajectories.

[0048] The specific steps include:

[0049] Step 1: Obtain the historical trajectory and map information of each traffic participant, express the scene based on the historical trajectory and map information, and realize the extraction and representation of the historical trajectory and map information.

[0050] Among them, the relative position information in the historical trajectory is extracted to obtain the historical trajectory points, and the position code is added to the historical trajectory points. The map information is extracted to obtain the map embedding features. The historical trajectory points, position codes and map embedding features are fused and analyzed from the perspective of time and space intersection to obtain the fusion information.

[0051] In this embodiment, the traffic participants are pedestrians or vehicles driven by pedestrians, and the vehicles can be motor vehicles or non-motor vehicles.

[0052] Step 1.1: Obtain the historical trajectory and map information of each traffic participant and add location codes to the historical trajectory points.

[0053] Step 1.1.1: Obtain the historical trajectory of each traffic participant.

[0054] The historical trajectory and corresponding heading angle information are obtained through the acquisition equipment, and the relative position information of each traffic participant in the historical trajectory is extracted to obtain the historical trajectory point.

[0055] Step 1.1.2: Add location codes to historical trajectory points.

[0056] In this embodiment, historical trajectory points record the actual coordinate sequences of the vehicle over a period of time. These are raw location data that simply describe the vehicle's location and are static coordinates. Position encoding is a dynamic spatiotemporal feature. It involves mapping raw coordinates into high-dimensional vectors through mathematical transformations such as sine and cosine functions. These vectors represent relative relationships between locations, such as time sequence and distance intervals. This helps the model understand spatiotemporal dependencies and enhances its ability to model time sequence and relative distances.

[0057] In this embodiment, the historical trajectory is obtained by real-time acquisition of the vehicle-mounted GPS, laser radar or visual positioning system, thereby further obtaining a sequence of historical trajectory points. H=(x t ，y t ） Perform independent encoding to obtain position encoding .

[0058] Specifically, for each traffic participant i, its historical trajectory and heading angle are converted to a certain reference system. This allows the trajectories of different vehicles at different times to be placed in the same coordinate system, making it easier for the model to capture relative motion relationships. The coordinate conversion formula is:

[0059] .

[0060] Where: 、 are the relative position (coordinates) and heading angle of traffic participant i at the tth moment. 、 are the absolute position (coordinates) and heading angle of traffic participant i at the tth moment. 、 are the absolute position (coordinate) and heading angle of the vehicle e at the tth moment respectively.

[0061] Then, a unique learnable location code is assigned to each traffic participant, and the location information of traffic participant i is obtained: The corresponding location information of traffic participant i at t=-2, t=-1, and t=0 are:

[0062] ,

[0063] ,

[0064] .

[0065] Where, 、 are the location information and location coding information of traffic participant i at time t respectively.

[0066] Step 1.1.3: Get map information.

[0067] Map information includes map elements such as lane centerlines, intersections, traffic signals, and traffic signs.

[0068] Step 1.2: Extract features from map information.

[0069] Step 1.2.1: Unify the location information in the map information with the coordinate system of the historical trajectory.

[0070] Specifically, the AV's (AV) position at the previous N moments (t = -n+1, ​​-n+2, ... -2, -1, 0) is used as the historical trajectory. With the position at t = 0 as the coordinate origin, all map location information, such as lane lines and intersection locations, is subtracted from the AV's (AV) position to obtain the relative position information. Here, n represents a sequence index.

[0071] Step 1.2.4: Extract the geometric features, topological structure features, and semantic features from the map information and concatenate them to obtain map embedding features.

[0072] In one specific implementation, geometric features such as lanes and intersections are first extracted from the map. Lanes are represented as polylines, and a fixed number of points are sampled and concatenated into a uniformly dimensional vector. For example, each lane is represented by 20 sample points, each expressed as a two-dimensional (x, y) coordinate, resulting in a 40-dimensional vector. For intersections, information such as the intersection outline, center location, and directions of connected lanes is represented as a uniformly dimensional vector. Next, topological features are extracted, representing the connections between lanes (such as upstream and downstream relationships) as vectors using an adjacency matrix or topological embedding (such as Node2Vec). The connection structure between intersections and lanes is represented as a vector using graph embedding techniques (such as Graph Embedding). Semantic features are then extracted, using one-hot encoding or an embedding layer to generate fixed-length semantic vectors for lane types such as straight, left-turn, and right-turn lanes. The location and type of traffic lights and traffic signs (such as red, green, and speed limit signs) are represented as fixed-dimensional semantic vectors. Finally, the geometric, topological, and semantic features are represented as vectors, concatenated, and fed into a four-layer CNN network to output a unified map embedding feature. The vectorized map embedding features are then passed through convolution and activation functions without pooling, maintaining the [L, N] format, where L is the number of lanes in the scene and N is the number of dimensions. This facilitates fine-grained interaction between subsequent vehicles and lane information, allowing the feature extraction component of the Transformer architecture to explicitly perceive which lanes each traffic participant interacts most closely with.

[0073] Step 1.3: Fusion analysis of the encoded historical trajectory location information and map embedding features from a spatiotemporal perspective.

[0074] Specifically, embed the map into features The row dimension is cascaded with the encoded historical trajectory position information, that is, each traffic participant and the corresponding map embedding feature are cascaded as features of the same type and uniformly placed on the long dimension to obtain fusion information. , arranged in chronological order, and then the feature extraction part of the Transformer structure is used to process the long dimension.

[0075] In this embodiment, for a certain frame time t, the historical trajectory information of the i-th traffic participant after encoding is spliced ​​to obtain , and then cascade with the global map features to obtain fusion information The fused information is fed into the Transformer in chronological order for trajectory prediction. The resulting scene representation has the advantage of being comprehensive. Its historical trajectory points provide direct observations of vehicle dynamic behavior, position encoding enhances the model's understanding of temporal dependencies, and map embedding incorporates prior environmental knowledge such as lane topology and traffic regulations.

[0076] Since the map is static, this embodiment uses CNN to extract global features, and the trajectory is time-varying, so relative coordinates combined with position encoding are used to maintain temporal consistency. The purpose of this embodiment is to unify the structured vector, the map embedding features extracted by CNN, and the temporal features into one scene. This embodiment uses relative coordinates to eliminate the difference in absolute position, allowing the model to focus more on the interactive relationship rather than just the positional relationship. In short, the goal of the scene expression part of this embodiment is to align and fuse the high-precision map with the historical trajectories of multiple vehicles to generate a unified feature representation suitable for use by downstream temporal models.

[0077] Step 2: Utilize the fused information to extract features from a spatiotemporal interaction perspective, gradually establishing the interaction relationships between traffic participants and obtaining the implicit behavioral intention features of each traffic participant. This embodiment first extracts the traffic participant's own intention features, then expands to extract local features, and finally expands to extract global features, gradually establishing the interaction relationships between traffic participants.

[0078] Step 2.1: Perform self-attention calculation on the fused information of each traffic participant to obtain its own intention features.

[0079] First, the fused information of each traffic participant at each historical moment is used as input, and the driving intention features of the participant are preliminarily extracted through an 8-layer Transformer architecture based on the self-attention mechanism.

[0080] The self-attention calculation formula is:

[0081] .

[0082] Where, The fusion information of each traffic participant, is a learnable parameter, is the attention dimension, are the dimensions of the input data, Q, V, and K represent query, value, and key, respectively.

[0083] Step 2.2: Establish the interactive relationship between traffic participants in the local area through the cross-attention mechanism to obtain the characteristics of traffic participants in the local area.

[0084] Secondly, with each traffic participant as the center and R as the radius, a local area is established. ( In the local area, the initial extracted driving intention features of the vehicle are used as Query (Q), and the initial driving intention features of other traffic participants are used as Value (V) and Key (K). The features of each traffic participant in the local area are obtained through the Transformer architecture based on the cross-attention mechanism. ( ), establish the interactive relationship between traffic participants in the local area, and further enhance the implicit behavioral intention characteristics of each traffic participant. Figure 1 As shown, and Characterize different local areas, 、 、 Used to characterize the characteristics of different traffic participants in the same local area.

[0085] Step 2.3: Obtain the implicit behavioral intention characteristics of all traffic participants globally through the interaction of the characteristics of traffic participants in each local area.

[0086] Finally, the characteristics of local area traffic participants are used as Query (Q), and the characteristics of other local area traffic participants are used as Value (V) and Key (K) to obtain the characteristics of all global traffic participants.

[0087] Step 3: Based on the multi-head attention mechanism, the implicit behavioral intention features of traffic participants are recursively predicted in a time series manner to obtain the predicted trajectory.

[0088] Step 3.1: Use the multi-head attention mechanism to predict the implicit behavioral intention characteristics of traffic participants and obtain the latent trajectory characteristics.

[0089] In this embodiment, the implicit behavioral intention features of traffic participants are input into a 4-layer Transformer structure with a multi-head attention mechanism to obtain the potential trajectory features at time t=1.

[0090] Step 3.2: Use the MLP multimodal predictor to predict the future trajectories of traffic participants based on the latent trajectory features.

[0091] Next, the latent trajectory features at time t=1 are fed into a two-layer MLP-based multimodal predictor to predict the trajectories of each traffic participant at time t=1. Third, the implicit behavioral intention features of each traffic participant are concatenated with the latent trajectory features at time t=1 to form the implicit behavioral intention features of the traffic participant at time t=2. This network architecture is repeated to obtain the predicted trajectories of each traffic participant at time t=2. Finally, the previous step is repeated until all trajectories are predicted.

[0092] In this embodiment, a total loss function is constructed by considering multi-objective losses. The total loss includes multi-modal optimal matching loss, modality diversity loss, kinematic smoothness loss, and interactive safety loss. as follows:

[0093] .

[0094] Where, ~ is the weight.

[0095] 1) Multimodal optimal matching loss : Among all modes, only the trajectory closest to the true value is optimized to ensure that at least one mode is highly consistent with the real data and avoid multimodal prediction blurring due to averaging.

[0096] .

[0097] Where M is the total number of predicted modalities (e.g., generating 6 candidate trajectories). T is the number of predicted time steps (e.g., predicting the next 5 seconds with 2 points per second, then (T=10)). is the predicted position of the mth mode at time t (here represents the predicted position information: two-dimensional coordinates). is the position of the true value trajectory at time t (here represents the true value position information: two-dimensional coordinates).

[0098] 2) Modal diversity loss : Encourage the generation of diverse candidate trajectories to cover different driving intentions (such as left turn, right turn, and straight ahead) and improve the prediction coverage capability.

[0099] .

[0100] Where, is the minimum spacing threshold between modalities (e.g. set to 2 meters).

[0101] 3) Kinematic smoothness loss : Constrain the second-order continuity of the trajectory to avoid sudden trajectories that are inconsistent with vehicle dynamics, such as sudden acceleration and braking.

[0102] .

[0103] 4) Loss of interactive security : Penalize the potential collision risk of different participants' trajectories within the same modality to improve the safety of prediction.

[0104] .

[0105] Where, Indicates the safety distance threshold. represents the predicted position of traffic participant i at time t in the mth mode. represents the predicted position of traffic participant j at time t in the mth mode, where i and j represent two different traffic participants.

[0106] Example 2:

[0107] A second embodiment of the present invention provides a Transformer-based long-time-series trajectory prediction system for autonomous driving, including:

[0108] The data processing module is configured to obtain the historical trajectory and map information of each traffic participant and use the Transformer structure to perform long-term trajectory prediction for autonomous driving based on the historical trajectory and map information. The Transformer structure includes:

[0109] A scenario expression module is configured to obtain the historical trajectory and map information of each traffic participant and perform scenario expression based on the historical trajectory and map information. The module extracts relative position information from the historical trajectory to obtain historical trajectory points, adds position codes to the historical trajectory points, extracts features from the map information to obtain map embedding features, and fuses and analyzes the historical trajectory points, position codes, and map embedding features from a spatiotemporal perspective to obtain fused information.

[0110] The spatiotemporal cross-scene feature extraction module is configured to use the fused information to extract features from the perspective of spatiotemporal interaction, gradually establish the interaction relationship between each traffic participant, and obtain the implicit behavioral intention characteristics of each traffic participant;

[0111] The temporal joint multimodal trajectory prediction module is configured to perform temporal joint prediction on the implicit behavioral intention features of traffic participants in a recursive manner based on the multi-head attention mechanism to obtain the predicted trajectory.

[0112] Example 3:

[0113] Embodiment 3 of the present invention provides a computer-readable storage medium storing a computer program, which is suitable for being loaded by a processor and executing the steps in the Transformer-based autonomous driving long-time series trajectory prediction method as described in Embodiment 1 of the present invention.

[0114] Example 4:

[0115] A fourth embodiment of the present invention provides a computer device, comprising:

[0116] a processor adapted to execute a computer program;

[0117] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of the Transformer-based autonomous driving long-time series trajectory prediction method as described in Example 1 of the present invention are implemented.

[0118] Embodiment 5:

[0119] Embodiment 5 of the present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the Transformer-based autonomous driving long-time series trajectory prediction method described in Embodiment 1 of the present invention.

[0120] Those skilled in the art will appreciate that the units and algorithmic steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0121] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data processing device such as a server or data center that integrates one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0122] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technical object of a person skilled in the art that can be easily conceived of within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A Transformer-based long-term trajectory prediction method for autonomous driving, characterized by: The following steps are involved: Obtain the historical trajectory and map information of each traffic participant, and use the Transformer structure to perform long-term trajectory prediction for autonomous driving based on the historical trajectory and map information. The steps include: Scenario expression is performed based on historical trajectory and map information. The relative position information in the historical trajectory is extracted to obtain historical trajectory points, and location codes are added to the historical trajectory points. Feature extraction is performed on the map information to obtain map embedding features. The historical trajectory points, location codes, and map embedding features are fused and analyzed from a spatiotemporal perspective to obtain fused information. By using the fusion information to extract features from the perspective of spatiotemporal interaction, the interaction relationship between traffic participants is established step by step, and the implicit behavioral intention characteristics of each traffic participant are obtained; The specific steps of extracting features from the perspective of spatiotemporal interaction using fused information and gradually establishing interactive relationships between traffic participants are as follows: Perform self-attention calculation on the fused information of each traffic participant to obtain its own intention features; The interactive relationship between traffic participants in the local area is established through the cross-attention mechanism to obtain the characteristics of traffic participants in the local area; Through the interaction of traffic participant characteristics in each local area, the implicit behavioral intention characteristics of each traffic participant in the world are obtained; Based on the multi-head attention mechanism, the implicit behavioral intention features of traffic participants are recursively predicted in a time series manner to obtain the predicted trajectory; The specific steps of recursively performing time series joint prediction on the implicit behavioral intention features of traffic participants based on the multi-head attention mechanism are as follows: The multi-head attention mechanism is used to predict the implicit behavioral intention characteristics of traffic participants and obtain the latent trajectory characteristics; Using MLP multimodal predictor to predict the future trajectory of traffic participants based on their latent trajectory features, The latent trajectory features at time t=1 are input into a two-layer MLP-based multimodal predictor to predict the trajectory of each traffic participant at time t=1. Again, the implicit behavioral intention features of each traffic participant are concatenated with the latent trajectory features at time t=1 to serve as the implicit behavioral intention features of the traffic participant at time t=2. The above network architecture is repeated to obtain the predicted trajectory of each traffic participant at time t=2. Finally, the previous step is repeated until all trajectories are predicted.

2. The Transformer-based long-time-series trajectory prediction method for autonomous driving according to claim 1, wherein: Map information includes lane centerlines, intersections, traffic signals, and traffic signs.

3. The Transformer-based long-time-series trajectory prediction method for autonomous driving according to claim 1, wherein: The specific steps for feature extraction of map information are: Unify the location information in the map information with the coordinate system of the historical trajectory; The geometric features, topological structure features and semantic features in the map information are extracted and spliced ​​to obtain map embedding features.

4. The Transformer-based long-time-series trajectory prediction method for autonomous driving according to claim 1, wherein: The specific steps for fusing and analyzing historical trajectory points, location codes, and map embedding features from a spatiotemporal perspective are as follows: Preprocess historical trajectory points, location encoding, and map embedding features; The preprocessed data is spliced ​​and dimensionally aligned.

5. A Transformer-based autonomous driving long-time trajectory prediction system using the method of claim 1, characterized in that: include: The data processing module is configured to obtain the historical trajectory and map information of each traffic participant and use the Transformer structure to perform long-term trajectory prediction for autonomous driving based on the historical trajectory and map information. The Transformer structure includes: A scenario expression module is configured to obtain the historical trajectory and map information of each traffic participant and perform scenario expression based on the historical trajectory and map information. The module extracts relative position information from the historical trajectory to obtain historical trajectory points, adds position codes to the historical trajectory points, extracts features from the map information to obtain map embedding features, and fuses and analyzes the historical trajectory points, position codes, and map embedding features from a spatiotemporal perspective to obtain fused information. The spatiotemporal cross-scene feature extraction module is configured to use the fused information to extract features from the perspective of spatiotemporal interaction, gradually establish the interaction relationship between each traffic participant, and obtain the implicit behavioral intention characteristics of each traffic participant; The temporal joint multimodal trajectory prediction module is configured to perform temporal joint prediction on the implicit behavioral intention features of traffic participants in a recursive manner based on the multi-head attention mechanism to obtain the predicted trajectory.

6. A computer program product, characterized in that The computer program product includes a computer program, which, when executed by a processor, implements the Transformer-based autonomous driving long-time series trajectory prediction method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded by a processor and executing the Transformer-based autonomous driving long-time series trajectory prediction method according to any one of claims 1 to 4.

8. A computer device, characterized in that: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the Transformer-based autonomous driving long-time series trajectory prediction method according to any one of claims 1 to 4.

Images