Trajectory prediction method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure disclose a trajectory prediction method and device, electronic equipment and storage medium, the method comprising: for each local area, determining the corresponding implicit state feature according to the target input feature and the surrounding input feature; combining the local map feature, the first attention module and the second attention module to determine the local area semantic feature; determining the target semantic feature according to the local area semantic feature, the relative pose feature and the third attention module of the local area and the surrounding area respectively; determining the global map interaction feature according to the global map feature, the target semantic feature and the multi-head attention module; fusing the target semantic feature of each traffic participant with each global map interaction feature, and decoding each traffic participant's corresponding multiple predicted trajectories, membership probability and probability distribution parameters of each predicted position through a trajectory prediction decoding module to perform multi-modal trajectory prediction, improve the stability and reliability of trajectory prediction.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of automatic driving, and particularly relates to a trajectory prediction method and device, electronic equipment and a storage medium. BACKGROUND

[0002] The existing interaction-based trajectory prediction method is mainly for future trajectory prediction of a single traffic participant. The traditional single traffic participant future trajectory prediction method presents a typical multi-input-single-output architecture. However, in order to realize safe driving of an automatic driving vehicle in a complex high-dynamic scene, the automatic driving vehicle needs to quickly and accurately predict the future trajectories of multiple surrounding traffic participants (vehicles, non-motor vehicles, pedestrians, etc.) at the same time, which presents a typical multi-input-multi-output characteristic. When using the traditional multi-input-single-output architecture for multi-traffic-participant trajectory prediction, multiple calls are needed, and the amount of calculation is huge, which is difficult to meet the real-time requirements of automatic driving.

[0003] Moreover, since the motion intention of a traffic participant is unknown, the future trajectory of the traffic participant presents a typical multi-modal characteristic. However, since the data collected in a real traffic scene only records a single modal, the accuracy of multi-modal trajectory prediction achieved by extending the multi-modal in a pre-defined behavior mode depends seriously on prior knowledge, which makes the multi-modal trajectory prediction model training have the risk of modal collapse and unstable training process. SUMMARY

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a trajectory prediction method, device, electronic equipment and storage medium to quickly and accurately predict the multi-modal trajectories of multiple traffic participants and improve the stability and reliability of multi-modal trajectory prediction.

[0005] In a first aspect, the embodiments of the present disclosure provide a trajectory prediction method, which comprises:

[0006] For each local area in a current traffic scene, determining a target traffic participant implicit state feature according to a target input feature of the target traffic participant in a historical time domain;

[0007] Determining a surrounding traffic participant implicit state feature according to a surrounding input feature of the surrounding traffic participant in the historical time domain; wherein the target traffic participant is a traffic participant located at the center of the local area, and the surrounding traffic participant is a traffic participant other than the target traffic participant in the local area;

[0008] determine a local region semantic feature of the local region according to the target traffic participant implicit state feature, the surrounding traffic participant implicit state feature, a local map feature of the local region, a first attention module, and a second attention module;

[0009] determine a surrounding region corresponding to the local region according to the current traffic scene, and determine a target semantic feature of the target traffic participant according to the local region semantic feature of the local region, a local region semantic feature of the surrounding region, a relative pose feature of the surrounding region and the local region, and a third attention module;

[0010] For each traffic participant, determine a plurality of global map interaction features of the traffic participant according to a global map feature of the current traffic scene, the target semantic feature of the traffic participant, and a multi-head attention module;

[0011] fuse the target semantic feature of each traffic participant with the plurality of global map interaction features corresponding thereto respectively to obtain a plurality of fusion features, and input the plurality of fusion features corresponding to each traffic participant into a trajectory prediction decoding module to obtain a plurality of predicted trajectories corresponding to each traffic participant, a membership probability corresponding to each predicted trajectory, and a probability distribution parameter of each predicted position in each predicted trajectory.

[0012] In a second aspect, the present disclosure also provides a trajectory prediction device, which comprises:

[0013] a target traffic participant feature extraction module configured to, for each local region in a current traffic scene, determine a target traffic participant implicit state feature according to a target input feature of a target traffic participant in a historical time domain;

[0014] a surrounding traffic participant feature extraction module configured to determine a surrounding traffic participant implicit state feature according to a surrounding input feature of a surrounding traffic participant in the historical time domain; wherein the target traffic participant is a traffic participant located at the center of the local region, and the surrounding traffic participant is a traffic participant other than the target traffic participant in the local region;

[0015] a local region semantic feature extraction module configured to determine a local region semantic feature of the local region according to the target traffic participant implicit state feature, the surrounding traffic participant implicit state feature, a local map feature of the local region, a first attention module, and a second attention module;

[0016] a target semantic feature extraction module configured to determine a surrounding area corresponding to the local area according to the current traffic scene, and determine a target semantic feature of the target traffic participant according to a local area semantic feature of the local area, a local area semantic feature of the surrounding area, a relative pose feature of the surrounding area and the local area, and a third attention module;

[0017] a global map interaction feature extraction module configured to determine, for each traffic participant, a plurality of global map interaction features of the traffic participant according to a global map feature of the current traffic scene, the target semantic feature of the traffic participant, and a multi-head attention module;

[0018] a trajectory and position prediction module configured to fuse the target semantic feature of each traffic participant with the plurality of global map interaction features corresponding to the traffic participant to obtain a plurality of fusion features, and input the plurality of fusion features corresponding to each traffic participant into a trajectory prediction decoding module to obtain a plurality of predicted trajectories corresponding to each traffic participant, a membership probability corresponding to each predicted trajectory, and a probability distribution parameter of each predicted position in each predicted trajectory.

[0019] In a third aspect, an electronic device is provided. The electronic device includes one or more processors; and a memory device storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the trajectory prediction method.

[0020] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the computer program implements the trajectory prediction method.

[0021] The trajectory prediction method provided in the embodiments of the present disclosure extracts the implicit state features of the target traffic participants and the implicit state features of the surrounding traffic participants in each local region in the current traffic scene, and then processes the implicit state features of the target traffic participants, the implicit state features of the surrounding traffic participants, and the local map features of the local region by using a first attention module and a second attention module to obtain local region semantic features of the local region. Further, the corresponding surrounding region of each local region is determined, and the local region semantic features of the local region, the local region semantic features of the surrounding region, and the relative pose features of the surrounding region and the local region are processed by using a third attention module to obtain target semantic features of the target traffic participants. For each traffic participant, the global map features of the current traffic scene and the target semantic features of the traffic participant are extracted by using a multi-head attention module to obtain a plurality of global map interaction features of the traffic participant, so as to facilitate subsequent multi-modal trajectory prediction. Then, the target semantic features of each traffic participant are fused with the plurality of global map interaction features, and the plurality of fusion results of each traffic participant after fusion are input into a trajectory prediction decoding module to obtain a plurality of predicted trajectories corresponding to each traffic participant, a membership probability corresponding to each predicted trajectory, and a probability distribution parameter of each predicted position in each predicted trajectory. The multi-modal trajectory prediction of the plurality of traffic participants is realized quickly and accurately, and the stability and reliability of the multi-modal trajectory prediction are improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above and other features, advantages, and aspects of the present disclosure will become more apparent by describing in detail the embodiments thereof with reference to the annexed drawings in which:

[0023] Figure 1 A flowchart of a trajectory prediction method in the embodiments of the present disclosure;

[0024] Figure 2 A schematic diagram of obtaining target traffic participant implicit state features in the embodiments of the present disclosure;

[0025] Figure 3 A schematic diagram of a current traffic scene and each local region in the embodiments of the present disclosure;

[0026] Figure 4 A structural schematic diagram of a trajectory prediction model in the embodiments of the present disclosure;

[0027] Figure 5 A structural schematic diagram of a trajectory prediction device in the embodiments of the present disclosure;

[0028] Figure 6 FIG. 1 is a schematic diagram of a structure of an electronic device according to an embodiment of the disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the disclosure will be described in more detail with reference to the accompanying drawings. While certain embodiments of the disclosure will be illustrated in the drawings, it is understood that the disclosure can be embodied in various forms and should not be interpreted in a limited sense. Rather, the purpose of the embodiments is to deliver a more thorough and complete understanding of the disclosure to the skilled in the art. It will be understood that the drawings and embodiments of the disclosure are for illustrative purposes only and are not intended to limit the scope of the disclosure.

[0030] It should be noted that the terms "first", "second", and the like in the disclosure are merely used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] The names of the messages or information exchanged between the plurality of devices in the embodiments of the disclosure are merely used for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0032] The commonly used trajectory prediction methods are mainly divided into three categories: physical-based trajectory prediction method, behavior-based trajectory prediction method and interaction-based trajectory prediction method. Among them, the physical-based trajectory prediction method uses kinematic model or dynamic model, taking the motion state information of the traffic participant at the current time as the input to predict the future trajectory of the traffic participant; the behavior-based trajectory prediction method takes the time sequence of the motion state information of the traffic participant in the historical time domain as the input, uses machine learning or deep learning model to identify the intention, and predicts the future trajectory based on the identified intention and the pre-defined trajectory prototype. However, these two methods regard the traffic participant as an independent individual in the traffic scene, ignoring the interaction relationship between the traffic participants, and the trajectory prediction accuracy in a long time domain (more than 3s) is low. The interaction-based trajectory prediction method introduces the correlation between the traffic participants, which can realize the accurate prediction of the future trajectory of the traffic participants in a long time domain, but this method mainly faces the future trajectory prediction of a single traffic participant, showing a typical multi-input-single-output architecture. However, in order to realize the safe driving of the autonomous vehicle in a complex high-dynamic scene, the autonomous vehicle needs to quickly and accurately predict the future trajectories of multiple surrounding traffic participants (vehicles, non-motor vehicles, pedestrians, etc.) at the same time, showing a typical multi-input-multi-output feature. When using the traditional multi-input-single-output architecture for multi-traffic participant trajectory prediction, multiple calls are required, the calculation amount is huge, and it is difficult to meet the real-time requirements of autonomous driving. Moreover, since the intention of the traffic participant is unknown, the future trajectory of the traffic participant shows a typical multi-modal characteristic. In order to avoid missing detection during trajectory prediction and thus affecting the safety of the autonomous vehicle, the trajectory prediction model needs to accurately capture the multi-modal characteristics of the future trajectory of the traffic participant. However, the data collected in the real traffic scene only records a single mode, and the accuracy of the multi-modal trajectory prediction realized by the way of extending the multi-modal through the pre-defined behavior mode depends seriously on the prior knowledge, which makes the multi-modal trajectory prediction model training have the risk of mode collapse and unstable training process.

[0033] In view of the above problems, the embodiment of the present disclosure provides a trajectory prediction method to quickly and accurately predict the multi-modal trajectory of multiple traffic participants and improve the stability and reliability of the multi-modal trajectory prediction.

[0034] Figure 1 A flowchart of a trajectory prediction method in the embodiment of the present disclosure is shown in FIG. 1. The method can be executed by a trajectory prediction device, which can be realized in the form of software and / or hardware, and the device can be configured in an electronic device. As shown in FIG. 1, the method can specifically include the following steps: Figure 1

[0035] S110, for each local area in the current traffic scene, determining the implicit state feature of the target traffic participant according to the target input feature of the target traffic participant in the historical time domain. ​

[0036] wherein the current traffic scene includes a road and a plurality of traffic participants. The local area is a local area constructed in the current traffic scene with each traffic participant as the center. The target traffic participant is a traffic participant located at the center of the local area. The history time domain includes a plurality of discrete time points, including a current time point and a plurality of historical time points. The target input feature is a feature used to describe the state and attribute of the target traffic participant. The target traffic participant implicit state feature is a feature obtained by further encoding the motion time sequence characteristics of the target input feature of the target traffic participant in the history time domain, and is used to describe the motion time sequence characteristics of the target traffic participant.

[0037] Specifically, the current traffic scene is divided into a plurality of local areas, and the target traffic participant corresponding to each local area is determined. Further, the state and attribute of the target traffic participant in the current traffic scene and each historical traffic scene are processed to obtain the target input feature of the target traffic participant in the history time domain. Motion time sequence encoding is performed on the target input feature of the target traffic participant in the history time domain to obtain the target traffic participant implicit state feature.

[0038] Exemplarily, Figure 2 An exemplary diagram for obtaining the target traffic participant implicit state feature is shown. As shown in Figure 2 The motion state vector of the target traffic participant at a plurality of discrete time points in the history time domain is taken as the target input feature wherein t0, t0-T h , t0-T h +1, …, t0-1 are historical time points, t0 is the current time point, and t0-T h , t0-T h +1, …, t0-1 are historical time points), the feature encoding at each discrete time point is obtained by feature mapping through a fully connected layer (FC) respectively Then, the LSTM (Long Short-Term Memory) network is used to extract the motion time sequence characteristics of the target traffic participant in the history time domain to obtain the target traffic participant implicit state feature containing the historical motion time sequence characteristics wherein i represents the number of the local area, which can also be understood as the number of the target traffic participant.

[0039] Building upon the above example, before determining the implicit state features of target traffic participants based on their target input features in the historical time domain for each local area in the current traffic scenario, it is also possible to pre-construct local areas centered on traffic participants, and to construct global map features for the current traffic scenario, as well as local map features for each local area:

[0040] For each traffic participant in the current traffic scenario, a local area centered on the traffic participant is constructed based on preset area parameters;

[0041] Based on the global lane semantic attributes and global road node geometric attributes of the current traffic scenario, construct global map features;

[0042] Based on the local lane semantic attributes and local road node geometric attributes of each local area, local map features of each local area are constructed;

[0043] The preset region parameters include the shape and area of ​​the local region. The shape of the local region can be circular, square, etc., and the area of ​​the local region can be determined according to the area of ​​the current traffic scene region according to a preset ratio. The preset ratio can be determined according to actual needs. Global lane semantic attributes and global road node geometric attributes are obtained by constructing road nodes based on the road topology structure in the high-precision vector map information of the current traffic scene, resulting in lane semantic attributes and road node geometric attributes (position, heading). Local lane semantic attributes and local road node geometric attributes correspond to each local region. Global map features are features obtained by fusing global lane semantic attributes and global road node geometric attributes. Local map features are features obtained by fusing local lane semantic attributes and local road node geometric attributes.

[0044] For example, Figure 3 This is a schematic diagram of the current traffic scene and its various local areas, such as... Figure 3 As shown, for each traffic participant (TP) in the current traffic scenario i A local scene, namely local regions 1-4, is constructed around the core of the current traffic scene. By decomposing the current traffic scene, the number of surrounding traffic participants and map elements interacting with each target traffic participant is reduced, improving computational efficiency. Fully connected layers are used for feature embedding of global lane semantic attributes and global road node geometric attributes, respectively, and these features are aggregated to obtain global map features. Fully connected layers are used to embed features for local lane semantic attributes and local road node geometric attributes, and then aggregated to obtain local map features.

[0045] S120, determine the implicit state feature of the surrounding traffic participant according to the surrounding input feature of the surrounding traffic participant in the historical time domain.

[0046] The surrounding traffic participant is a traffic participant in the local region except the target traffic participant. The surrounding input feature is used to describe the state and attribute features of the surrounding traffic participant, and the relative pose feature between the surrounding traffic participant and the target traffic participant. The implicit state feature of the surrounding traffic participant is a feature obtained by further encoding the motion time sequence characteristics of the surrounding traffic participant in the historical time domain, and is used to describe the motion time sequence characteristics of the surrounding traffic participant.

[0047] It should be noted that the process of determining the implicit state feature of the surrounding traffic participant is similar to the process of determining the implicit state feature of the target traffic participant, which will not be repeated here.

[0048] It should be further noted that, since the dimensions of the target input feature and the dimensions of the surrounding input feature are different, two separate channels are used to process the implicit state features, and the weight parameters between the two channels are not shared.

[0049] On the basis of the above examples, the target input feature of the target traffic participant in the historical time domain and the surrounding input feature of the surrounding traffic participant in the historical time domain need to be determined in advance, which can be determined by the following ways:

[0050] A local coordinate system corresponding to the local region is established with the target traffic participant as the origin and the heading direction of the target traffic participant as the positive direction of the horizontal axis;

[0051] The target input feature of the target traffic participant in the historical time domain is constructed according to the position information, speed information, heading angle information and type information of the target traffic participant at each discrete time in the historical time domain in the local coordinate system;

[0052] The surrounding input feature of the surrounding traffic participant in the historical time domain is constructed according to the position information, speed information, heading angle information, type information, relative position, relative speed and relative heading angle of the surrounding traffic participant with respect to the target traffic participant at each discrete time in the historical time domain in the local coordinate system.

[0053] For example, the ith local region contains the target traffic participant TP i and each surrounding traffic participant TP j j = 1, …, M i (M i ≤ N), N is the number of traffic participants in the current traffic scene, and M iThe number of peripheral traffic participants in the i-th local area at each discrete time t in the historical time domain is denoted as N (t), and the target input feature of the target traffic participant is denoted as h , h , the target input feature of the target traffic participant is denoted as The peripheral input feature of the peripheral traffic participant is denoted as , is the position of the target traffic participant in the local coordinate system, and are the speeds of the target traffic participant along the horizontal axis and the vertical axis of the local coordinate system, is the heading angle of the target traffic participant in the local coordinate system, c i is the type information of the target traffic participant; is the position of the peripheral traffic participant in the local coordinate system, and are the speeds of the peripheral traffic participant along the horizontal axis and the vertical axis of the local coordinate system, is the heading angle of the peripheral traffic participant in the local coordinate system, c j is the type information of the peripheral traffic participant, and is the relative position of the peripheral traffic participant TP j relative to the target traffic participant TP i , and is the relative speed of the peripheral traffic participant TP j relative to the target traffic participant TP i , is the relative heading angle of the peripheral traffic participant TP j relative to the target traffic participant TP i . The target input feature of the target traffic participant contains its absolute motion characteristics in the local coordinate system, and the peripheral input feature of the peripheral traffic participant contains its absolute motion characteristics in the local coordinate system and its relative motion characteristics relative to the target traffic participant, so that the information contained in the target input feature and each peripheral input feature is more abundant, facilitating more accurate analysis and processing subsequently.

[0054] S130, according to the target traffic participant implicit state feature, the peripheral traffic participant implicit state feature, the local map feature of the local area, the first attention module and the second attention module, determining the local area semantic feature of the local area.

[0055] The first attention module is used to extract spatial interaction features of the target traffic participant and the surrounding traffic participant at different levels. The second attention module is used to extract multi-level interaction features between the target traffic participant and a local region map. The first attention module and the second attention module can be a single-head attention module or a multi-head attention module. The local region semantic feature is an aggregation of the target traffic participant implicit state feature, the spatial interaction features of the target traffic participant and the surrounding traffic participant at different levels, and the multi-level interaction features between the target traffic participant and the local region map.

[0056] Specifically, the target traffic participant implicit state feature and the surrounding traffic participant implicit state feature are processed by the first attention module to obtain spatial interaction features of the target traffic participant and the surrounding traffic participant. The target traffic participant implicit state feature, the spatial interaction features of the target traffic participant and the surrounding traffic participant, and the local region map feature of the local region are processed by the second attention module to obtain target traffic participant-local map interaction features. The target traffic participant implicit state feature, the spatial interaction features of the target traffic participant and the surrounding traffic participant at different levels, and the multi-level interaction features between the target traffic participant and the local region map are aggregated to obtain the local region semantic feature of the local region.

[0057] On the basis of the above example, the local region semantic feature of the local region can be determined according to the target traffic participant implicit state feature, the surrounding traffic participant implicit state feature, the local region map feature of the local region, the first attention module, and the second attention module by the following steps:

[0058] Step 1: Determine the spatial interaction feature according to the target traffic participant implicit state feature, the surrounding traffic participant implicit state feature, and the first attention module.

[0059] The spatial interaction feature is used to represent the spatial interaction of the target traffic participant and the surrounding traffic participant at different levels.

[0060] Specifically, the target traffic participant implicit state feature and the surrounding traffic participant implicit state feature are processed by the first attention module to obtain the spatial interaction feature.

[0061] Optionally, the spatial interaction feature can be determined according to the target traffic participant implicit state feature, the surrounding traffic participant implicit state feature, and the first attention module by the following method:

[0062] The target traffic participant implicit state feature is input into the first attention module as a first query vector, and the surrounding traffic participant implicit state feature is input into the first attention module as a first key vector and a first value vector to obtain the spatial interaction feature.

[0063] wherein the first query vector, the first key vector and the first value vector are query vector, key vector and value vector in the first attention module.

[0064] For example, the first attention module is a multi-head attention module, wherein the target traffic participant implicit state vector is the first query vector Q1, the surrounding traffic participant implicit state vector is the first key vector K1 and the first value vector V1, and the spatial interaction feature between the target traffic participant and the surrounding traffic participant is extracted through the first attention module.

[0065]

[0066]

[0067] wherein W O , and are parameter weight matrices in the first attention module, which can be obtained by joint training of samples.

[0068] Step two, determining the target traffic participant-local map interaction feature according to the spatial interaction feature, the target traffic participant implicit state feature, the local map feature and the second attention module.

[0069] wherein the target traffic participant-local map interaction feature is a multi-level interaction feature between the target traffic participant and the map of the local region.

[0070] Specifically, the target traffic participant-local map interaction feature can be obtained by processing the spatial interaction feature, the target traffic participant implicit state feature and the local map feature through the second attention module.

[0071] Optionally, the target traffic participant-local map interaction feature can be determined according to the spatial interaction feature, the target traffic participant implicit state feature, the local map feature and the second attention module in the following manner:

[0072] taking the aggregation result of the target traffic participant implicit state feature and the spatial interaction feature as the local region dynamic semantic feature;

[0073] taking the local region dynamic semantic feature as the second query vector, taking the local map feature as the second key vector and the second value vector, and inputting into the second attention module to obtain the target traffic participant-local map interaction feature.

[0074] The aggregation result of the implicit state feature of the target traffic participant and the spatial interaction feature is a second query vector. The local map feature is a second key vector and a second value vector. The target traffic participant-local map interaction feature is extracted by a second attention module.

[0075] The aggregation result of the implicit state feature of the target traffic participant and the spatial interaction feature is a second query vector. The local map feature is a second key vector and a second value vector. The target traffic participant-local map interaction feature is extracted by a second attention module. The aggregation result of the implicit state feature of the target traffic participant and the spatial interaction feature is a second query vector. The local map feature is a second key vector and a second value vector. The target traffic participant-local map interaction feature is extracted by a second attention module.

[0076] Step three, the implicit state feature of the target traffic participant, the spatial interaction feature and the target traffic participant-local map interaction feature are aggregated to construct the local region semantic feature of the local region.

[0077] The aggregation result of the implicit state feature of the target traffic participant and the spatial interaction feature is a second query vector. The local map feature is a second key vector and a second value vector. The target traffic participant-local map interaction feature is extracted by a second attention module. The aggregation result of the implicit state feature of the target traffic participant and the spatial interaction feature is a second query vector. The local map feature is a second key vector and a second value vector. The target traffic participant-local map interaction feature is extracted by a second attention module.

[0078]

[0079] S140, according to the current traffic scene, determine the surrounding area corresponding to the local region, according to the local region semantic feature of the local region, the local region semantic feature of the surrounding area, the relative pose feature of the surrounding area and the local region and the third attention module, determine the target semantic feature of the target traffic participant.

[0080] The surrounding area is other areas with a region distance from the local region within a preset distance. The relative pose feature includes relative position, relative speed and relative heading information. The third attention module is used to extract the long-distance interaction between the local region and the surrounding area. The third attention module can be a single-head attention module or a multi-head attention module. The target semantic feature is a feature containing local region information and global scene information.

[0081] ​​​​Specifically, as a center of the local region, other regions with a region distance within a preset distance from the local region in the current traffic scene are found as the surrounding regions corresponding to the local region. Further, the local region semantic feature of the local region, the local region semantic feature of the surrounding region and the relative pose feature of the surrounding region and the local region are processed by the third attention module to obtain the feature of long-distance interaction between the local region and the surrounding region. The feature of long-distance interaction between the local region and the surrounding region and the local region semantic feature of the local region are aggregated to obtain the target semantic feature.

[0082] On the basis of the above examples, the target semantic feature of the target traffic participant can be determined according to the local region semantic feature of the local region, the local region semantic feature of the surrounding region, the relative pose feature of the surrounding region and the local region and the third attention module in the following manner:

[0083] The local region semantic feature of the local region is taken as a third query vector, and the aggregation result of the local region semantic feature of the surrounding region and the relative feature of the surrounding region and the local region is taken as a third key vector and a third value vector, which are input into the third attention module to obtain the region interaction semantic feature of the local region;

[0084] The local region semantic feature of the local region and the region interaction semantic feature are aggregated to construct the target semantic feature of the target traffic participant.

[0085] The third query vector, the third key vector and the third value vector are the query vector, the key vector and the value vector in the third attention module. The region interaction semantic feature is the feature of long-distance interaction between the local region and the surrounding region.

[0086] For example, the local region semantic feature of the local region is the third query vector Q3, the aggregation result of the local region semantic feature of the surrounding region and the relative feature of the surrounding region and the local region is the third key vector K3 and the third value vector V3, and the region interaction semantic feature of the local region is extracted by the third attention module The local region semantic feature of the local region is the third query vector Q3, the aggregation result of the local region semantic feature of the surrounding region and the relative feature of the surrounding region and the local region is the third key vector K3 and the third value vector V3, and the region interaction semantic feature of the local region is extracted by the third attention module The local region semantic feature of the local region is aggregated to obtain the target semantic feature of the target traffic participant

[0087] S150, for each traffic participant, a plurality of global map interaction features of the traffic participant are determined according to the global map feature of the current traffic scene, the target semantic feature of the traffic participant and the multi-head attention module.

[0088] The multi-head attention module is used to extract various interactions between traffic participants and the global map. Each attention head can independently output a global map interaction feature for a traffic participant. Different attention heads can reflect different interaction methods between the traffic participant and the global map, and different degrees of attention to global map features can reflect the multimodal characteristics of the traffic participant's future trajectory. The global map interaction feature is the interaction feature between the traffic participant and the current traffic scene (global map).

[0089] Specifically, for each traffic participant, the target semantic features of the traffic participant and the global map features can be used as input. A multi-head attention module is used to extract multiple different interaction features between the traffic participant and the global map, i.e., multiple global map interaction features. The number of attention heads in the multi-head attention module is the same as the number of predicted trajectories in subsequent traffic participant trajectory prediction. Taking any traffic participant as an example, by processing the global map features of the current traffic scene and the target semantic features of the traffic participant through the multi-head attention module, the global map interaction features corresponding to each attention head can be obtained. That is, the multi-head attention module can output multiple global map interaction features for that traffic participant.

[0090] Based on the above example, multiple global map interaction features of each traffic participant can be determined in the following way, according to the global map features of the current traffic scene, the target semantic features of the traffic participant, and the multi-head attention module:

[0091] For each traffic participant, the target semantic features of the traffic participant are used as the fourth query vector, and the global map features of the current traffic scene are used as the fourth key vector and the fourth value vector. These are input into the multi-head attention module to obtain the global map interaction features of the traffic participant.

[0092] Among them, the fourth query vector, the fourth key vector, and the fourth value vector are the query vector, key vector, and value vector in the multi-head attention module.

[0093] For example, the target semantic features of traffic participants Q4 is the fourth query vector, representing the global map features of the current traffic scenario. Given the fourth key vector K4 and the fourth value vector V4, a multi-head attention module is used to extract multiple global map interaction features between traffic participants and the current traffic scene.

[0094]

[0095] Where r is the number of the attention head in the multi-head attention module, and K is the pre-defined number of attention heads in the multi-head attention module. as well as The parameter weight matrix in the multi-head attention module can be obtained through joint training of samples.

[0096] S160. The target semantic features of each traffic participant are fused with their corresponding multiple global map interaction features to obtain multiple fused features. The multiple fused features corresponding to each traffic participant are then input into the trajectory prediction decoding module to obtain multiple predicted trajectories corresponding to each traffic participant, the membership probability corresponding to each predicted trajectory, and the probability distribution parameters of each predicted position in each predicted trajectory.

[0097] The fusion feature is obtained by fusing the target semantic feature of a traffic participant with a corresponding global map interaction feature. Therefore, each traffic participant corresponds to multiple fusion results, the number of which is consistent with the number of global map interaction features. The trajectory prediction decoding module is used to predict the multimodal trajectories of each traffic participant. The predicted trajectory is the predicted future trajectory of the traffic participant; one traffic participant corresponds to multiple predicted trajectories, the number of which is consistent with the number of global map interaction features. The membership probability is the confidence level of each of the multiple predicted trajectories for a traffic participant. The probability distribution parameter of the predicted position is the Laplace probability distribution parameter of the predicted position at each discrete time point on the predicted trajectory.

[0098] Specifically, for each traffic participant, the target semantic features of that participant are fused with multiple global map interaction features corresponding to that participant, resulting in multiple fused features for that participant. Then, the fused features for each traffic participant are input into the trajectory prediction decoding module, and the output results are multiple predicted trajectories for each traffic participant, the membership probability of each predicted trajectory, and the probability distribution parameters of each predicted location within each predicted trajectory.

[0099] For example, the future trajectories of traffic participants are parameterized as a mixture model, with each mixture component corresponding to a Laplace distribution. Taking a traffic participant as an example, the target semantic features of that traffic participant are used as the model. Interaction features with corresponding global maps The fused features serve as the input to the trajectory prediction decoding module, which outputs multiple modalities of traffic participants, i.e., multiple predicted trajectories, as well as the corresponding membership probability and its Laplace probability distribution parameters (probability distribution parameters of the predicted location) for each modality.

[0100] It should be noted that the module combined with S110-S120, i.e., the module for extracting the implicit state features of the target traffic participant and the implicit state features of the surrounding traffic participants, can be used as an encoding module. Furthermore, the encoding module, the first attention module, the second attention module, the multi-head attention module, the multi-head attention module, and the trajectory prediction decoding module are all obtained through joint training based on the trajectory samples of the traffic participants collected in the real traffic scene. The encoding module, the first attention module, the second attention module, the multi-head attention module, the multi-head attention module, and the trajectory prediction decoding module can be used as a whole trajectory prediction model. The target input features of the traffic participants in the historical time domain and the surrounding input features of the surrounding traffic participants in the historical time domain can be input into the trajectory prediction model to obtain a plurality of predicted trajectories corresponding to each traffic participant, a membership probability corresponding to each predicted trajectory, and a probability distribution parameter of each predicted position in each predicted trajectory.

[0101] Optionally, the training process of the trajectory prediction model: using the weighted regression loss and classification loss as the loss function to train the trajectory prediction model, wherein the regression loss uses the negative log-likelihood function to measure the membership probability of the real trajectory appearing near the best predicted trajectory, and the classification loss uses the cross-entropy function to measure the classification accuracy. In the training process, the EarlyStopping strategy is used to verify the training effect of the trajectory prediction model on the validation set to prevent overfitting of the trajectory prediction model. The model testing process: in the test set, the input features (each target input feature and each surrounding input feature) of the multi-traffic participant in the historical time domain and the high-precision vector map information (used to extract global map features and local map features of each local area) are input, and the multi-modal trajectory of the multi-traffic participant is predicted. The minimum average error minADE of each predicted trajectory corresponding to the real trajectory and the minimum error minFDE of the end position of the real trajectory and the end position of each predicted trajectory are used as evaluation indexes to analyze the prediction performance of the trajectory prediction model.

[0102] The trajectory prediction method provided by the embodiment extracts the implicit state features of the target traffic participants and the implicit state features of the surrounding traffic participants in each local region in the current traffic scene, and then processes the implicit state features of the target traffic participants, the implicit state features of the surrounding traffic participants, and the local map features of the local region by using the first attention module and the second attention module to obtain the local region semantic features of the local region. Further, the corresponding surrounding region of each local region is determined, and the local region semantic features of the local region, the local region semantic features of the surrounding region, and the relative pose features of the surrounding region and the local region are processed by using the third attention module to obtain the target semantic features of the target traffic participants. For each traffic participant, the global map features of the current traffic scene and the target semantic features of the traffic participant are extracted by using the multi-head attention module to obtain a plurality of global map interaction features of the traffic participant, so as to facilitate subsequent multi-modal trajectory prediction. Then, the target semantic features of each traffic participant are fused with the plurality of global map interaction features, and the plurality of fusion results of each traffic participant after fusion are input into the trajectory prediction decoding module to obtain a plurality of predicted trajectories corresponding to each traffic participant, a membership probability corresponding to each predicted trajectory, and a probability distribution parameter of each predicted position in each predicted trajectory. The method realizes fast and accurate multi-modal trajectory prediction of multiple traffic participants and improves the stability and reliability of multi-modal trajectory prediction.

[0103] Figure 4 FIG. 1 is a structural schematic diagram of a trajectory prediction model according to an embodiment of the present disclosure. As shown in the figure, the trajectory prediction model includes a preliminary feature extraction network 410, an interaction feature extraction network 420, and a trajectory prediction network 430. Figure 4

[0104] The preliminary feature extraction network 410 is configured to determine, for each local region in the current traffic scene, the target traffic participant implicit state features, the surrounding traffic participant implicit state features, and the local region semantic features of the local region according to the target input features of the target traffic participants in the historical time domain, the surrounding input features of the surrounding traffic participants in the historical time domain, and the high-precision vector map information.

[0105] ​Specifically, the preliminary feature extraction network 410 is configured to determine, for each local region in the current traffic scene, a target traffic participant implicit state feature according to target input features of the target traffic participant in a historical time domain, determine a surrounding traffic participant implicit state feature according to surrounding input features of the surrounding traffic participant in the historical time domain, and determine a local region semantic feature of the local region according to the target traffic participant implicit state feature, the surrounding traffic participant implicit state feature, a local map feature of the local region, a first attention module, and a second attention module. The local map feature is constructed according to a local lane semantic attribute and a local road node geometric attribute of a local region of high-precision vector map information. Details of the preliminary feature extraction network 410 can be referred to S110-S130, which will not be repeated here.

[0106] The interaction feature extraction network 420 is configured to determine, according to the current traffic scene, a surrounding region corresponding to the local region, determine a target semantic feature of the target traffic participant according to the local region semantic feature of the local region, a local region semantic feature of the surrounding region, a relative pose feature of the surrounding region and the local region, and a third attention module.

[0107] Specifically, details of the interaction feature extraction network 420 can be referred to S140, which will not be repeated here.

[0108] The trajectory prediction network 430 is configured to determine, for each traffic participant, a plurality of global map interaction features of the traffic participant according to a global map feature of the current traffic scene, a target semantic feature of the traffic participant, and a multi-head attention module, fuse the target semantic feature of each traffic participant with the plurality of global map interaction features corresponding thereto respectively to obtain a plurality of fusion features, and input the plurality of fusion features corresponding to each traffic participant into a trajectory prediction decoding module to obtain a plurality of predicted trajectories corresponding to each traffic participant, a membership probability corresponding to each predicted trajectory, and a probability distribution parameter of each predicted position in each predicted trajectory.

[0109] Specifically, details of the trajectory prediction network 430 can be referred to S150-S160, which will not be repeated here.

[0110] The trajectory prediction model provided in the embodiment utilizes traffic scene spatial decomposition, extracts spatial interaction features of a target traffic participant-surrounding traffic participants and target traffic participant-local map interaction features based on an attention mechanism in a local area based on a hierarchical attention architecture, then extracts interaction between spatially adjacent local areas and surrounding areas, i.e., regional interaction semantic features, through an attention mechanism at a scene level, reduces the amount of calculation through traffic scene spatial decomposition, and improves the real-time performance of multi-traffic participant trajectory prediction. Moreover, the multi-head attention module corresponds to multi-modal trajectory prediction, represents different interaction modes between traffic participants and map information based on multi-head attention, thereby realizing multi-modal trajectory prediction. In the local area, when modeling the spatial interaction relationship between the target traffic participant and the surrounding traffic participants, relative motion features are introduced into the surrounding input features of the surrounding traffic participants in the historical time domain, thereby enhancing the interaction modeling representation capability. The trajectory prediction model provided in the embodiment can realize fast and accurate prediction of multi-modal trajectories of surrounding multi-traffic participants of an autonomous vehicle, output a plurality of possible trajectories and corresponding probabilities of each surrounding traffic participant in the driving scene of the autonomous vehicle, and can also be used to evaluate the collision risk of the candidate behavior and reference trajectory of the autonomous vehicle when driving, thereby providing a reference basis for behavior decision and trajectory planning of the autonomous vehicle.

[0111] Figure 5 FIG. 1 is a structural schematic diagram of a trajectory prediction device according to an embodiment of the present disclosure. Figure 5 As shown in the figure, the device comprises a target traffic participant feature extraction module 510, a surrounding traffic participant feature extraction module 520, a local area semantic feature extraction module 530, a target semantic feature extraction module 540, a global map interaction feature extraction module 550, and a trajectory and position prediction module 560.

[0112] The target traffic participant feature extraction module 510 is configured to determine, for each local region in the current traffic scene, a target traffic participant implicit state feature according to a target input feature of the target traffic participant in a historical time domain; the surrounding traffic participant feature extraction module 520 is configured to determine a surrounding traffic participant implicit state feature according to a surrounding input feature of the surrounding traffic participant in the historical time domain; wherein the target traffic participant is a traffic participant located at the center of the local region, and the surrounding traffic participant is a traffic participant other than the target traffic participant in the local region; the local region semantic feature extraction module 530 is configured to determine a local region semantic feature of the local region according to the target traffic participant implicit state feature, the surrounding traffic participant implicit state feature, a local map feature of the local region, a first attention module and a second attention module; the target semantic feature extraction module 540 is configured to determine a surrounding region corresponding to the local region according to the current traffic scene, and determine a target semantic feature of the target traffic participant according to the local region semantic feature of the local region, a local region semantic feature of the surrounding region, a relative pose feature of the surrounding region and the local region, and a third attention module; the global map interaction feature extraction module 550 is configured to determine, for each traffic participant, a plurality of global map interaction features of the traffic participant according to a global map feature of the current traffic scene, a target semantic feature of the traffic participant, and a multi-head attention module; and the trajectory and position prediction module 560 is configured to fuse the target semantic feature of each traffic participant with a plurality of global map interaction features corresponding to the traffic participant to obtain a plurality of fusion features, and input the plurality of fusion features corresponding to each traffic participant into a trajectory prediction decoding module to obtain a plurality of predicted trajectories corresponding to each traffic participant, a membership probability corresponding to each predicted trajectory, and a probability distribution parameter of each predicted position in each predicted trajectory.

[0113] On the basis of the above examples, optionally, before the target traffic participant implicit state feature is determined according to the target traffic participant input feature in the historical time domain, and the surrounding traffic participant implicit state feature is determined according to the surrounding traffic participant input feature in the historical time domain, the method further includes: an input feature determination module, configured to establish a local coordinate system corresponding to the local region with the target traffic participant as the origin and with the heading direction of the target traffic participant as the positive direction of the horizontal axis; construct the target traffic participant input feature in the historical time domain according to the position information, speed information, heading angle information and type information of the target traffic participant at each discrete time in the historical time domain in the local coordinate system; and construct the surrounding traffic participant input feature in the historical time domain according to the position information, speed information, heading angle information, type information, relative position, relative speed and relative heading angle of the target traffic participant of the surrounding traffic participant at each discrete time in the historical time domain in the local coordinate system.

[0114] On the basis of the above examples, optionally, the local region semantic feature extraction module 530 is further configured to determine a space interaction feature according to the target traffic participant implicit state feature, the surrounding traffic participant implicit state feature and the first attention module; determine a target traffic participant-local map interaction feature according to the space interaction feature, the target traffic participant implicit state feature, the local map feature and the second attention module; and aggregate the target traffic participant implicit state feature, the space interaction feature and the target traffic participant-local map interaction feature to construct the local region semantic feature of the local region.

[0115] On the basis of the above examples, optionally, the local region semantic feature extraction module 530 is further configured to input the target traffic participant implicit state feature as a first query vector, the surrounding traffic participant implicit state feature as a first key vector and a first value vector into the first attention module to obtain the space interaction feature; and correspondingly, the local region semantic feature extraction module 530 is further configured to take the aggregation result of the target traffic participant implicit state feature and the space interaction feature as a local region dynamic semantic feature; input the local region dynamic semantic feature as a second query vector, the local map feature as a second key vector and a second value vector into the second attention module to obtain the target traffic participant-local map interaction feature.

[0116] On the basis of the above examples, optionally, the target semantic feature extraction module 540 is further configured to input, into a third attention module, the local region semantic feature of the local region as a third query vector, the local region semantic feature of the surrounding region and the aggregation result of the relative feature between the surrounding region and the local region as a third key vector and a third value vector, to obtain a region interaction semantic feature of the local region; and aggregate the local region semantic feature of the local region and the region interaction semantic feature to construct the target semantic feature of the target traffic participant.

[0117] On the basis of the above examples, optionally, before the target traffic participant implicit state feature is determined according to the target input feature of the target traffic participant in the historical time domain for each local region in the current traffic scene, the method further includes a region division and map feature extraction module configured to, for each traffic participant in the current traffic scene, construct a local region centered on the traffic participant according to a preset region parameter; construct a global map feature according to a global lane semantic attribute and a global road node geometric attribute of the current traffic scene; and construct a local map feature of each local region according to a local lane semantic attribute and a local road node geometric attribute of each local region; wherein the preset region parameter includes a local region shape and a local region area.

[0118] On the basis of the above examples, optionally, the global map interaction feature extraction module 550 is further configured to, for each traffic participant, input, into a multi-head attention module, the target semantic feature of the traffic participant as a fourth query vector, the global map feature of the current traffic scene as a fourth key vector and a fourth value vector, to obtain a global map interaction feature of the traffic participant.

[0119] The trajectory prediction apparatus provided by the embodiments of the present disclosure can execute the steps in the trajectory prediction method provided by the method embodiments of the present disclosure, and has the execution steps and beneficial effects which will not be repeated here.

[0120] Figure 6 FIG. 1 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. Figure 6 FIG. 1 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. Figure 6 The electronic device shown is merely an example, and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0121] As shown in FIG. 1, the electronic device 100 can include a processor 110, a memory 120 and a communication interface 130. Figure 6As shown, the electronic device 600 can include a processing device (e.g., a central processor, a graphics processor, etc.) 601 that can perform various appropriate actions and processes to implement the methods of embodiments as described in the present disclosure according to programs stored in a read-only memory (ROM) 602 or loaded into a random access memory (RAM) 603 from a storage device 608. Various programs and data required by the electronic device 600 for operations are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0122] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods illustrated by the flowcharts, thereby implementing the trajectory prediction method as described above. In such embodiments, the computer program can be downloaded and installed from a network through a communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0123] It should be noted that the computer readable medium of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0124] The computer readable medium described above can be contained in the electronic device described above; or can exist separately and not be assembled into the electronic device. The computer readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to:

[0125] For each local area in the current traffic scene, according to the target input features of the target traffic participant in the historical time domain, determine the implicit state features of the target traffic participant;

[0126] According to the peripheral input features of the peripheral traffic participant in the historical time domain, determine the implicit state features of the peripheral traffic participant; wherein the target traffic participant is a traffic participant located at the center of the local area, and the peripheral traffic participant is a traffic participant in the local area other than the target traffic participant;

[0127] According to the target traffic participant implicit state feature, the surrounding traffic participant implicit state feature, the local map feature of the local area, the first attention module, and the second attention module, a local area semantic feature of the local area is determined;

[0128] According to the current traffic scene, a surrounding area corresponding to the local area is determined, and according to the local area semantic feature of the local area, the local area semantic feature of the surrounding area, the relative pose feature of the surrounding area and the local area, and the third attention module, a target semantic feature of the target traffic participant is determined;

[0129] For each traffic participant, according to the global map feature of the current traffic scene, the target semantic feature of the traffic participant, and the multi-head attention module, a plurality of global map interaction features of the traffic participant are determined;

[0130] The target semantic feature of each traffic participant is fused with the plurality of global map interaction features corresponding thereto respectively to obtain a plurality of fusion features, and the plurality of fusion features corresponding to each traffic participant are input into a trajectory prediction decoding module respectively to obtain a plurality of predicted trajectories corresponding to each traffic participant, a membership probability corresponding to each predicted trajectory, and a probability distribution parameter of each predicted position in each predicted trajectory.

[0131] Optionally, when the one or more programs are executed by the electronic device, the electronic device can further perform other steps described in the above embodiments.

[0132] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples of the machine-readable storage medium will include one or more lines of electrical connection, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0133] The above description is merely exemplary of the disclosure and the application of the principles thereof and it is not intended to limit the scope of the disclosure to the specific forms set forth. The disclosure is susceptible to numerous modifications and alternative forms of implementation, all of which are intended to be within the scope of the disclosure. Indeed, the specific forms set forth are only exemplary embodiments of the disclosure and other embodiments are possible. The scope of the disclosure is therefore not intended to be limited to the specific forms set forth and any modifications and / or improvements are intended to be within the scope of the disclosure.

Claims

1. A trajectory prediction method, characterized in that, The method includes: For each local area in the current traffic scenario, the implicit state features of the target traffic participants are determined based on their target input features in the historical time domain. Based on the surrounding input characteristics of surrounding traffic participants in the historical time domain, the implicit state characteristics of surrounding traffic participants are determined; wherein, the target traffic participant is the traffic participant located at the center of the local area, and the surrounding traffic participants are the traffic participants in the local area other than the target traffic participant. Based on the implicit state features of the target traffic participant, the implicit state features of the surrounding traffic participants, the local map features of the local area, the first attention module, and the second attention module, the local area semantic features of the local area are determined. Based on the current traffic scenario, the surrounding area corresponding to the local area is determined. Based on the local semantic features of the local area, the local semantic features of the surrounding area, the relative pose features of the surrounding area and the local area, and the third attention module, the target semantic features of the target traffic participant are determined. For each traffic participant, based on the global map features of the current traffic scene, the target semantic features of the traffic participant, and the multi-head attention module, multiple global map interaction features of the traffic participant are determined. The target semantic features of each traffic participant are fused with their corresponding multiple global map interaction features to obtain multiple fused features. These multiple fused features are then input into the trajectory prediction and decoding module to obtain multiple predicted trajectories for each traffic participant, the membership probability of each predicted trajectory, and the probability distribution parameters of each predicted position in each predicted trajectory.

2. The method according to claim 1, characterized in that, Before determining the implicit state features of the target traffic participant based on the target input features in the historical time domain, and determining the implicit state features of the surrounding traffic participants based on the surrounding input features in the historical time domain, the method further includes: A local coordinate system is established with the target traffic participant as the origin and the target traffic participant's heading as the positive horizontal axis. Based on the location information, speed information, heading angle information, and type information of the target traffic participant at each discrete moment in the historical time domain under the local coordinate system, the target input features of the target traffic participant in the historical time domain are constructed. Based on the location information, speed information, heading angle information, type information, relative position, relative speed, and relative heading angle of the surrounding traffic participants at each discrete moment in the historical time domain under the local coordinate system, the surrounding input features of the surrounding traffic participants in the historical time domain are constructed.

3. The method according to claim 1, characterized in that, The step of determining the local semantic features of the local area based on the implicit state features of the target traffic participant, the implicit state features of the surrounding traffic participants, the local map features of the local area, the first attention module, and the second attention module includes: Based on the implicit state characteristics of the target traffic participant, the implicit state characteristics of the surrounding traffic participants, and the first attention module, spatial interaction characteristics are determined. Based on the spatial interaction features, the implicit state features of the target traffic participant, the local map features, and the second attention module, the target traffic participant-local map interaction features are determined. The implicit state features of the target traffic participants, the spatial interaction features, and the target traffic participants-local map interaction features are aggregated to construct the local semantic features of the local area.

4. The method according to claim 3, characterized in that, The step of determining spatial interaction features based on the implicit state features of the target traffic participant, the implicit state features of surrounding traffic participants, and the first attention module includes: Using the implicit state features of the target traffic participant as the first query vector, and the implicit state features of the surrounding traffic participants as the first key vector and the first value vector, the data are input into the first attention module to obtain spatial interaction features. Accordingly, determining the target traffic participant-local map interaction features based on the spatial interaction features, the implicit state features of the target traffic participant, the local map features, and the second attention module includes: The aggregation result of the implicit state features of the target traffic participants and the spatial interaction features is used as the local region dynamic semantic features; Using the local area dynamic semantic features as the second query vector and the local map features as the second key vector and second value vector, respectively, the data is input into the second attention module to obtain the target traffic participant-local map interaction features.

5. The method according to claim 1, characterized in that, The step of determining the target semantic features of the target traffic participant based on the local semantic features of the local region, the local semantic features of the surrounding region, the relative pose features of the surrounding region and the local region, and the third attention module includes: Using the local semantic features of the local region as the third query vector, and the aggregation result of the local semantic features of the surrounding region and the relative features between the surrounding region and the local region as the third key vector and the third value vector, the input is fed into the third attention module to obtain the regional interaction semantic features of the local region. The local semantic features and regional interaction semantic features of the local area are aggregated to construct the target semantic features of the target traffic participant.

6. The method according to claim 1, characterized in that, Before determining the implicit state features of the target traffic participant for each local area in the current traffic scenario based on the target input features of the target traffic participant in the historical time domain, the method further includes: For each traffic participant in the current traffic scenario, a local region centered on the traffic participant is constructed based on preset region parameters. Based on the global lane semantic attributes and global road node geometric attributes of the current traffic scenario, construct global map features; Based on the local lane semantic attributes and local road node geometric attributes of each local area, local map features of each local area are constructed; The preset region parameters include the shape and area of ​​the local region.

7. The method according to claim 1, characterized in that, For each traffic participant, based on the global map features of the current traffic scene, the target semantic features of the traffic participant, and the multi-head attention module, multiple global map interaction features of the traffic participant are determined, including: For each traffic participant, the target semantic features of the traffic participant are used as the fourth query vector, and the global map features of the current traffic scene are used as the fourth key vector and the fourth value vector. These are input into the multi-head attention module to obtain the global map interaction features of the traffic participant.

8. A trajectory prediction device, characterized in that, include: The target traffic participant feature extraction module is used to determine the implicit state features of the target traffic participants for each local area in the current traffic scene, based on the target input features of the target traffic participants in the historical time domain. The surrounding traffic participant feature extraction module is used to determine the implicit state features of surrounding traffic participants based on the surrounding input features of surrounding traffic participants in the historical time domain; wherein, the target traffic participant is the traffic participant located at the center of the local area, and the surrounding traffic participants are the traffic participants in the local area other than the target traffic participant. The local region semantic feature extraction module is used to determine the local region semantic features of the local region based on the implicit state features of the target traffic participant, the implicit state features of the surrounding traffic participants, the local map features of the local region, the first attention module, and the second attention module. The target semantic feature extraction module is used to determine the surrounding area corresponding to the local area based on the current traffic scene, and to determine the target semantic features of the target traffic participant based on the local area semantic features of the local area, the local area semantic features of the surrounding area, the relative pose features of the surrounding area and the local area, and the third attention module. The global map interaction feature extraction module is used to determine multiple global map interaction features of each traffic participant based on the global map features of the current traffic scene, the target semantic features of the traffic participant, and the multi-head attention module. The trajectory and location prediction module is used to fuse the target semantic features of each traffic participant with their corresponding multiple global map interaction features to obtain multiple fused features. The multiple fused features corresponding to each traffic participant are then input into the trajectory prediction and decoding module to obtain multiple predicted trajectories corresponding to each traffic participant, the membership probability corresponding to each predicted trajectory, and the probability distribution parameters of each predicted location in each predicted trajectory.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the trajectory prediction method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the trajectory prediction method as described in any one of claims 1-7.

Images