Trajectory prediction method, device and related equipment for traffic participants

Abstract

The application provides a traffic participant trajectory prediction method and device and related equipment, wherein the method comprises: obtaining N sets of historical trajectory information of N types of traffic participants, the historical trajectory information set comprising a heading angle feature, a time feature and a space feature; obtaining a trajectory direction according to the heading angle feature in the N sets of historical trajectory information; obtaining first trajectory information of the N types of traffic participants according to the trajectory direction, the time feature and the space feature; and generating predicted trajectory information of the N types of traffic participants according to the first trajectory information of the N types of traffic participants. The application first determines the trajectory direction of the traffic participant based on the heading angle feature, and then generates the first trajectory information of the N types of traffic participants in the complex traffic scene by comprehensively considering the time feature, the space feature and the trajectory direction, so as to realize the standardized processing of the N sets of historical trajectory information, and finally obtain the predicted trajectory information of the corresponding N types of traffic participants with high accuracy.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular to a method, device and related equipment for predicting the trajectory of traffic participants. Background Technology

[0002] In the field of road traffic, the trajectory of traffic participants can not only reflect their travel path on the road, but also their spatiotemporal characteristics during their journey.

[0003] Currently, trajectory prediction using related technologies is mostly geared towards simple traffic scenarios. That is, in the process of predicting the trajectory of traffic participants, related technologies usually assume that the traffic participants on the road only include vehicles. In complex traffic scenarios, the types of traffic participants are more diverse, such as motor vehicles, two-wheeled vehicles, and pedestrians. Different types of traffic participants have significantly different travel patterns. In other words, the accuracy of trajectory prediction in complex traffic scenarios is currently low. Summary of the Invention

[0004] This application provides a method, apparatus, and related equipment for predicting the trajectory of traffic participants, in order to solve the problem of low accuracy in trajectory prediction under complex traffic scenarios using related technologies.

[0005] To solve the above problems, this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a method for predicting the trajectory of traffic participants, the method comprising:

[0007] Obtain N historical trajectory information sets, each of which corresponds to a type of traffic participant, and each of the historical trajectory information sets includes the heading angle features, time features, and spatial features of the corresponding type of traffic participant, where N is an integer greater than 1;

[0008] Based on the heading angle features of N historical trajectory information sets, the trajectory directions of N types of traffic participants are obtained;

[0009] Based on the trajectory directions of the N types of traffic participants, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets, the first trajectory information of the N types of traffic participants is obtained;

[0010] Based on the first trajectory information of the N types of traffic participants, the trajectory characteristics of the N types of traffic participants are obtained;

[0011] Based on the trajectory characteristics of the N types of traffic participants, the predicted trajectory information of the N types of traffic participants is obtained.

[0012] In some implementations, obtaining the first trajectory information of the N types of traffic participants based on their trajectory directions, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets includes:

[0013] Acquire multiple scene trajectory information, wherein any two scene trajectory information among the multiple scene trajectory information correspond to different traffic scenes;

[0014] The target scene trajectory information is determined based on the multiple scene trajectory information, wherein the target scene trajectory information is the scene trajectory information that includes the largest number of traffic participants and / or the largest number of traffic participant categories among the multiple scene trajectory information;

[0015] The trajectory directions of the N types of traffic participants, the temporal features of the N historical trajectory information sets, and the spatial features of the N historical trajectory information sets are mapped to the target scene trajectory information to obtain the first trajectory information of the N types of traffic participants.

[0016] In some implementations, mapping the trajectory directions of the N types of traffic participants, the temporal features of the N historical trajectory information sets, and the spatial features of the N historical trajectory information sets to the target scene trajectory information to obtain the first trajectory information of the N types of traffic participants includes:

[0017] Based on the trajectory directions of the N types of traffic participants, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets, an undirected graph network of the N types of traffic participants is generated.

[0018] The undirected graph network is mapped onto the target scene trajectory information to obtain the first trajectory information of the N types of traffic participants.

[0019] In some implementations, obtaining the predicted trajectory information of the N types of traffic participants based on their trajectory characteristics includes:

[0020] The trajectory features of the N types of traffic participants are encoded using a recurrent neural network to obtain the first feature information of the N types of traffic participants;

[0021] The predicted trajectory information of the N types of traffic participants is obtained by decoding the first feature information of the N types of traffic participants using a multilayer perceptron.

[0022] In some implementations, the first trajectory information includes multiple feature data, wherein any two feature data points correspond to different time nodes;

[0023] The step of decoding the first feature information of the N types of traffic participants based on the multilayer perceptron to obtain the predicted trajectory information of the N types of traffic participants includes:

[0024] Based on the attention mechanism, multiple feature data in the first trajectory information of the first target class traffic participant are weighted and processed to obtain the second feature information of the first target class traffic participant. The first target class traffic participant is any one of the N types of traffic participants.

[0025] The second feature information of the first target class traffic participant is decoded by the multilayer perceptron to obtain the predicted trajectory information of the first target class traffic participant.

[0026] In some implementations, obtaining the trajectory features of the N types of traffic participants based on the first trajectory information of the N types of traffic participants includes:

[0027] The first trajectory information of the N types of traffic participants is standardized to obtain the second trajectory information of the N types of traffic participants;

[0028] Multiple feature extraction processes are performed on the second trajectory information of the N types of traffic participants to obtain multiple feature output information; wherein, the feature extraction includes sequentially performed graphic feature processing and temporal convolution processing;

[0029] By performing skip connections on the multiple feature output information, the trajectory features of the N types of traffic participants can be obtained.

[0030] In some implementations, the second target class of traffic participants is any one of the N classes of traffic participants;

[0031] The historical trajectory information set of the second target traffic participant includes historical trajectory sub-information, which includes the heading angle characteristics, temporal characteristics, and spatial characteristics of the target traffic participant. The second target traffic participant includes the target traffic participant.

[0032] Secondly, embodiments of this application also provide a trajectory prediction device for traffic participants, comprising:

[0033] The acquisition module is used to acquire N historical trajectory information sets, each of which corresponds to a type of traffic participant, and each of the historical trajectory information sets includes the heading angle features, time features and spatial features of the corresponding type of traffic participant, where N is an integer greater than 1.

[0034] The direction confirmation module is used to obtain the trajectory direction of N types of traffic participants based on the heading angle features of N historical trajectory information sets.

[0035] The data processing module is used to obtain the first trajectory information of the N types of traffic participants based on the trajectory direction of the N types of traffic participants, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets.

[0036] The feature extraction module is used to obtain the trajectory features of the N types of traffic participants based on the first trajectory information of the N types of traffic participants;

[0037] The prediction module is used to obtain the predicted trajectory information of the N types of traffic participants based on their trajectory characteristics.

[0038] Thirdly, embodiments of this application also provide a communication device, including: a transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is configured to read the program in the memory to implement the steps in the method described in the first aspect above.

[0039] Fourthly, embodiments of this application also provide a readable storage medium for storing a program, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0040] In this embodiment of the application, when faced with a complex traffic scenario, the system obtains N historical trajectory information sets of N types of traffic participants in the traffic scenario. First, it determines the trajectory direction of the N types of traffic participants based on the heading angle features in the historical trajectory information sets. Then, it generates the first trajectory information of the N types of traffic participants by combining the time features, spatial features, and determined trajectory directions in the historical trajectory information sets. This achieves the standardization of the N historical trajectory information sets. Finally, it performs data processing on the standardized first trajectory information to obtain the predicted trajectory information of the corresponding N types of traffic participants with high accuracy. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating a method for predicting the trajectory of traffic participants provided in an embodiment of this application;

[0043] Figure 2 This is a flowchart illustrating the encoding and decoding operations provided in an embodiment of this application;

[0044] Figure 3 This is a schematic diagram of the structure of a trajectory prediction device for traffic participants provided in an embodiment of this application;

[0045] Figure 4 This is a schematic diagram of the communication equipment provided in this application. Detailed Implementation

[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0047] The terms "first," "second," etc., used in the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. Additionally, the use of "and / or" in this application indicates at least one of the connected objects, such as A and / or B and / or C, representing seven possibilities: including A alone, B alone, C alone, and the presence of both A and B, both B and C, both A and C, and the presence of A, B, and C.

[0048] The following describes a method for predicting the trajectory of traffic participants provided in an embodiment of this application.

[0049] See Figure 1 , Figure 1 This is a flowchart illustrating a method for predicting the trajectory of traffic participants provided in an embodiment of this application, as shown below. Figure 1 As shown, the trajectory prediction method for traffic participants may include the following steps:

[0050] Step 101: Obtain N sets of historical trajectory information.

[0051] Each of the historical trajectory information sets corresponds to a type of traffic participant, and each of the historical trajectory information sets includes the heading angle features, time features, and spatial features of the corresponding type of traffic participant, where N is an integer greater than 1.

[0052] Step 102: Based on the heading angle features of N historical trajectory information sets, obtain the trajectory directions of N types of traffic participants.

[0053] Step 103: Obtain the first trajectory information of the N types of traffic participants based on the trajectory direction of the N types of traffic participants, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets.

[0054] Step 104: Obtain the trajectory features of the N types of traffic participants based on the first trajectory information of the N types of traffic participants.

[0055] Step 105: Obtain the predicted trajectory information of the N types of traffic participants based on their trajectory characteristics.

[0056] As described above, when facing complex traffic scenarios, this application obtains N historical trajectory information sets corresponding to N types of traffic participants in the traffic scenario, first determines the trajectory direction of the N types of traffic participants based on the heading angle features in the historical trajectory information sets, and then generates the first trajectory information of the N types of traffic participants by combining the time features, spatial features in the historical trajectory information sets and the determined trajectory direction. This achieves standardized processing of the N historical trajectory information sets. Finally, the standardized first trajectory information is processed to obtain predicted trajectory information corresponding to the N types of traffic participants with high accuracy.

[0057] For example, the aforementioned complex traffic scenario can be understood as a scenario in which the autonomous vehicle is located in a traffic environment where multiple types of traffic participants, such as motor vehicles, pedestrians, and two-wheeled vehicles, coexist.

[0058] For example, the process of obtaining the trajectory direction of the N types of traffic participants based on the heading angle features of N historical trajectory information sets can be as follows:

[0059] Taking pedestrian categories as an example, let the historical time step be T, and let the heading angle of a certain pedestrian Q at time t be hdg. Qt (t=1,…,T), the change in the heading angle of pedestrian Q over the historical time step T can be expressed as: Based on preset direction thresholds δ1, δ2, and δ3, different intervals [0,δ1), [δ1,δ2), and [δ2,δ3) are set for different trajectory directions. The values ​​of δ1, δ2, and δ3 are all from 0 to 2π. The interval [0,δ1) corresponds to the straight trajectory direction, the interval [δ1,δ2) corresponds to the left-turn trajectory direction, and the interval [δ2,δ3) corresponds to the right-turn trajectory direction. As shown above, based on the heading angle change value corresponding to pedestrian Q and the different intervals corresponding to the three trajectory directions, the trajectory direction of pedestrian Q in the historical time step T can be determined. For example, if the heading angle change value corresponding to pedestrian Q is in [0,δ1), then the trajectory direction of the pedestrian is determined to be straight.

[0060] It should be noted that pedestrian Q can be any pedestrian in the pedestrian category. Q can be understood as the unique number of the pedestrian in the current complex traffic scenario (used to distinguish the pedestrian from other traffic participants in the current complex traffic scenario). The process of determining the trajectory direction of other traffic participants (such as motor vehicles, two-wheeled vehicles, etc.) is similar to the process of determining the trajectory direction of pedestrians. To avoid repetition, it will not be described again.

[0061] Furthermore, after determining the trajectory direction of each traffic participant, the trajectory direction scaling factor for each type of traffic participant can be calculated accordingly. Specifically, Among them, c j This can be understood as the j-th type of traffic participant, where n is the total number of traffic participant categories in the current complex traffic scenario. This can be understood as summing the heading angle changes for each traffic participant included in the j-th type of traffic participant.

[0062] As shown above, the matrix representation of the historical trajectories of all traffic participants in the current complex traffic scenario can be:

[0063]

[0064] Where P represents the trajectory matrix of pedestrians (where c1 refers to pedestrians), B represents the trajectory matrix of two-wheeled vehicles (where c2 refers to two-wheeled vehicles), V represents the trajectory matrix of motor vehicles (where c3 refers to motor vehicles), and O represents the trajectory matrix of other traffic participants in the scene besides pedestrians, motor vehicles, and two-wheeled vehicles.

[0065] It should be noted that the process of calculating the trajectory direction scaling factor described above can be understood as a process of normalizing the trajectory directions of different types of traffic participants, which facilitates the subsequent data processing of matrix H.

[0066] It should be noted that for any one of the N types of traffic participants, the historical trajectory information set corresponding to that type of traffic participant will include at least one traffic participant and historical sub-trajectory information for each traffic participant. The historical sub-trajectory information includes the traffic participant's frame number ID, the traffic participant's identifier ID (used to distinguish it from other traffic participants), the traffic participant's category, the traffic participant's current longitude coordinates, the traffic participant's current latitude coordinates, the traffic participant's heading angle, and other information.

[0067] In addition, the above-mentioned N types of traffic participants include M traffic participants, which can be understood as all traffic participants appearing in the last frame with a historical time step of T, where M is an integer greater than N.

[0068] In some implementations, obtaining the first trajectory information of the N types of traffic participants based on their trajectory directions, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets includes:

[0069] Acquire multiple scene trajectory information, wherein any two scene trajectory information among the multiple scene trajectory information correspond to different traffic scenes;

[0070] The target scene trajectory information is determined based on the multiple scene trajectory information, wherein the target scene trajectory information is the scene trajectory information that includes the largest number of traffic participants and / or the largest number of traffic participant categories among the multiple scene trajectory information;

[0071] The trajectory directions of the N types of traffic participants, the temporal features of the N historical trajectory information sets, and the spatial features of the N historical trajectory information sets are mapped to the target scene trajectory information to obtain the first trajectory information of the N types of traffic participants.

[0072] Due to the influence of traffic scenario factors such as road speed, road conditions, weather, and traffic participant density, the matrix H of historical trajectories of all traffic participants in a complex traffic scenario will vary significantly with changes in the scenario. In this case, by pre-acquiring multiple scenario trajectory information H1,…,H corresponding to different scenarios… m Where m is the total number of scenes, and the target scene trajectory information H is determined from multiple scene trajectory information. a Subsequently, the trajectory directions of the N types of traffic participants, the temporal features of the N historical trajectory information sets, and the spatial features of the N historical trajectory information sets are mapped to the target scene trajectory information H. a In this process, the matrix H of historical trajectories of all traffic participants in the current complex traffic scenario is standardized to reduce the impact of trajectory data distribution differences caused by traffic scenario factors, improve the generalization ability of the prediction model, and enable the final output of predicted trajectory information for N types of traffic participants to have high accuracy. The prediction model can be understood as a model applying the trajectory prediction method for traffic participants provided in this application. In some embodiments, mapping the trajectory directions of the N types of traffic participants, the temporal features of the N historical trajectory information sets, and the spatial features of the N historical trajectory information sets to the target scenario trajectory information to obtain the first trajectory information of the N types of traffic participants includes:

[0073] Based on the trajectory directions of the N types of traffic participants, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets, an undirected graph network of the N types of traffic participants is generated.

[0074] The undirected graph network is mapped onto the target scene trajectory information to obtain the first trajectory information of the N types of traffic participants.

[0075] As described above, an undirected graph network is used to construct the spatiotemporal relationship between all traffic participants in the current complex traffic scenario based on the trajectory directions of N types of traffic participants and the temporal and spatial features of N historical trajectory information sets. This allows for the effective extraction of the interaction relationships between multiple traffic participants from the N historical trajectory information sets, thereby further improving the accuracy of the predicted trajectory information of the N types of traffic participants.

[0076] The undirected graph network of N types of traffic participants includes graph nodes and node connections. A graph node can be understood as any traffic participant in the current complex traffic scenario, and a node connection can be understood as the distance between two graph nodes.

[0077] For example, the process of generating an undirected graph network of N types of traffic participants can be as follows:

[0078] A spatial graph is constructed based on the spatial features of N historical trajectory information sets. The adjacency matrix corresponding to the spatial graph can be represented as:

[0079]

[0080] Among them, edge <v xt v yt >∈E S This can be understood as the graph node v at time t. xt And graph node v yt The spatial distance between them is less than the distance threshold; e dist The Euclidean distance between the two traffic participants is μ, which is a weighting coefficient (the weighting coefficient can be adaptively adjusted, for example, setting μ = 2).

[0081] A time map is constructed based on the spatial features of N historical trajectory information sets. The adjacency matrix corresponding to the time map can be represented as:

[0082]

[0083] Among them, dge <v αT v αβ >∈E TThis can be understood as traffic participant α appearing at time T (i.e., the last moment of the historical time step), and traffic participant α also appearing at time β, which is adjacent to time T, and the time interval between time T and time β is less than a time threshold; e div σ can be understood as the time interval between time T and time β, and σ can be understood as the acquisition frequency of the historical trajectory information set (in Hertz). For example, if the acquisition frequency is set to 1 frame per second, then σ = 1.

[0084] By summing up the aforementioned trajectory direction scaling factors, as well as the adjacency matrices of the corresponding spatial and temporal graphs, an undirected graph network (in the form of adjacency matrices) for N types of traffic participants can be generated.

[0085] It should be noted that each adjacency matrix of the corresponding spatial graph is a k×k matrix, where k is the total number of all traffic participants appearing in the last frame with a historical time step of T. The adjacency matrix of the corresponding temporal graph is also a k×k matrix.

[0086] In some implementations, obtaining the predicted trajectory information of the N types of traffic participants based on their trajectory characteristics includes:

[0087] The trajectory features of the N types of traffic participants are encoded using a recurrent neural network to obtain the first feature information of the N types of traffic participants;

[0088] The predicted trajectory information of the N types of traffic participants is obtained by decoding the first feature information of the N types of traffic participants using a multilayer perceptron.

[0089] Preferably, feature extraction is performed on the first trajectory information of the N types of traffic participants using a feature extraction model to obtain the trajectory features of the N types of traffic participants.

[0090] For example, by applying several gated recurrent units (GRUs) to encode the trajectory features of the N types of traffic participants, the hidden state vector output by a single gated recurrent unit at any time t can be represented as:

[0091] h t =GRU(h t-1 ,F t )

[0092] Among them, h t-1 F is the hidden state vector generated by the gated recurrent unit in the previous time step. t This is the spatiotemporal feature vector at time t output by the feature extraction model, where t is greater than or equal to 1 and less than or equal to the historical time step T.

[0093] At time t, the different hidden state vectors of several gated recurrent units are statistically analyzed:

[0094] d I =h 1t +h 2o +…+h ot

[0095] Among them, h ot This can be understood as the hidden state vector of the o-th gated loop unit at time t, where o can be understood as the total number of gated loop units, and d I The decoding process for the I-th traffic participant among all traffic participants appearing in the last frame with a historical time step of T can be as follows:

[0096] z pre =MLP(d I )

[0097] Among them, z pre This can be understood as the predicted trajectory of the I-th traffic participant. The Multilayer Perceptron (MLP), also known as the Artificial Neural Network (ANN), has multiple hidden layers in the middle in addition to the input and output layers. The simplest MLP contains only one hidden layer, i.e., a three-layer structure.

[0098] As described above, the trajectory features of N types of traffic participants are input into an encoder composed of several gated recurrent units. Then, the hidden state vectors from the encoder and the coordinates of each traffic participant in the historical time step are input into a decoder (i.e., a multilayer perceptron) to predict the position coordinates of each traffic participant in the future time step. Using a multilayer perceptron as the decoder reduces the accumulated error in the decoding stage, resulting in higher accuracy of the final output predicted trajectory information.

[0099] In some implementations, the first trajectory information includes multiple feature data, wherein any two feature data points correspond to different time nodes;

[0100] The step of decoding the first feature information of the N types of traffic participants based on the multilayer perceptron to obtain the predicted trajectory information of the N types of traffic participants includes:

[0101] Based on the attention mechanism, multiple feature data in the first trajectory information of the first target class traffic participant are weighted and processed to obtain the second feature information of the first target class traffic participant. The first target class traffic participant is any one of the N types of traffic participants.

[0102] The second feature information of the first target class traffic participant is decoded by the multilayer perceptron to obtain the predicted trajectory information of the first target class traffic participant.

[0103] like Figure 2 As shown, multiple feature data in the first trajectory information of the first target class traffic participants are weighted based on the attention mechanism to distinguish the importance of the hidden state vectors corresponding to different times in the historical time step, so as to make the accuracy of the final output predicted trajectory information higher.

[0104] For example, at time t, a weighted statistical analysis is performed on the different hidden state vectors of several gated recurrent units:

[0105] d I =w 1t h 1t +w 2t h 2t +…+w ot h ot

[0106] Among them, w ot This can be understood as the weight value of the o-th gated recurrent unit at time t.

[0107] The aforementioned weight values ​​can be set manually by the user or obtained through machine learning (i.e., pre-configure the corresponding initial weight for each moment within the historical time step, and then iteratively adjust the initial weight corresponding to each moment within the historical time step through sample training until the number of adjustments reaches the threshold or the loss function is less than the loss threshold, then the training ends and the final weight corresponding to each moment within the historical time step is obtained). This application embodiment does not limit this.

[0108] In some implementations, obtaining the trajectory features of the N types of traffic participants based on the first trajectory information of the N types of traffic participants includes:

[0109] The first trajectory information of the N types of traffic participants is standardized to obtain the second trajectory information of the N types of traffic participants;

[0110] Multiple feature extraction processes are performed on the second trajectory information of the N types of traffic participants to obtain multiple feature output information; wherein, the feature extraction includes sequentially performed graphic feature processing and temporal convolution processing;

[0111] By performing skip connections on the multiple feature output information, the trajectory features of the N types of traffic participants can be obtained.

[0112] For example, the feature extraction model can use a two-dimensional (1×1) kernel-sized convolutional layer to convolve the preprocessed data (i.e., the first trajectory information of the N types of traffic participants) to complete the aforementioned standardization process (referring to increasing the dimensionality of the preprocessed data).

[0113] Subsequently, the preprocessed data after dimensionality upgrade (i.e., the second trajectory information of the N types of traffic participants) is subjected to graphical feature processing (to capture the interrelationships between multiple traffic participants) and temporal convolution processing (to capture the features of traffic participants at the temporal level) to obtain multiple feature output information.

[0114] Finally, the multiple feature output information is summarized using a skip connection method to generate trajectory features for N types of traffic participants. This reduces the probability of losing the spatiotemporal features of traffic participants and further improves the accuracy of the predicted trajectory information of the N types of traffic participants obtained subsequently.

[0115] In some implementations, the second target class of traffic participants is any one of the N classes of traffic participants;

[0116] The historical trajectory information set of the second target traffic participant includes historical trajectory sub-information, which includes the heading angle characteristics, temporal characteristics, and spatial characteristics of the target traffic participant. The second target traffic participant includes the target traffic participant.

[0117] The target traffic participant can be understood as any one of the traffic participants in the second target class. As mentioned above, after obtaining the trajectory direction scaling factor corresponding to the second target class traffic participant, all traffic participants belonging to the second target class traffic participants will share the trajectory direction scaling factor. For example, if the second target class traffic participants are set to include the first traffic participant and the second traffic participant, and the trajectory direction scaling factor corresponding to the second target class traffic participant is 0.3, then the trajectory direction scaling factor of the first traffic participant is 0.3, and the trajectory direction scaling factor of the second traffic participant is also 0.3.

[0118] It should be noted that the aforementioned acquisition of the first trajectory information of the N types of traffic participants based on their trajectory directions, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets can be understood as follows:

[0119] For any traffic participant among all traffic participants appearing in the last frame of the historical time step T, generate the first sub-trajectory information of the traffic participant based on the trajectory direction scaling factor, time features and spatial features corresponding to the traffic participant;

[0120] By summarizing the first sub-trajectory information of all the aforementioned traffic participants, the first trajectory information of the N types of traffic participants is obtained.

[0121] The aforementioned acquisition of the trajectory features of the N types of traffic participants based on their first trajectory information can be understood as follows:

[0122] For any traffic participant among all traffic participants appearing in the last frame of the historical time step T, feature extraction is performed on the first sub-trajectory information of the traffic participant based on the aforementioned feature extraction model to obtain the sub-trajectory features of the traffic participant.

[0123] By summarizing the sub-trajectory features of all the aforementioned traffic participants, the trajectory features of the N types of traffic participants are obtained.

[0124] The aforementioned method of obtaining the predicted trajectory information of the N types of traffic participants based on their trajectory characteristics can be understood as follows:

[0125] For any traffic participant among all traffic participants appearing in the last frame of the historical time step T, the sub-trajectory features of that traffic participant are sequentially encoded and decoded to obtain the predicted sub-trajectory information of that traffic participant.

[0126] By summarizing the predicted sub-trajectory information of all the aforementioned traffic participants, the predicted trajectory information of the N types of traffic participants can be obtained.

[0127] Preferably, the aforementioned encoding and decoding processes are performed using a trajectory prediction model. For example, the training process for the feature extraction model and the trajectory prediction model can be as follows:

[0128] Obtain a first initial model for feature extraction and a second initial model for trajectory prediction;

[0129] The initial sample trajectory information is standardized to obtain standard sample trajectory information;

[0130] The standard sample trajectory information is input into the first initial model, and the output data of the first initial model is sent into the second initial model to obtain the sample predicted trajectory;

[0131] Based on the difference between the actual trajectory of the sample and the initial sample trajectory information, the first initial model and the second initial model are adjusted to obtain the feature extraction model and the trajectory prediction model.

[0132] It should be noted that the standardization process in the above example can be understood as trajectory direction filtering (i.e., obtaining the trajectory direction scaling factor for each traffic participant in the initial sample trajectory information) and spatiotemporal feature processing (i.e., constructing the time map and spatial map for each traffic participant in the initial sample trajectory information). Specifically, to unify the data format of the trajectory matrix generated based on the trajectory direction scaling factor, time map, and spatial map, the trajectory information H1,…,H from the aforementioned multiple scenarios can be determined. m H is not part of the target scene trajectory information a Any scene trajectory information H b Mapping to target scene trajectory information H a The mapping process can be:

[0133] Using target scene trajectory information H a Based on this, a high-dimensional Hilbert kernel space is constructed, and the scene trajectory information H is incorporated. b Mapped onto the Hilbert kernel space, and the target scene trajectory information H is calculated. a and scene trajectory information H b The similarity value γ between ab :

[0134]

[0135] Among them, h a For the target scene trajectory information H a subvectors, h a For the target scene trajectory information H a The number of subvectors in h b For scene trajectory information H b subvectors, h b For scene trajectory information H b The number of subvectors in the vector.

[0136] Summarize trajectory information from multiple scenes (referring to trajectory information H1, ..., H that does not belong to multiple scenes) m H is not part of the target scene trajectory information a Multiple scene trajectory information) are compared using similarity value γ ab Generate a relevant factor matrix R. At this point, the aforementioned trajectory direction filtering process can be adjusted to determine the scene source corresponding to the initial sample trajectory information, and search for similar values ​​matching the scene source in the relevant factor matrix R based on the scene source. After obtaining the time map and spatial map of the initial sample trajectory information, standard sample trajectory information is quickly generated based on the obtained time map, spatial map and the similar value.

[0137] By setting the above parameters, the impact of differences in training data distribution caused by traffic scenario factors can be reduced (during the training phase of the prediction model), thereby improving the generalization ability of the prediction model and enabling it to achieve better prediction results.

[0138] For ease of understanding, the explanation is as follows:

[0139] Graph Neural Networks (GCNs) function similarly to Convolutional Neural Networks (CNNs) as feature extractors, but their target data is graph data. GCNs ingeniously design a method for extracting features from graph data, allowing us to use these features to perform tasks such as node classification, graph classification, and link prediction.

[0140] Recurrent Neural Networks (RNNs) are a type of recurrent neural network that takes sequential data as input, recursively propagates the data in the direction of the sequence, and all nodes (recurrent units) are connected in a chain-like manner. A Gated Recurrent Unit (GRU) is a type of RNN. Like LSTM (Long-Short Term Memory), it was proposed to address problems related to long-term memory and gradients during backpropagation. Compared to LSTM, GRU has one less gate and fewer parameters, yet it achieves comparable functionality. GRU offers advantages over LSTM in terms of lower hardware computation requirements and shorter computation time.

[0141] Multilayer Perceptron (MLP): Also known as an Artificial Neural Network (ANN), a multilayer perceptron can have multiple hidden layers in addition to the input and output layers. The simplest MLP contains only one hidden layer, resulting in a three-layer structure. All layers in a multilayer perceptron are fully connected. The bottom layer is the input layer, followed by hidden layers, and finally the output layer.

[0142] Attention Mechanism: The attention mechanism is a data processing method in machine learning, widely used in various types of machine learning tasks such as language processing, image recognition, and speech recognition. The idea behind the attention mechanism is similar to human text translation, where attention is focused on the context corresponding to each word in the source sentence. Similarly, in an attention mechanism model, when translating the current word, the model searches for several corresponding words in the source sentence and combines this with previously translated parts to make the appropriate translation. In this way, the decoder can see all the relevant information of the words encoded by the encoder during prediction, rather than being limited to the fixed-length vector in the original model, thus reducing long-range information loss to some extent.

[0143] The various optional implementation methods described in the embodiments of this application can be combined with each other or implemented individually without conflict. The embodiments of this application do not limit this.

[0144] See Figure 3 , Figure 3 This is a schematic diagram of the traffic participant trajectory prediction device provided in the embodiments of this application, as shown below. Figure 3 As shown, the traffic participant trajectory prediction device 200 includes:

[0145] The acquisition module 201 is used to acquire N historical trajectory information sets, each of which corresponds to a type of traffic participant, and each of the historical trajectory information sets includes the heading angle features, time features and spatial features of the corresponding type of traffic participant, where N is an integer greater than 1.

[0146] The direction confirmation module 202 is used to obtain the trajectory direction of N types of traffic participants based on the heading angle features of N historical trajectory information sets.

[0147] Data processing module 203 is used to obtain the first trajectory information of the N types of traffic participants based on the trajectory direction of the N types of traffic participants, the time characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets;

[0148] The feature extraction module 204 is used to obtain the trajectory features of the N types of traffic participants based on the first trajectory information of the N types of traffic participants;

[0149] The prediction module 205 is used to obtain the predicted trajectory information of the N types of traffic participants based on their trajectory characteristics.

[0150] In some embodiments, the data processing module 203 includes:

[0151] The target scene determination unit is used to acquire multiple scene trajectory information, wherein any two scene trajectory information correspond to different traffic scenes; and determine target scene trajectory information based on the multiple scene trajectory information, wherein the target scene trajectory information is the scene trajectory information that includes the most traffic participants and / or the most traffic participant categories among the multiple scene trajectory information.

[0152] The mapping unit is used to map the trajectory direction of the N types of traffic participants, the temporal features of the N historical trajectory information sets, and the spatial features of the N historical trajectory information sets to the target scene trajectory information, so as to obtain the first trajectory information of the N types of traffic participants.

[0153] In some embodiments, the mapping unit includes:

[0154] Based on the trajectory directions of the N types of traffic participants, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets, an undirected graph network of the N types of traffic participants is generated.

[0155] The undirected graph network is mapped onto the target scene trajectory information to obtain the first trajectory information of the N types of traffic participants.

[0156] In some embodiments, the prediction module 205 includes:

[0157] The encoding unit is used to encode the trajectory features of the N types of traffic participants according to the recurrent neural network to obtain the first feature information of the N types of traffic participants;

[0158] The decoding unit is used to decode the first feature information of the N types of traffic participants based on the multilayer perceptron to obtain the predicted trajectory information of the N types of traffic participants.

[0159] In some implementations, the first trajectory information includes multiple feature data, wherein any two feature data points correspond to different time nodes, and the decoding unit includes:

[0160] Based on the attention mechanism, multiple feature data in the first trajectory information of the first target class traffic participant are weighted and processed to obtain the second feature information of the first target class traffic participant. The first target class traffic participant is any one of the N types of traffic participants.

[0161] The second feature information of the first target class traffic participant is decoded by the multilayer perceptron to obtain the predicted trajectory information of the first target class traffic participant.

[0162] In some embodiments, the feature extraction module 204 includes:

[0163] The first trajectory information of the N types of traffic participants is standardized to obtain the second trajectory information of the N types of traffic participants;

[0164] Multiple feature extraction processes are performed on the second trajectory information of the N types of traffic participants to obtain multiple feature output information; wherein, the feature extraction includes sequentially performed graphic feature processing and temporal convolution processing;

[0165] By performing skip connections on the multiple feature output information, the trajectory features of the N types of traffic participants can be obtained.

[0166] In some implementations, the second target class of traffic participants is any one of the N classes of traffic participants;

[0167] The historical trajectory information set of the second target traffic participant includes historical trajectory sub-information, which includes the heading angle characteristics, temporal characteristics, and spatial characteristics of the target traffic participant. The second target traffic participant includes the target traffic participant.

[0168] This application also provides a communication device; please refer to [link to relevant documentation]. Figure 4 The communication device may include a processor 301, a memory 302, and a program 3021 stored in the memory 302 and capable of running on the processor 301.

[0169] When program 3021 is executed by processor 301, it can achieve the following: Figure 1 Any steps in the corresponding method embodiments and the achievement of the same beneficial effects will not be repeated here.

[0170] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by hardware related to program instructions, and the program can be stored in a readable medium.

[0171] This application embodiment also provides a readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described functions. Figure 1 Any step in the corresponding method embodiment can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0172] The computer-readable storage medium of this application embodiment can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0173] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0174] The program code contained on the storage medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0175] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or terminal. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

Claims

1. A method for predicting the trajectory of traffic participants, characterized in that, The method includes: Obtain N historical trajectory information sets, each of which corresponds to a type of traffic participant, and each of the historical trajectory information sets includes the heading angle features, time features, and spatial features of the corresponding type of traffic participant, where N is an integer greater than 1; Based on the heading angle features of N historical trajectory information sets, the trajectory directions of N types of traffic participants are obtained; Based on the trajectory directions of the N types of traffic participants, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets, the first trajectory information of the N types of traffic participants is obtained; Based on the first trajectory information of the N types of traffic participants, the trajectory characteristics of the N types of traffic participants are obtained; Based on the trajectory characteristics of the N types of traffic participants, the predicted trajectory information of the N types of traffic participants is obtained; The step of obtaining the first trajectory information of the N types of traffic participants based on their trajectory directions, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets includes: Acquire multiple scene trajectory information, wherein any two scene trajectory information pieces correspond to different traffic scenes; The target scene trajectory information is determined based on the multiple scene trajectory information, wherein the target scene trajectory information is the scene trajectory information that includes the largest number of traffic participants and / or the largest number of traffic participant categories among the multiple scene trajectory information; The trajectory directions of the N types of traffic participants, the temporal features of the N historical trajectory information sets, and the spatial features of the N historical trajectory information sets are mapped to the target scene trajectory information to obtain the first trajectory information of the N types of traffic participants.

2. The method according to claim 1, characterized in that, The step of mapping the trajectory directions of the N types of traffic participants, the temporal features of the N historical trajectory information sets, and the spatial features of the N historical trajectory information sets to the target scene trajectory information to obtain the first trajectory information of the N types of traffic participants includes: Based on the trajectory directions of the N types of traffic participants, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets, an undirected graph network of the N types of traffic participants is generated. The undirected graph network is mapped onto the target scene trajectory information to obtain the first trajectory information of the N types of traffic participants.

3. The method according to claim 1, characterized in that, The step of obtaining the predicted trajectory information of the N types of traffic participants based on their trajectory characteristics includes: The trajectory features of the N types of traffic participants are encoded using a recurrent neural network to obtain the first feature information of the N types of traffic participants; The predicted trajectory information of the N types of traffic participants is obtained by decoding the first feature information of the N types of traffic participants using a multilayer perceptron.

4. The method according to claim 3, characterized in that, The first trajectory information includes multiple feature data, and any two feature data points correspond to different time nodes; The step of decoding the first feature information of the N types of traffic participants based on the multilayer perceptron to obtain the predicted trajectory information of the N types of traffic participants includes: Based on the attention mechanism, multiple feature data in the first trajectory information of the first target class traffic participant are weighted and processed to obtain the second feature information of the first target class traffic participant. The first target class traffic participant is any one of the N types of traffic participants. The second feature information of the first target class traffic participant is decoded by the multilayer perceptron to obtain the predicted trajectory information of the first target class traffic participant.

5. The method according to claim 3, characterized in that, The step of obtaining the trajectory features of the N types of traffic participants based on the first trajectory information of the N types of traffic participants includes: The first trajectory information of the N types of traffic participants is standardized to obtain the second trajectory information of the N types of traffic participants; Multiple feature extraction processes are performed on the second trajectory information of the N types of traffic participants to obtain multiple feature output information; wherein, the feature extraction includes sequentially performed graphic feature processing and temporal convolution processing; By performing skip connections on the multiple feature output information, the trajectory features of the N types of traffic participants can be obtained.

6. The method according to claim 1, characterized in that, The second target category of traffic participants is any one of the N categories of traffic participants; The historical trajectory information set of the second target traffic participant includes historical trajectory sub-information, which includes the heading angle characteristics, temporal characteristics, and spatial characteristics of the target traffic participant. The second target traffic participant includes the target traffic participant.

7. A trajectory prediction device for traffic participants, characterized in that, The device includes: The acquisition module is used to acquire N historical trajectory information sets, each of which corresponds to a type of traffic participant, and each of the historical trajectory information sets includes the heading angle features, time features and spatial features of the corresponding type of traffic participant, where N is an integer greater than 1. The direction confirmation module is used to obtain the trajectory direction of N types of traffic participants based on the heading angle features of N historical trajectory information sets. The data processing module is used to obtain the first trajectory information of the N types of traffic participants based on the trajectory direction of the N types of traffic participants, the temporal characteristics of the N historical trajectory information sets, and the spatial characteristics of the N historical trajectory information sets. The feature extraction module is used to obtain the trajectory features of the N types of traffic participants based on the first trajectory information of the N types of traffic participants; The prediction module is used to obtain the predicted trajectory information of the N types of traffic participants based on their trajectory characteristics. The data processing module includes: The target scene determination unit is used to acquire multiple scene trajectory information, wherein any two scene trajectory information correspond to different traffic scenes; and determine target scene trajectory information based on the multiple scene trajectory information, wherein the target scene trajectory information is the scene trajectory information that includes the most traffic participants and / or the most traffic participant categories among the multiple scene trajectory information. The mapping unit is used to map the trajectory direction of the N types of traffic participants, the temporal features of the N historical trajectory information sets, and the spatial features of the N historical trajectory information sets to the target scene trajectory information, so as to obtain the first trajectory information of the N types of traffic participants.

8. A communication device, comprising: A transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is configured to read the program in the memory to implement the steps in the method for predicting the trajectory of a traffic participant as claimed in any one of claims 1 to 6.

9. A readable storage medium for storing a program, characterized in that, When the program is executed by the processor, it implements the steps in the method for predicting the trajectory of traffic participants as described in any one of claims 1 to 6.

Images