Obstacle trajectory prediction method and related device

Abstract

The invention provides an obstacle trajectory prediction method and a related device, and relates to the technical field of artificial intelligence, in particular to the technical field of automatic driving, intelligent driving and trajectory prediction. According to the specific implementation scheme, historical behavior characteristics of traffic participants and topological characteristics of road topology of a road section where a target obstacle is located are coded, and scene coding characteristics are obtained; the traffic participants at least comprise a target obstacle; determining a potential moving path of the target obstacle based on the scene coding features and first probability distribution predicted by a path anchor point set on the road topology; and on the basis of a candidate anchor point set on the potential moving path and second probability distribution predicted by the scene coding features, determining at least one moving track point of the target obstacle on the potential moving path in a future specified duration. In the embodiment of the invention, the trajectory prediction is progressively decoupled into the potential moving path and the moving trajectory point, so that the trajectory of the obstacle can be effectively predicted.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the field of autonomous driving, intelligent driving, and trajectory prediction technology. Background Art

[0002] With the rapid development of science and technology and the advent of the era of automobile intelligence, autonomous vehicles will occupy an extremely important position in the future intelligent transportation system. Autonomous vehicles need to predict the movement trajectory of surrounding obstacles in order to plan a collision-free path for the autonomous vehicle in real time to ensure the safety and stable operation of the autonomous vehicle.

[0003] Therefore, how to predict the movement trajectory of obstacles is a problem worth studying. Summary of the invention

[0004] The present disclosure provides a method for predicting the trajectory of an obstacle and a related device.

[0005] According to one aspect of the present disclosure, a method for predicting a trajectory of an obstacle is provided, comprising:

[0006] The historical behavior characteristics of the traffic participants and the topological characteristics of the road topology of the road section where the target obstacle is located are encoded to obtain scene coding characteristics; the traffic participants at least include the target obstacle; wherein the historical behavior characteristics of the traffic participants include the action trajectory and / or motion state within a specified time period in the past;

[0007] Determine a potential moving path of the target obstacle based on a first probability distribution predicted by the scene coding features and the set of path anchor points on the road topology; wherein the first probability distribution is used to describe the probability of the target obstacle moving on each candidate path of the road topology;

[0008] Based on the set of candidate anchor points on the potential moving path and the second probability distribution predicted by the scene coding features, at least one moving trajectory point of the target obstacle on the potential moving path within a specified time period in the future is determined; the second probability distribution is used to describe the probability of the target obstacle moving to each candidate anchor point in the candidate anchor point set within the specified time period in the future.

[0009] According to another aspect of the present disclosure, there is provided a device for predicting a trajectory of an obstacle, comprising:

[0010] A coding module is used to encode the historical behavior characteristics of traffic participants and the topological characteristics of the road topology of the road section where the target obstacle is located to obtain a scene coding feature; the traffic participants at least include the target obstacle; wherein the historical behavior characteristics of the traffic participants include the action trajectory and / or motion state within a specified time period in the past;

[0011] A first prediction module is used to determine a potential moving path of a target obstacle based on a first probability distribution predicted by a scene coding feature and a set of path anchor points on a road topology; wherein the first probability distribution is used to describe the probability of the target obstacle moving on each candidate path of the road topology;

[0012] The second prediction module is used to determine at least one moving trajectory point of the target obstacle on the potential moving path within a specified time period in the future based on the set of candidate anchor points on the potential moving path and the second probability distribution predicted by the scene coding features; the second probability distribution is used to describe the probability of the target obstacle moving to each candidate anchor point in the candidate anchor point set within the specified time period in the future.

[0013] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0017] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0018] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0019] According to another aspect of the present disclosure, a vehicle is provided, comprising the aforementioned electronic device.

[0020] The disclosed embodiment predicts the potential moving path before predicting the trajectory of the obstacle. Since the prediction of the trajectory points within the future specified time period is subject to the potential moving path as a constraint, the prediction of the short-term trajectory points can make the path more realistic, thereby improving the accuracy of the target obstacle trajectory prediction.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0023] Figure 1 is a flow chart of an obstacle trajectory prediction method according to an embodiment of the present disclosure;

[0024] Figure 2 is a schematic diagram of a process for determining scene coding features according to an embodiment of the present disclosure;

[0025] Figure 3 is a schematic diagram of the structure of a neural network for interactive learning provided in one embodiment of the present disclosure;

[0026] Figure 4 is a schematic diagram of a structure for determining scene coding features according to an embodiment of the present disclosure;

[0027] Figure 5 is a schematic diagram of determining a path anchor point set according to an embodiment of the present disclosure;

[0028] Figure 6 is a schematic diagram of a structure for determining a potential moving path and at least one moving trajectory point according to an embodiment of the present disclosure;

[0029] Figure 7 is a structural schematic diagram of an obstacle trajectory prediction method according to an embodiment of the present disclosure;

[0030] Figure 8 is a schematic structural diagram of an obstacle trajectory prediction device according to another embodiment of the present disclosure;

[0031] Fig. 9 The block diagram of an electronic device used to implement the obstacle trajectory prediction method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0033] In addition, the terms "first", "second" and the like are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the term "first", "second" and the like may explicitly or implicitly include one or more of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0034] In order to predict the movement trajectory of obstacles and realize the interaction planning between the autonomous driving vehicle and obstacles, the present disclosure provides a method for predicting the trajectory of obstacles. Figure 1 As shown, it is a flow chart of the method, which includes the following contents:

[0035] S101, encode the historical behavior characteristics of the traffic participants and the topological characteristics of the road topology of the section where the target obstacle is located to obtain scene coding characteristics; the traffic participants at least include the target obstacle; wherein the historical behavior characteristics of the traffic participants include the movement trajectory and / or movement state within a specified time period in the past.

[0036] Among them, the target obstacles can be movable objects around the autonomous driving vehicle, such as motor vehicles, non-motor vehicles, pedestrians, etc.

[0037] The traffic participants may include a target obstacle, and the obstacles around the target obstacle.

[0038] The historical behavior characteristics of traffic participants include the action trajectory and / or motion state within a specified time period in the past. The motion state refers to the state of the traffic participant relative to a certain reference system when in motion. The motion state may include at least one of the multiple states such as static, uniform speed, acceleration, deceleration, linear motion, and curved motion.

[0039] In a possible implementation, the historical behavior feature may be composed of the state features of the past t time steps, and the state features of each time step include at least one of the position information, the velocity statistics information, and the acceleration information. Among them, for each time step, the position information and acceleration information of the time step refer to the instantaneous position and acceleration of the time step, and the velocity statistics information of the time step may be the average speed within a period of time based on the time step, or the maximum speed or minimum speed within the period of time. During implementation, the corresponding state features may be selected according to the actual situation.

[0040] During implementation, the road topology can be determined on the map based on the road section where the target obstacle is located. The road topology contains multiple paths that the target obstacle can travel from the current road section. The road topology is described by topological features. Topological features are used to describe each path, and each path can be composed of multiple path points. For example, the road topology of the target obstacle includes 32 paths. In order to avoid the path being too long, each path can be limited to a maximum of 128 road sections, and multiple path points can be obtained by uniformly sampling on the path. Therefore, the topological features of the road topology are composed of the path points of each candidate path in the road topology.

[0041] In the disclosed embodiment, based on the state features and topological features, a vectorized representation method is used to describe the historical behavior features of traffic participants and the road topology. In this way, the historical behavior features of traffic participants and the road topology can be uniformly abstracted into features with semantic labels and attribute information such as the current state. The disclosed embodiment does not limit the vectorized representation method.

[0042] During the implementation process, if the number of candidate paths that can be traveled on the current section of the target obstacle does not reach the first specified number, masks can be used to fill in the gaps. For example, the first specified number is 32, and these 32 candidate paths can be represented by numbers 1-32 respectively. If there are only 30 paths in the road topology of the target obstacle, and therefore the required 32 paths are not reached, 0 can be used to fill in the remaining two candidate paths. For each path point on the candidate path, if the path points of the candidate path do not reach the second specified number, the remaining path points on the path are filled in with 0. For example, the second specified number is 128, and there are 100 path points on candidate path 1, then the remaining 28 path points are filled in with 0.

[0043] Similarly, when the historical behavior characteristics of traffic participants do not reach the corresponding upper limit, this method is also used to fill the gap.

[0044] S102, determining a potential moving path of the target obstacle based on a first probability distribution predicted by the scene coding features and a set of path anchor points on the road topology; wherein the first probability distribution is used to describe the probability of the target obstacle moving on each candidate path of the road topology.

[0045] The embodiment of the present disclosure samples candidate paths based on the road topology where the target obstacle is located to obtain a path anchor point set. The embodiment of the present disclosure does not limit the sampling method, such as uniform sampling or non-uniform sampling.

[0046] S103, based on the set of candidate anchor points on the potential moving path and the second probability distribution predicted by the scene coding features, determine at least one moving trajectory point of the target obstacle on the potential moving path within a specified time period in the future; the second probability distribution is used to describe the probability of the target obstacle moving to each candidate anchor point in the candidate anchor point set within the specified time period in the future.

[0047] It can be understood that in order to predict the future trajectory of the target obstacle, in addition to predicting the potential moving path, it is also necessary to predict the moving trajectory points of the target obstacle in a short period of time, so as to combine the potential moving path and the moving trajectory points to obtain the future trajectory prediction result of the target obstacle.

[0048] In the disclosed embodiment, the potential moving path can represent the long-term path of the target obstacle. After obtaining the potential moving path, the candidate anchor point set is further obtained to obtain the short-term moving trajectory points of the target obstacle. The short-term duration is represented by the future specified duration, and its specific value is flexibly determined according to the actual situation. By predicting the long-term path and short-term trajectory points of the target obstacle, the future moving trajectory of the target obstacle can be smoother and more realistic.

[0049] It should be noted that the embodiments of the present disclosure do not limit the method of predicting the first probability distribution and the second probability distribution. The possible implementation methods of determining the first probability distribution and the second probability distribution will be described in detail below.

[0050] In summary, the embodiments of the present disclosure progressively decouple trajectory prediction into potential movement paths and movement trajectory points.

[0051] Due to the randomness and subjectivity of surrounding obstacles in the spatiotemporal coupling space, it is necessary to predict the motion trajectory of obstacles around the autonomous driving vehicle. The potential moving path prediction is carried out with reference to the road topology around the target obstacle, so it can effectively alleviate the uncertainty of future trajectory prediction in the spatiotemporal coupling space. Moreover, the scene coding features can well describe the spatiotemporal characteristics around the target obstacle, and under the dual constraints of the scene coding features and the road topology, the potential moving path of the target obstacle can be accurately predicted. In addition, in the disclosed embodiment, the prediction of trajectory points within the future specified time length is based on the potential moving path as a constraint condition. Therefore, the prediction of short-term trajectory points can make the path more realistic, thereby improving the accuracy of the target obstacle trajectory prediction. Based on this advantage, the downstream planning module can realize the interactive planning between the target obstacle and the autonomous driving vehicle by detecting whether the potential moving path of the target obstacle overlaps with the planned path of the autonomous driving vehicle.

[0052] For ease of understanding, the following describes in detail how to determine scene coding features, how to predict a potential moving path, and how to predict short-term trajectory points on the potential moving path.

[0053] 1) Determine scene coding features

[0054] In the disclosed embodiment, the scene coding feature is used to describe the spatiotemporal characteristics of the target obstacle. Based on this, the historical behavior characteristics of the traffic participants and the topological characteristics of the road topology of the section where the target obstacle is located can be input into the spatiotemporal coding network to obtain the scene coding feature.

[0055] Through the spatiotemporal coding network, scene coding features can be learned autonomously and high-level feature expression capabilities can be improved, making the extracted scene coding features more conducive to trajectory prediction.

[0056] In some possible implementations, the spatiotemporal coding network can be an encoder of a wayformer (path self-attention model). That is, the historical behavior characteristics and topological characteristics of traffic participants are input into the encoder to obtain scene coding features. The encoder can be implemented as a transform (self-attention model).

[0057] In some other possible implementations, the following operations may be performed based on the spatiotemporal coding network to determine the scene coding features, which may be implemented as follows: Figure 2 The method shown:

[0058] S201, spatially encode the historical behavior characteristics and topological characteristics of traffic participants to obtain spatial encoding features.

[0059] The future trajectory (i.e., potential moving path) of the target obstacle is related to its historical behavior characteristics and the road topology structure in which it is located. Therefore, the spatial coding features extracted based on the historical behavior characteristics and topological features of traffic participants in the embodiment of the present disclosure can comprehensively describe the historical behavior laws of the target obstacle.

[0060] In a possible implementation manner, S201 may be implemented as follows:

[0061] Step A1: extract historical behavior features of traffic participants using a first feature extraction method to obtain a first spatial feature.

[0062] As described above, when the traffic participant is a target obstacle, the historical behavior characteristics of the traffic participant may be the historical behavior characteristics of the target obstacle. When the traffic participant includes the target obstacle and obstacles around the target obstacle (hereinafter referred to as interactive obstacles), the historical behavior characteristics of the traffic participant include the historical behavior characteristics of the target obstacle and the historical behavior characteristics of the interactive obstacle.

[0063] The first spatial feature can be understood as the dynamic environmental feature of the target obstacle. In the case where traffic participants include interactive obstacles, the first spatial feature not only considers the dynamic changes of the environment during the movement of the target obstacle itself, but also considers the impact of the interactive obstacles on the target obstacle. The first spatial feature obtained in this way can more comprehensively describe the historical behavior law of the target obstacle.

[0064] In some embodiments, a separate spatial coding network is designed to obtain high-level spatial coding features of target obstacles in order to obtain spatial coding features. In the spatial coding network, the first feature extraction method can use an LSTM network (Long Short-Term Memory), which can extract the first spatial feature from the historical behavior characteristics of all traffic participants to reflect the movement law of the target obstacle.

[0065] In other embodiments, the movement trajectory of the target obstacle may also be affected by changes in factors such as moving vehicles, moving entities, and pedestrians in the environment. Therefore, in the spatial encoding network, the historical behavior features of the interactive obstacles and the target obstacles may be pooled to obtain a pooled vector, and then the pooled vector may be sent to an MLP (Multi-Layer Perceptron) to extract the first spatial feature.

[0066] Step A2: extract the topological features to obtain the second spatial features.

[0067] Among them, the second spatial feature can be understood as the static environmental feature of the road topology.

[0068] In addition to the dynamic environment, the static environment also affects the motion trajectory of the target obstacle, so it is necessary to extract the topological features. Among them, static features are features that do not change over time, such as multiple candidate paths. But it is not limited to this, it can also be features of other immovable objects related to the target obstacle, such as crosswalks, stop lines, etc.

[0069] During implementation, a CNN network (Convolutional Neutal Network) can be used to extract topological features. First, the high-precision map provides rich map road network information to obtain the road topology to construct vectorized topological features, and then the CNN network is used to extract static high-level features from the road topological features as the second spatial features.

[0070] During implementation, a convolution calculation may be performed on the input historical behavior feature to obtain a first spatial feature, or a convolution calculation may be performed on the input topological feature to obtain a second spatial feature. Among them, one or more convolution layers may be used to perform the convolution calculation. When multiple convolution layers are used, the multiple convolution layers may be in parallel or in series in structure.

[0071] It should be noted that during the execution process, the execution order of step A1 and step A2 is not limited.

[0072] Step A3: interactively learn the first spatial feature and the second spatial feature to obtain a spatial coding feature.

[0073] The interactive learning of the first spatial feature and the second spatial feature is to integrate the dynamic environmental features and static features of the target obstacle so as to comprehensively describe the movement law of the target obstacle.

[0074] In some embodiments, Figure 3 As shown, the trained MCG neural network (Modular Complementary Gradient) can be used to interactively learn the input first spatial feature and the second spatial feature to obtain the first sub-coding feature of the first spatial feature and the second sub-coding feature of the second spatial feature. The first sub-coding feature and the second sub-coding feature are regarded as spatial coding features. Figure 3 As shown, the MCG neural network model includes two inputs and two outputs.

[0075] In other embodiments, an attention mechanism can also be used for interactive learning. The attention mechanism can find the correlation between features, and continuously update the attention weights through training, so that the model can focus on the key parts, so as to achieve better results. For example, a multi-head attention layer or a cross-attention mechanism can be used to interactively learn the first spatial feature and the second spatial feature. Among them, the multi-head attention layer and the cross-attention mechanism both have three inputs and usually one output. For example, the first spatial feature can be used as the input Q (Query) and K (Key), and the second spatial feature can be used as the input V (Value), and the output is the spatial coding feature.

[0076] In the disclosed embodiment, the first spatial feature can be represented as a high-level historical behavior feature of a traffic participant, and the second spatial feature can be represented as a high-level feature of a topological feature. Compared with the original historical behavior feature and topological feature, the first spatial feature and the second spatial feature can more accurately describe the historical behavior patterns of traffic participants and the road topology to improve the feature expression capability of the spatial coding feature. In addition, interactive learning of the first spatial feature and the second spatial feature can enable the spatial coding feature to accurately represent the historical behavior patterns of the target obstacle on a given road topology over a period of time in the past, which helps to improve the accuracy of the target obstacle trajectory prediction.

[0077] The following is a detailed description of the method for obtaining the first spatial feature and the second spatial feature, which can be executed as follows:

[0078] 1) First spatial feature

[0079] When the traffic participant is a target obstacle, feature extraction may be performed on the historical behavior features of the target obstacle to obtain a first spatial feature.

[0080] When the traffic participants include target obstacles and interactive obstacles, obtaining the first spatial feature may be implemented as follows: Steps B1 to B3:

[0081] Step B1: extract the historical behavior characteristics of the target obstacle to obtain the first subspace characteristics.

[0082] As described above, the historical behavior characteristics of the target obstacle may include at least one of the position information, velocity statistics information, and acceleration information of the target obstacle. In the embodiment of the present disclosure, taking the above three types of information as an example, feature extraction is performed on at least one of the position information, velocity statistics information, and acceleration information of the target obstacle to obtain the first subspace feature of the target obstacle.

[0083] In some embodiments, a deep cross neural network can be used to extract the historical behavior characteristics of the target obstacle. The deep cross network can cross different features at multiple levels, capture the nonlinear relationship between each feature, and better learn the interaction between features. For example, the position information, velocity statistics information, and acceleration information of the target obstacle are input into the deep cross neural network for interactive learning, and the dependency relationship between the above three feature information can be extracted, thereby obtaining the first subspace feature.

[0084] In addition to using a deep cross neural network, in the embodiment of the present disclosure, a convolution module and a pooling operation may also be used to extract the first subspace feature from the historical behavior features of the target obstacle.

[0085] In some other possible embodiments, obtaining the first subspace feature may also be performed as follows:

[0086] Step C1: Perform convolution processing on the historical behavior characteristics of the target obstacle to obtain a first convolution processing result.

[0087] like Figure 4 As shown in FIG. 1 , the historical behavior characteristics of the target obstacle are input into the one-dimensional convolution layer to obtain the first convolution processing result of the target obstacle. The convolution operation can extract the characteristics of the target obstacle and reduce the dimension of the historical behavior characteristics of the target obstacle for subsequent processing.

[0088] Step C2: performing a pooling operation on the first convolution processing result to obtain a first subspace feature.

[0089] like Figure 4As shown in FIG. 1 , the historical behavior characteristics of the target obstacle are input into the one-dimensional convolution layer, and then a pooling operation is performed to obtain the first subspace characteristics.

[0090] In some embodiments, the pooling operation on the first convolution processing result may adopt maximum pooling, wherein in each pooling window, the maximum pooling operation selects the maximum value in the window as output, that is, outputs the most significant feature in the first convolution processing result, that is, the first subspace feature.

[0091] In some other embodiments, the pooling operation for the first convolution processing result may also be performed by average pooling. In each pooling window, average pooling calculates the average value of all feature values ​​in the window as output, that is, outputs the average feature in the first convolution processing result, that is, the first subspace feature. At the same time, average pooling is smoother and can retain more detail information.

[0092] In the disclosed embodiment, the convolution layer and the pooling layer are used in combination to form a deep network structure, the convolution layer is used to obtain the first convolution processing result of the target obstacle from the target obstacle, and the pooling layer is used to extract the main features in the first convolution processing result, that is, the first subspace feature. The alternating use of the convolution layer and the pooling operation not only reduces the dimension of the historical behavior characteristics of the target obstacle, but also retains important feature information, so as to improve the accuracy of trajectory prediction.

[0093] Step B2: extract the historical behavior features of the interactive obstacles to obtain the second subspace features.

[0094] As described above, the interactive obstacle may be a movable object around the target obstacle, such as a motor vehicle, a non-motor vehicle, a pedestrian, etc. The historical behavior characteristics of the interactive obstacle are similar to the historical behavior characteristics of the target obstacle, and may include at least one of the position information, velocity statistics information, and acceleration information of the interactive obstacle.

[0095] In the disclosed embodiment, the historical behavior characteristics of the interactive obstacle including the position information, velocity statistics information and acceleration information of the interactive obstacle are taken as an example for description, and feature extraction is performed on the position information, velocity statistics information and acceleration information of the interactive obstacle to obtain the second subspace characteristics of the position information, velocity statistics information and acceleration information of the interactive obstacle.

[0096] During implementation, convolution processing can be used to extract the historical features of the interactive obstacles, or a multi-layer perceptron can be used to perform pooling operations for feature extraction. The present disclosure does not limit the method of feature extraction.

[0097] It should be noted that during the execution process, the execution order of step B1 and step B2 is not limited.

[0098] In some other possible embodiments, obtaining the second subspace feature may be performed as follows:

[0099] Step D1: Perform convolution processing on the historical behavior characteristics of the interactive obstacle to obtain a second convolution processing result.

[0100] Step D2: performing a pooling operation on the second convolution processing result to obtain a second subspace feature.

[0101] like Figure 4 As shown, after the historical behavior features of the interactive obstacles are input into the one-dimensional convolutional layer, a pooling operation is performed to obtain the second subspace features, so as to reduce the dimension of the historical behavior features of the interactive obstacles and obtain high-level features with expressive ability.

[0102] In some embodiments, the pooling operation performed on the second convolution processing result may adopt maximum pooling or average pooling to obtain the second subspace feature.

[0103] In the disclosed embodiment, the convolution layer and the pooling layer are stacked to form a deep network structure. The convolution layer is used to obtain the second convolution processing result of the interactive obstacle from the historical behavior characteristics of the interactive obstacle, and the pooling layer is used to extract the main features in the second convolution processing result, that is, the second subspace features. The alternating use of the convolution layer and the pooling operation not only reduces the dimension of the historical behavior characteristics of the interactive obstacle, but also retains important feature information, thereby improving the accuracy of trajectory prediction.

[0104] Step B3: interactively learn the first subspace feature and the second subspace feature to obtain the first space feature.

[0105] In implementation, an MCG neural network may be used to interactively learn the first subspace feature and the second subspace feature, or an attention mechanism may be used to interactively learn the first subspace feature and the second subspace feature. These interactive learning methods have been described above, and the embodiments of the present disclosure will not be described one by one.

[0106] In the disclosed embodiment, the first subspace feature represents the high-level historical behavior feature of the target obstacle, and the second subspace feature represents the high-level historical behavior feature of the interactive obstacle. The first subspace feature and the second subspace feature are interactively learned to obtain the first spatial feature, which integrates the historical behavior feature of the target obstacle and the historical behavior feature of the interactive obstacle, and can represent the impact of the interactive obstacle on the target obstacle, thereby improving the accuracy of trajectory prediction.

[0107] 2) Second space characteristics

[0108] Obtaining the second spatial feature can be implemented as follows:

[0109] Step E1, performing convolution processing on the topological features to obtain a third convolution processing result.

[0110] like Figure 4 As shown, the topological feature is input into the one-dimensional convolution layer for convolution processing to obtain the third convolution processing result of the topological feature, and then a pooling operation is performed to obtain the second spatial feature.

[0111] Step E2: performing a pooling operation on the third convolution processing result to obtain a second spatial feature.

[0112] During implementation, a pooling operation is performed on the third convolution processing result, and average pooling or maximum pooling can be used.

[0113] In the disclosed embodiment, convolution and pooling operations are performed on the topological features, which reduces the dimension of the topological features and facilitates the extraction of significant features in the topological features. At the same time, the amount of calculation of subsequent operations is reduced to improve the efficiency of trajectory prediction.

[0114] S202, time-encoding the historical behavior characteristics of the traffic participants to obtain time-encoded characteristics.

[0115] The purpose of time coding is to extract the movement law of the target obstacle in the past multiple time steps. Among them, the historical behavior characteristics of the target obstacle can be time-coded to obtain the time coding characteristics of the target obstacle.

[0116] Because the historical movement law of the target obstacle is essentially affected by the interactive obstacle, in the embodiment of the present disclosure, the historical behavior characteristics of the target obstacle and the interactive obstacle can be time-coded to obtain the time-coded characteristics.

[0117] Regardless of whether the traffic participant is a target obstacle or includes a target obstacle and an interactive obstacle, in order to describe the time coding features of the time characteristics of the target obstacle from multiple aspects, S202 may be executed as follows:

[0118] Step F1: extract the historical behavior features of traffic participants using the second feature extraction method to obtain time features.

[0119] In some embodiments, in order to obtain time coding features, a separate time coding network can be designed to obtain high-level historical behavior features of traffic participants. In the time coding network, the second feature extraction method can use an LSTM network, which can extract time features from the historical behavior features of traffic participants to reflect the movement patterns of traffic participants.

[0120] In other embodiments, the change of the interactive obstacle may also affect the motion trajectory of the target obstacle. Therefore, in the time coding network, the historical behavior features of the traffic participants including the interactive obstacles may be pooled to extract the time features.

[0121] In the embodiment of the present disclosure, obtaining the time feature may be performed as follows:

[0122] Step G1: extract the historical behavior characteristics of the target obstacle to obtain the first sub-time feature.

[0123] As described above, the historical behavior characteristics of the target obstacle may include at least one of the position information, velocity statistics information, and acceleration information of the target obstacle. In the disclosed embodiment, the historical behavior characteristics of the target obstacle include the position information, velocity statistics information, and acceleration information of the target obstacle, and the temporal dependency relationship between the position information, velocity statistics information, and acceleration information of the target obstacle is feature extracted to obtain the first sub-time feature.

[0124] like Figure 4 As shown, the historical behavior features of the target obstacle are input into the first cross learning module, and the feature extraction operation is completed to obtain the first sub-time feature. The first cross learning module can be a deep cross network.

[0125] Step G2: extract the historical behavior features of the interactive obstacle to obtain a second sub-time feature.

[0126] As described above, the historical behavior characteristics of the interactive obstacle may include at least one of the position information, velocity statistics information, and acceleration information of the interactive obstacle. The disclosed embodiment is described as follows: the historical behavior characteristics of the interactive obstacle include the position information, velocity statistics information, and acceleration information of the target obstacle, and the temporal dependency relationship between the position information, velocity statistics information, and acceleration information of the interactive obstacle is feature extracted to obtain the second sub-time feature.

[0127] In some embodiments, a deep cross network may be used to extract historical behavior features of interactive obstacles.

[0128] like Figure 4 As shown, the historical behavior characteristics of the interactive obstacles are input into the second cross-learning module to complete the feature extraction operation and obtain the second sub-time characteristics.

[0129] It should be noted that during the execution process, the execution order of step G1 and step G2 is not limited.

[0130] Step G3: Fusing the first sub-time feature and the second sub-time feature to obtain a time feature.

[0131] The purpose of fusing the first sub-time feature and the second sub-time feature is to combine the historical behavior characteristics of the target obstacle in the time dimension and the historical behavior characteristics of the interactive obstacle in the time dimension to describe the movement law of the target obstacle in the time dimension.

[0132] In the disclosed embodiment, the first sub-time feature and the second sub-time feature are fused to obtain the relationship between the target obstacle and the interactive obstacle in the time dimension, which reflects the influence of the interactive obstacle on the target obstacle trajectory prediction and helps to improve the accuracy of the target obstacle trajectory prediction.

[0133] In the embodiment of the present disclosure, the fusion process of the first sub-time feature and the second sub-time feature may be performed as follows:

[0134] Step H1: perform a splicing operation on the first sub-time feature and the second sub-time feature to obtain a splicing feature.

[0135] like Figure 4 As shown, after extracting the historical behavior features of the target obstacle and the historical behavior features of the interactive obstacle, a first sub-time feature and a second sub-time feature are obtained. Then, the first sub-time feature and the second sub-time feature are spliced ​​to obtain a spliced ​​feature.

[0136] In some embodiments, the concat method can be used to concatenate the first sub-time feature and the second sub-time feature. The concat method can increase the feature dimension without changing the amount of information under each dimensional feature. In addition, the concat method directly connects two features. For example, the dimension of the first sub-time feature is x, the dimension of the second sub-time feature is y, and the dimension of the output concatenated feature is x+y. Concat can retain all feature details to facilitate subsequent trajectory prediction.

[0137] In some other embodiments, the first sub-time feature and the second sub-time feature may be concatenated in an add manner, which can increase the amount of information in each dimension while keeping the dimension unchanged.

[0138] like Figure 4 As shown, after the splicing features are obtained, a pooling operation can be performed on the splicing features in step H2 to obtain the time features.

[0139] In some embodiments, the pooling operation on the spliced ​​features may adopt maximum pooling or average pooling to obtain the temporal features.

[0140] In the disclosed embodiment, a splicing operation is performed on the first sub-time feature and the second sub-time feature, and multiple features are spliced ​​to enrich the amount of information represented by the feature. More rich feature information can be captured by the splicing operation. In addition, a pooling operation is performed on the spliced ​​features to reduce the dimension of the features, retain the main feature information, and reduce the amount of calculation. The combination of splicing and pooling operations provides better feature representation capabilities and improves the efficiency and accuracy of trajectory prediction.

[0141] Step F2: interactively learning the time feature and the first spatial feature to obtain the time coding feature; wherein the first spatial feature is obtained by extracting the historical behavior feature of the traffic participant based on the first feature extraction method.

[0142] During implementation, the first spatial feature may be obtained by referring to the implementation method of step A1, and the embodiments of the present disclosure will not be described in detail here.

[0143] Among them, the interactive learning of time features and the first spatial features realizes the coupling of time features and spatial features of traffic participants. The time coding feature integrates the time and space characteristics of traffic participants and can comprehensively describe the movement laws of target obstacles.

[0144] In some embodiments, the Figure 3 The MCG neural network shown interactively learns the time feature and the first spatial feature to obtain a first sub-time coding feature and a second sub-time coding feature, and regards the first sub-time coding feature and the second sub-time coding feature as time features.

[0145] In some other embodiments, an attention mechanism may be used to interactively learn temporal features and spatial features. Taking the multi-head attention mechanism and the cross attention mechanism as examples, the temporal features may be used as inputs Q and K, the first spatial features may be used as inputs V, and the output may be the temporal encoding features.

[0146] In the disclosed embodiment, the time feature can be represented as a high-level historical behavior feature of a traffic participant in the time dimension, and the first spatial feature can be represented as a high-level historical behavior feature of a traffic participant in the spatial dimension. The time feature and the first spatial feature can more accurately describe the historical movement law of the traffic participant. Interactive learning of the time feature and the first spatial feature can enable the time coding feature to represent the historical behavior of the target obstacle within the corresponding time range, which helps to improve the accuracy of the target obstacle trajectory prediction.

[0147] In the embodiment of the present disclosure, step S201 is executed to obtain spatial coding features, and step S202 is executed to obtain temporal coding features. It should be noted that the execution order of S201 and S202 is not limited during the execution process. In order to obtain scene coding features, the temporal coding features and the spatial coding features can be fused in S203 to obtain scene coding features.

[0148] In other embodiments, the temporal coding features and the spatial coding features may be fused in an aggregation manner, wherein the fusion manner may be a concat manner or an add manner to obtain the scene coding features.

[0149] In other embodiments, interactive learning of the temporal feature and the first spatial feature may also realize the spatiotemporal coupling among the target obstacle, the interactive obstacle and the topological feature, thereby obtaining the scene coding feature.

[0150] In the embodiment of the present disclosure, since the autonomous driving vehicle should be affected synchronously by time and space during its movement, the scene coding features extracted after the fusion of time coding features and space coding features in the embodiment of the present disclosure can better simulate the interaction between the target obstacle and the interactive obstacle in the real scene, so as to more accurately predict the trajectory of the target obstacle from the road topology.

[0151] 2) Predict potential movement paths

[0152] In the disclosed embodiment, the potential moving path is used to describe the possible moving path of the target obstacle in the future. In the disclosed embodiment, the potential moving path is predicted based on the scene coding features and the path anchor point set. The path anchor point set comes from multiple candidate paths included in the road topology of the target obstacle. In some possible implementations, equal-interval sampling or non-equal-interval sampling is performed on the candidate path to obtain the path anchor point set. For example, Figure 5 As shown, there are four paths on the road topology where the current target obstacle is located. Equal-interval sampling or non-equal-interval sampling is performed on the four paths to obtain an anchor point sequence for each path. The anchor point sequences of all paths constitute a path anchor point set.

[0153] Based on this, the potential moving path is predicted, which can be implemented as follows: the scene coding features and the path anchor point set on the road topology are input into the path intention prediction network to obtain a first probability distribution; based on the first probability distribution, the potential moving path is screened out from multiple candidate paths.

[0154] Among them, the path intention prediction network can accurately predict the first probability distribution based on the powerful learning and processing capabilities of the neural network model, so as to accurately extract the potential moving path.

[0155] In some possible implementations, the first probability distribution can be predicted based on wayformer. Wayformer provides a compact model description, which consists of an attention-based encoder and a decoder. In the encoder, the fusion of input information and feature selection are realized. In the decoder, based on the scene coding features and path anchor point set output by the encoder, the first probability distribution can be accurately predicted.

[0156] In some other possible implementations, the following operations may be performed based on the path prediction network to obtain a first probability distribution: interactively learning scene coding features and path anchor point sets to obtain path anchor intermediate features; and performing classification operations on the path anchor intermediate features to obtain a first probability distribution.

[0157] After the first probability distribution is obtained, a candidate path with the largest probability may be selected as a potential moving path based on the first probability distribution, or a candidate path with a probability greater than a probability threshold may be selected as a potential moving path.

[0158] In the disclosed embodiment, the scene coding feature and the path anchor point set are interactively learned so that the path anchor intermediate feature learns the overall feature expression of the two, and the overall feature expression is an important feature, which is convenient for classification operation to obtain an accurate first probability distribution. Based on the accurate first probability distribution, it is convenient to accurately determine the potential movement path.

[0159] In some embodiments, determining the path anchor intermediate features may be implemented as follows:

[0160] Step I1: extract features from the path anchor point set to obtain initial features of the path anchor.

[0161] When the number of paths in the road topology where the target obstacle is located is large and the paths are long, the path anchor point set contains a large amount of data. Feature extraction of the path anchor point set with a large amount of data can not only reduce the feature dimension and the amount of data storage, but also extract the high-level features of the road topology to facilitate trajectory prediction.

[0162] In some embodiments, the path anchor point set may be extracted by alternating convolution and pooling to obtain the path anchor initial features. Specifically, the following may be performed:

[0163] Step J1: Map the path anchor point set to the feature space of the target dimension to obtain the target features of the path anchor point set.

[0164] like Figure 6As shown, the path anchor point set is input into the multi-layer perceptron for processing to obtain the target features of the path anchor point set. After obtaining the target features of the path anchor point set, the target features can be pooled in step J2 to obtain the path anchor initial features.

[0165] In some embodiments, the pooling operation on the target feature may adopt maximum pooling or average pooling to obtain the path anchor initial feature.

[0166] In the disclosed embodiment, a multi-layer perceptron and a pooling layer are stacked to form a deep network structure. The multi-layer perceptron extracts higher-level and more abstract features in the path anchor point set to obtain the target features of the path anchor point set, and the pooling layer is used to obtain the main features in the target features of the path anchor point set, that is, the path anchor initial features. The alternating use of the multi-layer perceptron and the pooling operation not only reduces the dimension of the path anchor point set, but also retains important feature information, which helps to improve the accuracy of trajectory prediction.

[0167] Step I2: interactively learn the path anchor initial features and the scene encoding features to obtain the path anchor intermediate features.

[0168] In some embodiments, an MCG neural network can be used to interactively learn the path anchor initial features and scene coding features to obtain high-level path anchor initial features and high-level scene coding features, and the high-level path anchor initial features and high-level scene coding features are used as path anchor intermediate features.

[0169] In other embodiments, an attention mechanism may also be used to interactively learn the path anchor initial features and the scene encoding features. Taking the multi-head attention mechanism and the cross attention mechanism as examples, the path anchor initial features may be used as inputs Q and K, the scene encoding features may be used as inputs V, and the output may be the path anchor intermediate features.

[0170] Among them, Figure 6 As shown, the MLP and softmax (normalized exponential function) layers can be stacked to form a classification branch, the path anchor intermediate features are used as the input of the classification branch, and the softmax layer outputs the first probability distribution p(P i |X). Then, the moving paths are screened from high to low according to the probability distribution to obtain at least one potential moving path with a higher probability.

[0171] In the disclosed embodiment, by extracting features from the path anchor point set to obtain high-level path features, and combining the scene coding features to obtain the path intermediate features, it is beneficial to allow the scene coding features and the path anchor point set to perform in-depth interactive learning, thereby facilitating the improvement of classification accuracy. Classification prediction based on the path anchor intermediate features can not only better capture the nonlinear relationship in the path anchor intermediate features and improve the classification accuracy, but also convert the classification problem into a probability distribution, and perform classification by calculating the probability of the category, making the output easier to understand and improving the accuracy of trajectory prediction.

[0172] In the disclosed embodiment, the road topology includes multiple candidate paths. Each anchor point sequence in the path anchor point set represents a candidate path in the road topology. The anchor point sequence on each candidate path is the trajectory point sequence of the path. Since the potential moving path is a path among the multiple candidate paths, the potential moving path can be expressed by its trajectory point sequence.

[0173] However, the road has a certain width, and the actual moving trajectory of the target obstacle may not move according to the anchor point in the trajectory point sequence. In order to more accurately predict the movement trajectory of the target obstacle on the potential moving path, the potential moving path will be corrected in the embodiment of the present disclosure. Therefore, in addition to using the classification branch to obtain the first probability distribution, it is also necessary to use the regression branch to correct the trajectory point sequence of the potential moving path, which can be implemented as follows:

[0174] Step K1: perform regression operation on the intermediate features of the path anchor to obtain the offset of each trajectory point on the potential path.

[0175] like Figure 6 As shown in the figure, the regression branch is composed of MLP, the intermediate features of the path anchor are input into MLP, and the offset of each trajectory point on the potential path is output, which can be expressed as the mean information of the offset v(P i ),i=1,…,L. Figure 6 The softmax in can be used to make classification predictions based on the offset of each trajectory point on the potential path output by MLP.

[0176] Since the prediction of the potential moving path is divided into two subtasks, namely the classification subtask and the regression subtask, it is necessary to design a potential moving path loss value function to ensure the accuracy of the target obstacle trajectory prediction. late Using the binary cross entropy function L BCE (·) and Smooth-L1 loss function (smoothed L1 norm loss function) L L1 (·) constrains the two subtasks respectively, which can be expressed by expression (1):

[0177]

[0178] Where u represents the potential moving path label information, that is, the real moving trajectory of the obstacle as the training sample; P represents the set of at least one potential path intention, that is, all candidate moving paths in the road topology of the training sample; X represents the combination of the historical behavior characteristics of the training sample, the historical behavior characteristics of the interactive obstacles of the training sample, and the topological characteristics of the training sample; L represents the number of candidate paths of the training sample; i represents the i-th path that can be traveled on the road section where the current target obstacle is located; p(P i |X) represents the first probability distribution of the i-th candidate path as a training sample, v(P i ) represents the predicted mean offset information of each trajectory point on the i-th candidate path, Δa P,u Represents the true value of the offset of the trajectory point on the potential moving path (i.e., Ground truth).

[0179] Step K2: correcting the trajectory point sequence of the potential moving path based on the offset of each trajectory point on the potential path.

[0180] Each trajectory point uses (x, y) to represent its position coordinates. The offset is represented by (△x, △y). Correspondingly, the modified trajectory point is (x+△x, y+△y)

[0181] In the disclosed embodiment, the offset of each trajectory point on the potential path is obtained through regression operation, and the trajectory point sequence of the potential moving path is corrected by the offset, so as to improve the accuracy of the target obstacle trajectory prediction.

[0182] 3) Predict short-term trajectory points on potential movement paths

[0183] In order to facilitate the interaction between the autonomous driving vehicle and the target obstacle, reasonable interaction decisions are made to control the autonomous driving vehicle. In the embodiment of the present disclosure, after determining the long-term potential moving path of the target obstacle, the short-term moving trajectory points of the target obstacle are also predicted. In the embodiment of the present disclosure, based on the set of candidate anchor points on the moving path, at least one moving trajectory point of the target obstacle on the potential moving path within a specified time period in the future (i.e., the short term in the future) is determined. Among them, the candidate anchor point set can be obtained in the following way:

[0184] Method 1: Filter anchor points on the potential moving path from the path anchor point set to obtain a candidate anchor point set.

[0185] As described above, the candidate anchor point set is a set of anchor point sequences sampled on each candidate path in the road topology of the target obstacle. For example, there are 32 candidate paths in the road topology of the target obstacle. Each candidate path is sampled at a fixed interval of 2 meters. Each candidate path supports up to 128 road sections. 25 anchor points are obtained by sampling (such as uniform interpolation) in these 128 road sections to construct a path anchor point sequence. The path anchor point sequence of the potential moving path can be used as a candidate anchor point set. It is also possible to screen out multiple points that are important for trajectory point prediction from the path anchor point sequence of the potential moving path as a candidate anchor point set.

[0186] In the disclosed embodiment, anchor points on a potential moving path are screened out from a path anchor point set, some invalid and repeated data are screened out, the amount of calculation for subsequent processing is reduced, and the efficiency of trajectory prediction is improved.

[0187] Method 2: Sampling the potential moving paths to obtain a set of candidate anchor points.

[0188] The uncorrected potential moving path may be a path with the largest first probability distribution, or may be multiple paths with larger first probability distribution. In addition, the sampling operation on the potential path is divided into two cases:

[0189] Case 1: Sampling operation is performed on the uncorrected potential moving path to obtain a set of candidate anchor points.

[0190] Wherein, when the uncorrected potential moving path is a path with the largest first probability distribution, the path is sampled at equal intervals or non-equal intervals to obtain a candidate anchor point sequence of the path.

[0191] For the case where the uncorrected potential moving paths are multiple paths with a larger first probability distribution, each uncorrected potential moving path is sampled at equal or non-equal intervals to obtain a candidate anchor point sequence on each uncorrected potential moving path, and the candidate anchor point sequences on all uncorrected potential moving paths constitute a candidate anchor point set.

[0192] Case 2: Sampling operation is performed on the corrected potential moving path to obtain a set of candidate anchor points.

[0193] Wherein, if the corrected potential moving path is the path with the largest first probability distribution, the path is sampled at equal intervals or non-equal intervals to obtain a candidate anchor point sequence of the path. When there is only one potential moving path, the candidate anchor point sequence is a set of candidate anchor points on the potential moving path.

[0194] For the case where the corrected potential moving paths are multiple paths with a larger first probability distribution, each corrected potential moving path is sampled at equal or non-equal intervals to obtain a candidate anchor point sequence on each corrected potential moving path, and the candidate anchor point sequences on all potential moving paths constitute a candidate anchor point set.

[0195] In the embodiment of the present disclosure, a candidate anchor point set is obtained by sampling on a potential moving path, providing a data set with small computational complexity and low resource consumption.

[0196] Based on the candidate anchor point set, at least one trajectory point of the target obstacle on the potential moving path within a specified future time period is predicted, which can be implemented as follows: the candidate anchor point set and scene coding features on the potential moving path are input into the trajectory point prediction model to obtain a second probability distribution; based on the second probability distribution, at least one moving trajectory point of the target obstacle on the potential moving path within a specified future time period is screened out from the candidate anchor point set.

[0197] In the disclosed embodiment, the powerful learning and reasoning capabilities of the trajectory point prediction model can be used to accurately predict the second probability distribution, thereby improving the accuracy of wardrobe trajectory point prediction.

[0198] In some possible implementations, the wayformer decoder may also be used to predict the second probability distribution. It should be noted that when the wayformer decoder is used to predict the first probability distribution and the second probability distribution, the decoders predicting the two probability distributions are two decoders with the same structure and trained independently.

[0199] In some other possible implementations, the disclosed embodiments further provide another trajectory point prediction model to predict the second probability distribution. Specifically, the following operations may be performed based on the trajectory point prediction model: the candidate anchor point set and the scene coding feature are fused to obtain fused features; the fused features are classified and predicted to obtain the second probability distribution.

[0200] In the disclosed embodiment, by fusing the candidate anchor point set and the scene coding features, it is helpful to extract high-level features that can describe the future movement trend of the target obstacle. Based on this feature, the second probability distribution can be accurately obtained, so as to ultimately accurately predict the trajectory points on the potential movement path in the short term.

[0201] In some embodiments, in step L1, feature extraction may be performed on the candidate anchor point set to obtain trajectory point features.

[0202] In some embodiments, convolution processing and pooling operations can be used to extract features from the candidate anchor point set, or a multi-layer perceptron can be used to extract features from the candidate anchor point set. The present disclosure embodiment does not limit the feature extraction method. Specifically, it can be executed as follows:

[0203] Step M1: Expand the candidate anchor point set to the specified dimension to obtain the trajectory point encoding features.

[0204] In some embodiments, the dimension of the candidate anchor point set may be expanded by feature construction to obtain trajectory point coding features, wherein feature construction creates new features based on existing features in the candidate anchor point set to capture the hidden associations in the candidate anchor point set.

[0205] Step M2: interactively learn the trajectory point coding features and the third space features of the potential moving path to obtain the trajectory point features.

[0206] like Figure 4 As shown, the third spatial feature may be a spatial coding feature portion of a potential moving path extracted from the spatial coding feature where the target obstacle is located.

[0207] In some embodiments, an MCG neural network may be used to interactively learn the trajectory point encoding features and the third space features, or an attention mechanism may be used to interactively learn. In the embodiments of the present disclosure, there is no limitation on the interactive learning method.

[0208] Among them, Figure 6 The implementation step M2 shown is to input the trajectory point coding features and the third spatial features into the fifth interactive learning module (such as a multi-layer MCG neural network) and output the trajectory point features.

[0209] In the disclosed embodiment, the trajectory point coding features are used as high-level features of the candidate anchor point set, which can represent the significant features of the candidate anchor point set. The trajectory point coding features and the third space features of the potential moving path are interactively learned to represent the interaction relationship between the target obstacle and the interactive obstacle at the trajectory point and the road topology features at the trajectory point. Based on various information, the accuracy of short-term trajectory point prediction can be improved.

[0210] Step L2: Fusing the trajectory point features and the scene coding features to obtain fused features.

[0211] As explained above, the scene coding features integrate temporal coding features and spatial coding features, and can reflect the temporal and spatial characteristics of the target obstacles.

[0212] In some embodiments, the trajectory point features and the scene coding features may be fused by a cancat method or by an add method. In the embodiments disclosed herein, the feature fusion method is not limited.

[0213] In the present disclosure, Figure 6 As shown, step L2 can be executed as follows:

[0214] The scene coding features and trajectory point features are used as two inputs of a fusion processing method (such as concat) and the fusion features are output.

[0215] Among them, Figure 6 As shown, classification prediction is performed on the fusion features to obtain at least one moving trajectory point of the target obstacle on the potential moving path within a specified time period in the future. Figure 6 In the above example, the MLP and softmax layers are stacked to form a classification branch, the fusion features are used as the input of the classification branch, and the softmax layer outputs the second probability distribution Get at least one moving trajectory point on a potential moving path within a specified future time period.

[0216] In some embodiments, at least one trajectory point on the potential moving path within the future specified time period may be a trajectory point of the target obstacle at a time point 3 seconds in the future, that is, a trajectory point that is most likely to be reached by the target obstacle in the next 3 seconds.

[0217] In other embodiments, at least one trajectory point on the potential moving path within the future specified time period may be multiple trajectory points of the target obstacle within the next 3 seconds. Assuming that there are 10 time steps in 3 seconds, the at least one trajectory point in step L3 includes a trajectory point for each of the 10 time steps. The total time length of the 10 time steps is 3 seconds. Therefore, in the disclosed embodiment, 10 future trajectory points can also be predicted.

[0218] In the disclosed embodiment, feature extraction of the candidate anchor point set to obtain trajectory point coding features can reduce the feature dimension of the candidate anchor point set and retain important feature information. The trajectory point features and scene coding features are fused to obtain fused features, which can represent the spatiotemporal characteristics of the target obstacle at the trajectory point, and better describe the road topology characteristics of the target obstacle at the trajectory point in the real scene and the interactive relationship with the interactive obstacles. In addition, the fused features can also reflect the historical behavior characteristics of the target obstacle at the trajectory point. At the same time, the fused features are classified and processed to obtain the movement law of the target obstacle in the short term, which helps to improve the accuracy of trajectory point prediction in the short term.

[0219] In addition to using the classification branch to obtain the second probability distribution, in the embodiment of the present disclosure, the actual trajectory point in the short term may not completely coincide with the trajectory point in the candidate anchor point set. Therefore, it is also necessary to use the regression branch to correct at least one moving trajectory point. It can be executed as follows:

[0220] Step N1: perform regression operation on the fused features to obtain the offset of at least one moving trajectory point.

[0221] like Figure 6 As shown in the figure, the regression branch is composed of MLP, the fusion feature is input into MLP, and the offset of at least one moving trajectory point is output. To correct the position of at least one moving trajectory point.

[0222] A trajectory point loss function is also designed to ensure the accuracy of target obstacle trajectory prediction. The trajectory point loss function L long Using the binary cross entropy function L BCE (·) and the mooth-L1 loss function L L1 (·) respectively constrain the trajectory point prediction, which can be expressed by expression (2):

[0223]

[0224] Among them, w represents the intention label information of the trajectory point, that is, the true value of the trajectory point in the short term; u represents the label information of the potential moving path; X represents the combination of the historical behavior characteristics of the training sample, the historical behavior characteristics of the interactive obstacles of the training sample, and the topological characteristics of the training sample; M represents the dimension of the preset candidate anchor point sequence; j represents the jth trajectory point in the potential moving path where the target obstacle is located; represents the second probability distribution of each trajectory point in path u as a trajectory point in the short term, Indicates the predicted offset of at least one moving trajectory point in the short term. It represents the true value of the offset between at least one trajectory point in a short period of time and the actual label information of at least one trajectory point.

[0225] Step N2: correct at least one moving trajectory point based on the offset.

[0226] In the disclosed embodiment, the offset of at least one moving trajectory point on the potential moving path is obtained through a regression operation, and at least one moving trajectory point is corrected by the offset, so as to better predict the short-term movement of the target obstacle in the real scene and provide real and reliable data for interactive decision-making.

[0227] Taking the first cross-learning module and the second cross-learning module as deep cross networks, and the learning module as an example, combined with Figure 7The solution provided by the embodiment of the present disclosure is described in detail:

[0228] Step O1, the method of extracting the target obstacles, interactive obstacles and topological features in the spatial coding network to obtain the first subspace features, the second subspace features and the second space features has been described in the previous text, and the embodiments of the present disclosure will not be repeated here one by one.

[0229] The first interactive learning module and the second interactive learning module in the spatial coding network are exemplified by the first MCG neural network and the second MCG neural network. As described above, the MCG neural network includes two inputs and two outputs.

[0230] Step O2: input the first subspace feature and the second subspace feature into the first MCG neural network (ie, MCG1), and output the updated first subspace feature and the updated second subspace feature. The updated second subspace feature may not be used later.

[0231] Step O3, use the updated first subspace feature and the second space feature as the input of the second MCG neural network (ie, MCG2), output the first sub-coding feature and the second sub-coding feature, and regard the first sub-coding feature and the second sub-coding feature as space coding features.

[0232] In the time coding network, the first cross-learning module and the second cross-learning module take the first deep cross neural network and the second deep cross neural network as examples, and the third interactive learning module is replaced by the third MCG neural network, which is implemented as follows:

[0233] Step O4: input the historical behavior characteristics of the target obstacle into the first deep cross neural network (i.e., cross neural network 1) to obtain the second sub-time characteristics, and input the historical behavior characteristics of the interactive obstacle into the second deep cross neural network (i.e., cross neural network 2) to obtain the first sub-time characteristics.

[0234] Step O5, concatenate and pool the first sub-time feature and the second sub-time feature to obtain the time feature. The specific operation of obtaining the time feature has been described in the previous text, and the embodiment of the present disclosure will not be repeated here.

[0235] Step O6, input the time feature and the first spatial feature into the third MCG neural network (ie, MCG3) to obtain a first sub-time coding feature and a second sub-time coding feature. The first sub-time coding feature and the second sub-time coding feature are regarded as time coding features.

[0236] Step 07: Fusing the temporal coding feature and the spatial coding feature to obtain the scene coding feature. The implementation method of determining the scene coding feature has been described above, and the embodiments of the present disclosure will not be described in detail here.

[0237] In the implementation process of determining the potential moving path, the fourth interactive learning module takes the fourth MCG neural network as an example (ie, MCG4) and is implemented as follows:

[0238] The implementation method of step O8, extracting features from the path anchor point set to obtain the initial features of the path anchor, has been described above and will not be repeated here in detail in the embodiments of the present disclosure.

[0239] Step O9, interactive learning of the path anchor initial features and the scene encoding features is implemented as follows:

[0240] The path anchor initial features and scene coding features are input into the fourth MCG neural network, and the updated path anchor initial features (i.e., path anchor intermediate features) and updated scene coding features are output. The updated scene coding features may no longer be used in subsequent operations.

[0241] Step O10: Perform classification and regression operations on the intermediate features of the path anchors to determine the potential moving path. The implementation of the classification and regression operation has been described above, and the embodiment of the present disclosure will not be repeated here.

[0242] Step O11: determine a set of candidate anchor points, extract features from the set of candidate anchor points, and obtain trajectory point coding features. The implementation process has been described above, and the embodiments of the present disclosure will not be described in detail here.

[0243] Step O12: interactively learn based on the third spatial feature and the trajectory point coding feature, and then perform fusion processing to obtain fusion features. The interactive learning and fusion processing methods have been described above, and the embodiments of the present disclosure will not be repeated here. Figure 7 The fusion feature can be obtained based on at least one MCG5 arranged in series. Each MCG5 will obtain the updated third spatial feature and the updated trajectory point coding feature, and the updated trajectory point coding feature can be used as the fusion feature.

[0244] Step O13: Perform classification and regression operations on the fused features to obtain at least one moving trajectory point. The implementation of the classification and regression operation has been described above, and the embodiments of the present disclosure will not be described in detail here.

[0245] It should be noted that the embodiments of the present disclosure do not limit how to obtain the scene coding features, the model structure of the first probability distribution and the second probability distribution. During implementation, the corresponding model structure can be designed and selected according to actual needs.

[0246] Based on the same technical concept, the embodiment of the present disclosure also provides an obstacle trajectory prediction device 800, such as Figure 8 As shown, including:

[0247] The encoding module 801 is used to encode the historical behavior characteristics of the traffic participants and the topological characteristics of the road topology of the road section where the target obstacle is located to obtain the scene encoding characteristics; the traffic participants at least include the target obstacle; wherein the historical behavior characteristics of the traffic participants include the action trajectory and / or motion state within a specified time period in the past;

[0248] The first prediction module 802 is used to determine a potential moving path of the target obstacle based on a first probability distribution predicted by the scene coding feature and the path anchor point set on the road topology; wherein the first probability distribution is used to describe the probability of the target obstacle moving on each candidate path of the road topology;

[0249] The second prediction module 803 is used to determine at least one moving trajectory point of the target obstacle on the potential moving path within a specified time period in the future based on the set of candidate anchor points on the potential moving path and the second probability distribution predicted by the scene coding features; the second probability distribution is used to describe the probability of the target obstacle moving to each candidate anchor point in the candidate anchor point set within the specified time period in the future.

[0250] In some embodiments, the first prediction module includes:

[0251] A first input submodule is used to input the scene coding features and the path anchor point set on the road topology into the path intention prediction network to obtain a first probability distribution;

[0252] The first screening submodule is used to screen out potential moving paths from multiple candidate paths based on a first probability distribution.

[0253] In some embodiments, the first input submodule is specifically configured to perform the following operations based on the path intention prediction network:

[0254] Interactively learn the scene encoding features and the path anchor point set to obtain the path anchor intermediate features;

[0255] The intermediate features of the path anchors are classified to obtain the first probability distribution.

[0256] In some embodiments, the first input submodule is specifically used to:

[0257] Extract features from the path anchor point set to obtain the initial features of the path anchor;

[0258] The path anchor initial features and scene encoding features are interactively learned to obtain the path anchor intermediate features.

[0259] In some embodiments, it also includes:

[0260] The first regression module is used to perform regression operation on the intermediate features of the path anchor to obtain the offset of each trajectory point on the potential moving path;

[0261] The first correction module is used to correct the trajectory point sequence of the potential movement path based on the offset of each trajectory point on the potential movement path.

[0262] In some embodiments, the first input submodule is specifically used to:

[0263] Map the path anchor point set to the feature space of the target dimension to obtain the target features of the path anchor point set;

[0264] Pooling operation is performed on the target features to obtain the initial features of the path anchor.

[0265] In some embodiments, a first anchor point determination module is further included, which is used to filter out anchor points on the potential moving path from the path anchor point set to obtain a candidate anchor point set.

[0266] In some embodiments, a second anchor point determination module is further included, which is used to perform sampling operations on potential moving paths to obtain a set of candidate anchor points.

[0267] In some embodiments, the second prediction module includes:

[0268] A second input submodule, used to input the candidate anchor point set and scene coding features on the potential moving path into the trajectory point prediction model to obtain a second probability distribution;

[0269] The second screening submodule is used to screen out at least one moving trajectory point of the target obstacle on a potential moving path within a specified future time period from the candidate anchor point set based on the second probability distribution.

[0270] In some embodiments, the second input submodule is specifically configured to perform the following operations based on the trajectory point prediction model:

[0271] The candidate anchor point set and the scene coding feature are fused to obtain the fused feature;

[0272] Classify and predict the fused features to obtain the second probability distribution.

[0273] In some embodiments, it also includes:

[0274] A second regression module is used to perform a regression operation on the fused features to obtain an offset of at least one moving trajectory point;

[0275] The second correction module is used to correct at least one moving trajectory point based on the offset.

[0276] In some embodiments, the second input submodule is specifically used to:

[0277] Extract features from the candidate anchor point set to obtain trajectory point features;

[0278] The trajectory point features and scene coding features are fused to obtain fused features.

[0279] In some embodiments, the second input submodule is specifically used to:

[0280] Expand the candidate anchor point set to the specified dimension to obtain the trajectory point encoding features;

[0281] The trajectory point encoding features and the third space features of the potential moving path are interactively learned to obtain the trajectory point features.

[0282] In some embodiments, the encoding module is specifically configured to:

[0283] The historical behavior characteristics of traffic participants and the topological characteristics of the road topology of the section where the target obstacle is located are input into the spatiotemporal coding network to obtain the scene coding features.

[0284] In some embodiments, the encoding module is specifically configured to perform the following operations based on the spatiotemporal encoding network:

[0285] Spatially encode the historical behavior characteristics and topological characteristics of traffic participants to obtain spatial encoding features; and,

[0286] Time-code the historical behavior characteristics of traffic participants to obtain time-coded features;

[0287] The temporal coding features and the spatial coding features are fused to obtain the scene coding features.

[0288] In some embodiments, the encoding module is specifically configured to:

[0289] Extracting historical behavior features of traffic participants using a first feature extraction method to obtain a first spatial feature; and,

[0290] Extract the topological features to obtain the second spatial features;

[0291] The first spatial feature and the second spatial feature are interactively learned to obtain a spatial coding feature.

[0292] In some embodiments, the traffic participant further includes an interactive obstacle, and the interactive obstacle is an obstacle around the target obstacle, wherein the encoding module is specifically used to:

[0293] Extract the historical behavior characteristics of the target obstacle to obtain the first subspace features; and

[0294] Extract the historical behavior characteristics of the interactive obstacles to obtain the second subspace features;

[0295] The first subspace feature and the second subspace feature are interactively learned to obtain the first space feature.

[0296] In some embodiments, the encoding module is specifically configured to:

[0297] Perform convolution processing on the historical behavior characteristics of the target obstacle to obtain a first convolution processing result;

[0298] A pooling operation is performed on the first convolution processing result to obtain the first subspace feature.

[0299] In some embodiments, the encoding module is specifically configured to:

[0300] Perform convolution processing on the historical behavior characteristics of the interactive obstacles to obtain a second convolution processing result;

[0301] A pooling operation is performed on the second convolution processing result to obtain the second subspace feature.

[0302] In some embodiments, the encoding module is specifically configured to:

[0303] Performing convolution processing on the topological features to obtain a third convolution processing result;

[0304] A pooling operation is performed on the third convolution processing result to obtain the second spatial feature.

[0305] In some embodiments, the encoding module is specifically configured to:

[0306] Using the second feature extraction method to extract the historical behavior features of traffic participants to obtain time features; and,

[0307] The time feature and the first spatial feature are interactively learned to obtain the time coding feature; wherein the first spatial feature is obtained by extracting the historical behavior features of the traffic participants based on the first feature extraction method.

[0308] In some embodiments, the traffic participant further includes an interactive obstacle, and the interactive obstacle is an obstacle around the target obstacle, wherein the encoding module is specifically used to:

[0309] Extract the historical behavior characteristics of the target obstacle to obtain the first sub-time feature; and

[0310] Extract the historical behavior characteristics of the interactive obstacles to obtain the second sub-time characteristics;

[0311] The first sub-time feature and the second sub-time feature are fused to obtain a time feature.

[0312] In some embodiments, the encoding module is specifically configured to:

[0313] Performing a splicing operation on the first sub-time feature and the second sub-time feature to obtain a splicing feature;

[0314] Pooling operation is performed on the concatenated features to obtain temporal features.

[0315] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0316] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0317] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0318] Fig. 9 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0319] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0320] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0321] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as the obstacle trajectory prediction method. For example, in some embodiments, the obstacle trajectory prediction method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the obstacle trajectory prediction method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the obstacle trajectory prediction method in any other appropriate manner (e.g., by means of firmware).

[0322] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0323] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0324] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0325] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0326] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0327] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0328] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0329] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for predicting an obstacle trajectory, comprising: The historical behavior characteristics of the traffic participants and the topological characteristics of the road topology of the road section where the target obstacle is located are encoded to obtain scene coding characteristics; the traffic participants at least include the target obstacle; wherein the historical behavior characteristics of the traffic participants include the action trajectory and / or motion state within a specified time period in the past; Determine a potential moving path of the target obstacle based on the scene coding feature and a first probability distribution predicted by a set of path anchor points on the road topology; wherein the first probability distribution is used to describe the probability of the target obstacle moving on each candidate path of the road topology; Based on the set of candidate anchor points on the potential moving path and the second probability distribution predicted by the scene coding features, at least one moving trajectory point of the target obstacle on the potential moving path within a specified future time period is determined; the second probability distribution is used to describe the probability of the target obstacle moving to each candidate anchor point in the candidate anchor point set within the specified future time period.

2. The method according to claim 1, wherein: The determining of the potential moving path of the target obstacle based on the scene coding feature and the first probability distribution predicted by the path anchor point set on the road topology includes: Inputting the scene coding features and the path anchor point set on the road topology into a path intention prediction network to obtain the first probability distribution; The potential moving path is screened out from the multiple candidate paths based on the first probability distribution.

3. The method according to claim 2, wherein: The step of inputting the scene coding features and the set of path anchor points on the road topology into a path intention prediction network to obtain the first probability distribution includes performing the following operations based on the path intention prediction network: Interactively learning the scene coding feature and the path anchor point set to obtain a path anchor intermediate feature; A classification operation is performed on the intermediate features of the path anchor to obtain the first probability distribution.

4. The method according to claim 3, wherein: The interactive learning of the scene coding feature and the path anchor point set to obtain the path anchor intermediate feature includes: Extracting features from the path anchor point set to obtain initial features of the path anchor; The path anchor initial feature and the scene encoding feature are interactively learned to obtain the path anchor intermediate feature.

5. The method according to claim 3 or 4, further comprising: Performing a regression operation on the intermediate features of the path anchor to obtain the offset of each trajectory point on the potential moving path; The trajectory point sequence of the potential movement path is corrected based on the offset of each trajectory point on the potential movement path.

6. The method according to claim 4, wherein: The step of extracting features from the path anchor point set to obtain initial features of the path anchor includes: Mapping the path anchor point set to a feature space of a target dimension to obtain target features of the path anchor point set; A pooling operation is performed on the target features to obtain the path anchor initial features.

7. The method according to any one of claims 1 to 6, further comprising determining the candidate anchor point set based on the following method: The anchor points on the potential moving path are screened out from the path anchor point set to obtain the candidate anchor point set.

8. The method according to any one of claims 1 to 6, further comprising determining the candidate anchor point set based on the following method: A sampling operation is performed on the potential moving path to obtain the candidate anchor point set.

9. The method according to any one of claims 1 to 8, wherein: The determining, based on the candidate anchor point set on the potential movement path and the second probability distribution predicted by the scene coding feature, at least one movement trajectory point of the target obstacle on the potential movement path within a specified future time period comprises: Inputting the candidate anchor point set on the potential moving path and the scene coding feature into a trajectory point prediction model to obtain the second probability distribution; Based on the second probability distribution, at least one moving trajectory point of the target obstacle on the potential moving path within the future specified time period is screened out from the candidate anchor point set.

10. The method according to claim 9, wherein: The step of inputting the candidate anchor point set on the potential moving path and the scene coding feature into a trajectory point prediction model to obtain the second probability distribution includes performing the following operations based on the trajectory point prediction model: Fusing the candidate anchor point set with the scene coding feature to obtain a fused feature; Classify and predict the fused features to obtain the second probability distribution.

11. The method according to claim 10, further comprising: Performing a regression operation on the fused features to obtain an offset of the at least one moving trajectory point; The at least one movement trajectory point is corrected based on the offset.

12. The method according to claim 10 or 11, wherein: The step of fusing the candidate anchor point set and the scene coding feature to obtain a fused feature includes: Performing feature extraction on the candidate anchor point set to obtain trajectory point features; The trajectory point feature and the scene coding feature are fused to obtain the fused feature.

13. The method according to claim 11, wherein: The extracting features of the candidate anchor point set to obtain trajectory point features includes: Expanding the candidate anchor point set to a specified dimension to obtain trajectory point encoding features; The trajectory point coding feature and the third spatial feature of the potential moving path are interactively learned to obtain the trajectory point feature.

14. The method according to any one of claims 1 to 13, wherein: The encoding of the historical behavior characteristics of the traffic participants and the topological characteristics of the road topology of the road section where the target obstacle is located to obtain the scene encoding characteristics includes: The historical behavior characteristics of the traffic participants and the topological characteristics of the road topology of the section where the target obstacle is located are input into a spatiotemporal coding network to obtain the scene coding characteristics.

15. The method according to claim 14, wherein: The step of inputting the historical behavior characteristics of the traffic participants and the topological characteristics of the road topology of the road section where the target obstacle is located into a spatiotemporal coding network to obtain the scene coding characteristics comprises performing the following operations based on the spatiotemporal coding network: Performing spatial coding on the historical behavior characteristics of the traffic participants and the topological characteristics to obtain spatial coding characteristics; and, Performing time coding on the historical behavior characteristics of the traffic participants to obtain time coding characteristics; The time coding feature and the space coding feature are fused to obtain the scene coding feature.

16. The method according to claim 15, wherein: The spatial coding of the historical behavior characteristics and the topological characteristics of the traffic participants to obtain spatial coding characteristics includes: extracting the historical behavior features of the traffic participants using a first feature extraction method to obtain a first spatial feature; and, Extracting the topological features to obtain second spatial features; The first spatial feature and the second spatial feature are interactively learned to obtain the spatial coding feature.

17. The method according to claim 16, wherein the traffic participants further include interactive obstacles, the interactive obstacles being obstacles around the target obstacle, wherein: The first feature extraction method is used to extract the historical behavior features of the traffic participants to obtain the first spatial features, including: Extracting the historical behavior characteristics of the target obstacle to obtain a first subspace feature; and Extracting the historical behavior characteristics of the interactive obstacle to obtain a second subspace feature; The first subspace feature and the second subspace feature are interactively learned to obtain the first space feature.

18. The method according to claim 17, wherein: The extracting the historical behavior features of the target obstacle to obtain the first subspace features includes: Performing convolution processing on the historical behavior characteristics of the target obstacle to obtain a first convolution processing result; A pooling operation is performed on the first convolution processing result to obtain the first subspace feature.

19. The method according to claim 17 or 18, wherein: The extracting the historical behavior features of the interactive obstacle to obtain the second subspace features includes: Performing convolution processing on the historical behavior characteristics of the interactive obstacle to obtain a second convolution processing result; A pooling operation is performed on the second convolution processing result to obtain the second subspace feature.

20. The method according to any one of claims 16 to 19, wherein: The extracting the topological features to obtain the second spatial features includes: Performing convolution processing on the topological feature to obtain a third convolution processing result; A pooling operation is performed on the third convolution processing result to obtain the second spatial feature.

21. The method according to any one of claims 15 to 20, wherein: The time coding of the historical behavior characteristics of the traffic participants to obtain time coding characteristics includes: Using a second feature extraction method to extract the historical behavior features of the traffic participants to obtain a time feature; and, The time feature and the first spatial feature are interactively learned to obtain the time coding feature; wherein the first spatial feature is obtained by extracting the historical behavior feature of the traffic participant based on a first feature extraction method.

22. The method according to claim 21, wherein: The traffic participant also includes an interactive obstacle, and the interactive obstacle is an obstacle around the target obstacle. The second feature extraction method is used to extract the historical behavior features of the traffic participant to obtain the time feature, including: Extracting the historical behavior characteristics of the target obstacle to obtain a first sub-time feature; and, Extracting the historical behavior characteristics of the interactive obstacle to obtain a second sub-time feature; The first sub-time feature and the second sub-time feature are fused to obtain the time feature.

23. The method according to claim 22, wherein: The fusing the first sub-time feature and the second sub-time feature to obtain the time feature includes: Performing a splicing operation on the first sub-time feature and the second sub-time feature to obtain a splicing feature; A pooling operation is performed on the spliced ​​features to obtain the time features.

24. An obstacle trajectory prediction device, comprising: A coding module, used for coding the historical behavior characteristics of traffic participants and the topological characteristics of the road topology of the road section where the target obstacle is located, to obtain a scene coding feature; the traffic participants at least include the target obstacle; wherein the historical behavior characteristics of the traffic participants include the action trajectory and / or motion state within a specified time period in the past; A first prediction module, configured to determine a potential moving path of the target obstacle based on a first probability distribution predicted by the scene coding feature and a set of path anchor points on the road topology; wherein the first probability distribution is used to describe the probability of the target obstacle moving on each candidate path of the road topology; A second prediction module is used to determine at least one moving trajectory point of the target obstacle on the potential moving path within a specified future time period based on the set of candidate anchor points on the potential moving path and the second probability distribution predicted by the scene coding features; the second probability distribution is used to describe the probability of the target obstacle moving to each candidate anchor point in the candidate anchor point set within the specified future time period.

25. The device according to claim 24, wherein: The first prediction module comprises: A first input submodule, configured to input the scene coding features and the path anchor point set on the road topology into a path intention prediction network to obtain the first probability distribution; The first screening submodule is used to screen the potential moving path from the multiple candidate paths based on the first probability distribution.

26. The device according to claim 25, wherein The first input submodule is specifically configured to perform the following operations based on the path intention prediction network: Interactively learning the scene coding feature and the path anchor point set to obtain a path anchor intermediate feature; A classification operation is performed on the intermediate features of the path anchor to obtain the first probability distribution.

27. The device according to claim 26, wherein: The first input submodule is specifically used for: Extracting features from the path anchor point set to obtain initial features of the path anchor; The path anchor initial feature and the scene encoding feature are interactively learned to obtain the path anchor intermediate feature.

28. The apparatus according to claim 26 or 27, further comprising: A first regression module, used for performing a regression operation on the intermediate features of the path anchor to obtain an offset of each trajectory point on the potential moving path; The first correction module is used to correct the trajectory point sequence of the potential movement path based on the offset of each trajectory point on the potential movement path.

29. The device according to claim 27, wherein: The first input submodule is specifically used for: Mapping the path anchor point set to a feature space of a target dimension to obtain target features of the path anchor point set; A pooling operation is performed on the target features to obtain the path anchor initial features.

30. The apparatus according to any one of claims 24-29, further comprising a first anchor point determination module, configured to filter out anchor points on the potential moving path from the path anchor point set to obtain the candidate anchor point set.

31. The apparatus according to any one of claims 24-29, further comprising a second anchor point determination module, configured to perform a sampling operation on the potential moving path to obtain the candidate anchor point set.

32. The device according to any one of claims 24 to 31, wherein: The second prediction module comprises: A second input submodule, configured to input the candidate anchor point set on the potential moving path and the scene coding feature into a trajectory point prediction model to obtain the second probability distribution; The second screening submodule is used to screen out at least one moving trajectory point of the target obstacle on the potential moving path within the future specified time period from the candidate anchor point set based on the second probability distribution.

33. The device according to claim 32, wherein: The second input submodule is specifically configured to perform the following operations based on the trajectory point prediction model: Fusing the candidate anchor point set with the scene coding feature to obtain a fused feature; Classify and predict the fused features to obtain the second probability distribution.

34. The apparatus of claim 33, further comprising: A second regression module, used for performing a regression operation on the fused features to obtain an offset of the at least one moving trajectory point; The second correction module is used to correct the at least one moving trajectory point based on the offset.

35. The device according to claim 33 or 34, wherein: The second input submodule is specifically used for: Performing feature extraction on the candidate anchor point set to obtain trajectory point features; The trajectory point feature and the scene coding feature are fused to obtain the fused feature.

36. The device according to claim 34, wherein The second input submodule is specifically used for: Expanding the candidate anchor point set to a specified dimension to obtain trajectory point encoding features; The trajectory point coding feature and the third spatial feature of the potential moving path are interactively learned to obtain the trajectory point feature.

37. The device according to any one of claims 24 to 36, wherein: The encoding module is specifically used for: The historical behavior characteristics of the traffic participants and the topological characteristics of the road topology of the section where the target obstacle is located are input into a spatiotemporal coding network to obtain the scene coding characteristics.

38. The device according to claim 37, wherein The encoding module is specifically configured to perform the following operations based on the spatiotemporal encoding network: Performing spatial coding on the historical behavior characteristics of the traffic participants and the topological characteristics to obtain spatial coding characteristics; and, Performing time coding on the historical behavior characteristics of the traffic participants to obtain time coding characteristics; The time coding feature and the space coding feature are fused to obtain the scene coding feature.

39. The device according to claim 38, wherein The encoding module is specifically used for: extracting the historical behavior features of the traffic participants using a first feature extraction method to obtain a first spatial feature; and, Extracting the topological features to obtain second spatial features; The first spatial feature and the second spatial feature are interactively learned to obtain the spatial coding feature.

40. The device according to claim 39, wherein the traffic participant further comprises an interactive obstacle, wherein the interactive obstacle is an obstacle around the target obstacle, wherein: The encoding module is specifically used for: Extracting the historical behavior characteristics of the target obstacle to obtain a first subspace feature; and Extracting the historical behavior characteristics of the interactive obstacle to obtain a second subspace feature; The first subspace feature and the second subspace feature are interactively learned to obtain the first space feature.

41. The device according to claim 40, wherein The encoding module is specifically used for: Performing convolution processing on the historical behavior characteristics of the target obstacle to obtain a first convolution processing result; A pooling operation is performed on the first convolution processing result to obtain the first subspace feature.

42. The device according to claim 40 or 41, wherein The encoding module is specifically used for: Performing convolution processing on the historical behavior characteristics of the interactive obstacle to obtain a second convolution processing result; A pooling operation is performed on the second convolution processing result to obtain the second subspace feature.

43. The device according to any one of claims 39 to 42, wherein: The encoding module is specifically used for: Performing convolution processing on the topological feature to obtain a third convolution processing result; A pooling operation is performed on the third convolution processing result to obtain the second spatial feature.

44. The device according to any one of claims 38 to 43, wherein: The encoding module is specifically used for: Using a second feature extraction method to extract the historical behavior features of the traffic participants to obtain a time feature; and, The time feature and the first spatial feature are interactively learned to obtain the time coding feature; wherein the first spatial feature is obtained by extracting the historical behavior feature of the traffic participant based on a first feature extraction method.

45. The device according to claim 44, wherein The traffic participant also includes an interactive obstacle, and the interactive obstacle is an obstacle around the target obstacle, wherein the encoding module is specifically used for: Extracting the historical behavior characteristics of the target obstacle to obtain a first sub-time feature; and, Extracting the historical behavior characteristics of the interactive obstacle to obtain a second sub-time feature; The first sub-time feature and the second sub-time feature are fused to obtain the time feature.

46. ​​The device according to claim 45, wherein The encoding module is specifically used for: Performing a splicing operation on the first sub-time feature and the second sub-time feature to obtain a splicing feature; A pooling operation is performed on the spliced ​​features to obtain the time features.

47. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 23.

48. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-23.

49. A vehicle comprising the electronic device of claim 47.

Images