A method and apparatus for predicting the trajectory of an obstacle

Abstract

This specification provides a method and apparatus for predicting the trajectory of obstacles. Based on the motion features extracted from the historical motion trajectories of each obstacle, the interaction weights between each pair of obstacles are determined. The motion features are then weighted according to the determined interaction weights to obtain weighted spatial interaction features. A recurrent neural subnetwork obtains the spatiotemporal interaction features of each obstacle based on the input spatial interaction features. A pre-trained second model determines the predicted motion trajectory of each obstacle based on the spatiotemporal interaction features. The interaction weights reflect the degree of influence between each obstacle and each other obstacle. Therefore, the spatial interaction features not only depict the motion features but also the interactions between obstacles, resulting in higher accuracy in predicting the future trajectories of obstacles.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method and apparatus for predicting the trajectory of obstacles. Background Technology

[0002] Currently, in the field of autonomous driving technology, a reference trajectory is usually planned for a period of time in the future so that the autonomous driving equipment can travel along the reference trajectory.

[0003] Because of the presence of obstacles on the road, the reference trajectory planned for autonomous driving equipment needs to ensure that the autonomous driving equipment can avoid the obstacles. For obstacles that can participate in traffic and whose positions change over time, it is usually necessary to predict the future trajectory of the obstacles in order for the autonomous driving equipment to avoid them accurately.

[0004] In existing technologies, the future trajectory of an obstacle is often predicted solely based on its historical state information. However, in the real world, each obstacle considers not only its own trajectory (e.g., how to reach its destination in the shortest distance) but also how to avoid other obstacles. In other words, obstacles objectively influence each other. Therefore, methods that predict the trajectory of obstacles based solely on their historical state information fail to capture the interactions between them, resulting in poor accuracy. Summary of the Invention

[0005] This specification provides an obstacle trajectory prediction method and apparatus to partially solve the problems existing in the prior art.

[0006] The embodiments in this specification adopt the following technical solutions:

[0007] A method for predicting the trajectory of an obstacle includes:

[0008] Determine the historical trajectory of each obstacle;

[0009] For each obstacle, the historical motion trajectory of the obstacle within a preset time period is input into a pre-trained first model, and the motion features of the obstacle are extracted through the feature extraction sub-network of the first model.

[0010] The motion features of each obstacle are input into the attention subnetwork of the first model to obtain the interaction weights between each pair of obstacles.

[0011] The determined interaction weights are used to weight the motion characteristics of each obstacle, and the weighted motion characteristics of each obstacle are used as the spatial interaction characteristics of each obstacle.

[0012] The spatial interaction features of each obstacle are input into the recurrent neural subnetwork of the first model to obtain the spatiotemporal interaction features of each obstacle.

[0013] The spatiotemporal interaction features output by the first model are input into a pre-trained second model, and the predicted motion trajectory of each obstacle is determined through the second model.

[0014] Optionally, the motion features of the obstacle are extracted, specifically including:

[0015] Extract the position and velocity features of the obstacle;

[0016] The motion features of each obstacle are input into the attention subnetwork of the first model, specifically including:

[0017] The positional features of each obstacle are input into the attention subnetwork of the first model;

[0018] The spatial interaction features of each obstacle are input into the recurrent neural subnetwork of the first model to obtain the spatiotemporal interaction features of each obstacle, specifically including:

[0019] The spatial interaction features of each obstacle are input into the recurrent neural subnetwork of the first model to obtain the coarse spatiotemporal interaction features of each obstacle.

[0020] Based on the positional characteristics, velocity characteristics, and coarse spatiotemporal interaction characteristics of each obstacle, the spatiotemporal interaction characteristics of each obstacle are determined.

[0021] Optionally, the recurrent neural subnetwork is a first LSTM long short-term memory network;

[0022] Based on the positional characteristics, velocity characteristics, and coarse spatiotemporal interaction characteristics of each obstacle, the spatiotemporal interaction characteristics of each obstacle are determined, specifically including:

[0023] Based on the positional characteristics and coarse spatiotemporal interaction characteristics of each obstacle, determine the positional spatiotemporal interaction characteristics; based on the velocity characteristics and coarse spatiotemporal interaction characteristics of each obstacle, determine the velocity spatiotemporal interaction characteristics.

[0024] Based on the spatiotemporal interaction characteristics of location and velocity, the spatiotemporal interaction characteristics of each obstacle are determined.

[0025] Optionally, the second model specifically includes: an encoding end and a decoding end;

[0026] The spatiotemporal interaction features output by the first model are input into a pre-trained second model. The second model is then used to determine the predicted motion trajectories of each obstacle, specifically including:

[0027] The spatiotemporal interaction features output by the first model are input into the encoding end to obtain the encoded features output by the encoding end;

[0028] The encoded features are input into the decoding terminal, and the trajectory points of each obstacle at the target time are predicted through the decoding terminal.

[0029] Optionally, the historical motion trajectory of each obstacle is the trajectory point of each obstacle at each historical moment before the reference time;

[0030] The target time is a future time after the reference time;

[0031] The spatiotemporal interaction features output by the first model are the spatiotemporal interaction features at each historical moment.

[0032] Optionally, the encoding end is a second LSTM;

[0033] The spatiotemporal interaction features output by the first model are input into the encoding end to obtain the encoded features output by the encoding end, specifically including:

[0034] The spatiotemporal interaction features of each historical moment are input into the second LSTM to obtain the hidden layer features output by the hidden layer of the second LSTM based on the spatiotemporal interaction features of each historical moment, which are then used as the output encoding features of the encoding end.

[0035] Optionally, the encoded features are input into the decoding end, and the decoding end is used to predict the trajectory points of each obstacle at the target time, specifically including:

[0036] The encoded features and the spatiotemporal interaction features of the previous historical moment of the reference moment are input into the decoding end, and the trajectory points of each obstacle at the target moment are predicted through the decoding end.

[0037] Optionally, the decoding end is a third LSTM;

[0038] After predicting the trajectory points of each obstacle at the target time, the method further includes:

[0039] The reference time is redefined, and the target time is redefined based on the redefined reference time.

[0040] The hidden layer features output from the previous historical moment of the redefined reference time, and the spatiotemporal interaction features of the previous historical moment of the redefined reference time are input into the decoding end, so as to predict the trajectory points of each obstacle at the redefined target time through the decoding end.

[0041] Optionally, the decoding end predicts the trajectory points of each obstacle at the target time, specifically including:

[0042] For each obstacle, the spatiotemporal interaction vector corresponding to the obstacle in the spatiotemporal interaction features of the previous historical moment of the reference time is concatenated with a number of predetermined random noises, and the concatenated spatiotemporal interaction vectors are input into the decoding end.

[0043] For each spliced ​​spatiotemporal interaction vector, based on the spatiotemporal interaction vector and the encoding features output by the encoding end, the trajectory point of the obstacle at the target time is determined by the decoding end.

[0044] Optionally, the pre-training of the first and second models specifically includes:

[0045] Identify the obstacles in each sample and the corresponding sample trajectory for each obstacle;

[0046] Based on a pre-set reference time, for each sample obstacle, the trajectory before the reference time in the sample trajectory corresponding to the sample obstacle is taken as the initial trajectory of the sample obstacle, and the trajectory after the reference time is taken as the labeled trajectory of the sample obstacle.

[0047] For each sample obstacle, the initial trajectory of the sample obstacle is input into the first model, and the motion features of the sample obstacle are extracted through the feature extraction sub-network of the first model.

[0048] The motion features of each sample obstacle are input into the attention subnetwork of the first model to obtain the pairwise interaction weights between each sample obstacle.

[0049] The determined interaction weights are used to weight the motion characteristics of each sample obstacle, and the weighted motion characteristics of each sample obstacle are used as the spatial interaction characteristics of each sample obstacle.

[0050] The spatial interaction features of each sample obstacle are input into the recurrent neural subnetwork of the first model to obtain the spatiotemporal interaction features of each sample obstacle.

[0051] The spatiotemporal interaction features output by the first model are input into the second model, and the predicted trajectory of each obstacle is determined through the second model.

[0052] The parameters in the first and second models are adjusted with the goal of minimizing the difference between the predicted trajectory and the labeled trajectory of each obstacle.

[0053] Optionally, the predicted trajectory of each obstacle is determined using the second model, specifically including:

[0054] For each sample obstacle, the second model is used to determine several predicted trajectories for that sample obstacle;

[0055] With the goal of minimizing the difference between the predicted trajectory and the labeled trajectory of each obstacle, the parameters in the first and second models are adjusted, specifically including:

[0056] For each sample obstacle, the predicted trajectory that differs least from the labeled trajectory of that sample obstacle is determined as the target trajectory of that sample obstacle;

[0057] The parameters in the first and second models are adjusted with the goal of minimizing the difference between the target trajectory and the labeled trajectory of each obstacle.

[0058] This specification provides an obstacle trajectory prediction device, comprising:

[0059] The trajectory determination module is used to determine the historical movement trajectory of each obstacle.

[0060] The feature extraction module is used to input the historical motion trajectory of each obstacle within a preset time period into a pre-trained first model, and extract the motion features of the obstacle through the feature extraction sub-network of the first model.

[0061] The attention weight module is used to input the motion features of each obstacle into the attention subnetwork of the first model to obtain the interaction weights between each pair of obstacles.

[0062] The spatial feature module is used to weight the motion features of each obstacle based on the determined interaction weights, and then use the weighted motion features of each obstacle as the spatial interaction features of each obstacle.

[0063] The spatiotemporal feature module is used to input the spatial interaction features of each obstacle into the recurrent neural subnetwork of the first model to obtain the spatiotemporal interaction features of each obstacle.

[0064] The trajectory prediction module is used to input the spatiotemporal interaction features output by the first model into a pre-trained second model, and through the second model, determine the predicted motion trajectory of each obstacle.

[0065] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described obstacle trajectory prediction method.

[0066] This specification provides an unmanned driving device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described obstacle trajectory prediction method.

[0067] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0068] Based on the motion features extracted from the historical motion trajectories of each obstacle, the interaction weights between each pair of obstacles are determined. The corresponding motion features are then weighted with the determined interaction weights to obtain the weighted spatial interaction features. The recurrent neural subnetwork obtains the spatiotemporal interaction features of each obstacle based on the input spatial interaction features. The pre-trained second model determines the predicted motion trajectory of each obstacle based on the spatiotemporal interaction features of each obstacle. Here, the interaction weights reflect the degree of influence between each obstacle and each other obstacle. Therefore, the spatial interaction features not only depict the motion features but also the interactions between obstacles, resulting in higher accuracy in predicting the future trajectories of obstacles. Attached Figure Description

[0069] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0070] Figure 1 This is a flowchart illustrating a trajectory prediction method for an obstacle as described in this specification.

[0071] Figure 2 This is a schematic diagram of the structure of a trajectory prediction model in this specification;

[0072] Figure 3 This is a schematic diagram of the structure of the second model in a trajectory prediction model described in this specification;

[0073] Figure 4 This is a schematic diagram of a method for re-determining the target time in this specification;

[0074] Figure 5 This is a schematic diagram illustrating a method for selecting hidden layer features in the decoding end, as described in this specification.

[0075] Figure 6A and Figure 6B This is a schematic diagram illustrating two methods for predicting several motion trajectories for obstacles as described in this specification.

[0076] Figure 7 This is a flowchart illustrating a trajectory prediction model training method described in this specification.

[0077] Figure 8 A schematic diagram of an obstacle trajectory prediction device provided in this specification;

[0078] Figure 9 The corresponding information provided in this specification Figure 1 A schematic diagram of an unmanned driving device. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0080] In the field of autonomous driving technology, in order to guide the decision-making, planning and control of autonomous driving equipment, it is necessary to predict the future trajectories of various obstacles around the autonomous driving equipment, so as to help the autonomous driving equipment to avoid each obstacle based on the predicted future trajectories of each obstacle.

[0081] The obstacle trajectory prediction method provided in this specification is implemented using a corresponding trajectory prediction model, which outputs the predicted motion trajectory of each obstacle based on the historical motion trajectory of each obstacle.

[0082] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0083] Example 1:

[0084] Figure 1 This is a flowchart illustrating a trajectory prediction method for an obstacle as described in this specification, which specifically includes the following steps:

[0085] S100: Determine the historical trajectory of each obstacle.

[0086] The trajectory prediction method provided in this specification can be executed by an unmanned driving device (hereinafter referred to as an autonomous vehicle), or by an electronic device capable of transmitting information with or controlling the autonomous vehicle, such as a laptop, mobile phone, or server. This specification does not impose any limitations on this. For ease of description, this specification uses an autonomous vehicle as the executing entity to illustrate the trajectory prediction method provided in this specification.

[0087] The unmanned vehicles described in this specification may include autonomous vehicles and vehicles with driver assistance functions. Unmanned vehicles may be delivery vehicles used in the delivery field.

[0088] When planning the trajectory of an autonomous vehicle, obstacles are typically considered to be those surrounding the vehicle's location, such as obstacles 20 meters away. Of course, any distance can be considered as the vehicle's surroundings. Since the vehicle's driving tendency is known, obstacles in that direction (e.g., obstacles in front of the vehicle) can also be considered. Alternatively, obstacles in the vehicle's environment can be considered as surroundings, such as obstacles on the same road as the vehicle. It can be seen that there are various existing methods for selecting obstacles, and this specification does not limit this approach.

[0089] It should be noted that, on the one hand, the autonomous vehicle itself travels under the guidance of a planned trajectory, so the trajectory of the autonomous vehicle is known and does not need to be predicted, and the autonomous vehicle itself does not need to be regarded as an obstacle; on the other hand, since the autonomous vehicle also participates in traffic, the movement of the autonomous vehicle will affect the future movement of other obstacles around the autonomous vehicle, that is, it will interact with other obstacles, so the autonomous vehicle itself can also be regarded as an obstacle, but this specification does not limit this.

[0090] Based on whether an obstacle moves actively, it can be divided into static obstacles and dynamic obstacles. Static obstacles usually do not move actively and their positions do not change over time, such as utility poles and streetlights. Dynamic obstacles, on the other hand, refer to obstacles that can participate in traffic, such as vehicles and pedestrians. Because the positions of dynamic obstacles change over time and there is interaction between dynamic obstacles, the movement of one dynamic obstacle can affect the movement of other dynamic obstacles.

[0091] In the obstacle trajectory prediction method provided in this specification, based on the aforementioned characteristics of dynamic obstacles, the obstacles whose trajectories are to be predicted must include dynamic obstacles. As for static obstacles, on the one hand, their position does not change with time, so there is no movement trajectory. On the other hand, the existence of static obstacles will unilaterally affect the movement of dynamic obstacles. That is to say, dynamic obstacles will actively avoid static obstacles. Therefore, static obstacles can also be regarded as obstacles referred to in this specification, and the fixed position of the static obstacle can be taken as the trajectory point of the static obstacle that does not change with time. Alternatively, the influence of static obstacles on dynamic obstacles can be considered separately. For example, the predicted movement trajectory of the dynamic obstacle can be constrained not to pass through the position of the static obstacle, but the predicted trajectory of the static obstacle is not output. This specification does not impose any restrictions on this. For the sake of convenience, the obstacle referred to below is only a dynamic obstacle, which can be included in the obstacle trajectory prediction method provided in this specification.

[0092] Since the obstacles are around the autonomous vehicle, the motion information of each obstacle over a historical period can be collected by the sensors mounted on the autonomous vehicle. Of course, the motion information of each obstacle over a historical period can also be collected by the sensors mounted in other locations, and then the corresponding terminal device or server can send the motion information to the autonomous vehicle.

[0093] Based on the historical motion information of each obstacle, its historical trajectory can be determined. This historical trajectory may include not only the path taken by each obstacle, but also when, in what state, and where along the historical trajectory each obstacle was. Furthermore, if the historical trajectory is not a smooth curve, but rather a line connecting several sequential trajectory points, then the historical trajectory can include: each trajectory point, the time taken for the corresponding obstacle to pass through each trajectory point, and the obstacle's state of travel (e.g., speed, acceleration) when passing through each trajectory point.

[0094] The following steps S102~S110 can be performed as follows: Figure 2 The trajectory prediction model shown is used to achieve this.

[0095] S102: For each obstacle, input the historical motion trajectory of the obstacle within a preset time period into the pre-trained first model, and extract the motion features of the obstacle through the feature extraction sub-network of the first model.

[0096] Since the time spans of the acquired historical motion trajectories of each obstacle may differ, for ease of comparison, the historical motion trajectories of each obstacle within the same preset time period can be extracted. This ensures that the historical motion trajectories occurred within the same historical time period, providing a possibility for characterizing the interactions of each obstacle. For example, a reference time can be determined, and then the historical motion trajectories within the preset time period before the reference time can be extracted. Typically, the current time when the autonomous vehicle executes this method can be used as the reference time. The following description uses the historical motion trajectory of the input trajectory prediction model as an example of the historical motion trajectory within the preset time period before the reference time to illustrate the method provided in this specification.

[0097] The captured historical motion trajectories of each obstacle are input into a pre-trained first model. The feature extraction sub-network of the first model obtains the motion features of each obstacle, which characterize the motion state of each obstacle within a preset historical time period. It is understood that, for computational simplicity, the input historical motion trajectory may not be smooth and continuous, but rather may consist of trajectory points corresponding to several historical moments within a preset time period prior to the reference moment. In this case, the output of the feature extraction sub-network can be the motion features of each obstacle at each corresponding historical moment. For ease of description, this will be used as an example below.

[0098] Generally, due to the need for real-time control of autonomous vehicles and the requirement for the versatility of this method, the motion information collected by the sensing device can be preprocessed before being input into the feature extraction subnetwork. This allows the trajectory prediction method provided in this specification to be executed with minimal computation at different times and when the autonomous vehicle travels to different locations. For example, for position information, if the position coordinates obtained by the sensing device are in a geodetic coordinate system, the position coordinates of each obstacle in this geodetic coordinate system can be converted into coordinates in a relative coordinate system that is easier to calculate. For example, the offset of each obstacle relative to the center point can be used as the coordinates of each obstacle. The center point can be the position coordinates of the autonomous vehicle at the corresponding historical moment, or it can be the average coordinates of each obstacle at each historical moment. Of course, if the coordinates of each obstacle obtained by the sensing device are coordinates with minimal computation, such as the coordinates of each obstacle relative to the autonomous vehicle, then there is no need to preprocess the motion information before inputting it into the feature extraction subnetwork.

[0099] S104: Input the motion features of each obstacle into the attention subnetwork of the first model to obtain the interaction weights between each pair of obstacles.

[0100] S106: The determined interaction weights are used to weight the motion characteristics of each obstacle, and the weighted motion characteristics of each obstacle are used as the spatial interaction characteristics of each obstacle.

[0101] Since the motion features input into the attention subnetwork can reflect the interactions between obstacles, the attention subnetwork can determine the interaction weights of each obstacle based on the input motion features. This reflects the degree of mutual influence between each pair of obstacles, i.e., the tightness of the interaction. This is because obstacles themselves adjust their positions under the influence of other obstacles, which is the interaction mentioned above. For an obstacle, the influence of other obstacles varies in magnitude. For example, the closer an obstacle is to the obstacle, the greater the probability of collision. Therefore, the obstacle will usually prioritize avoiding closer obstacles. In this case, it can be considered that the interaction between the obstacle and the closer the obstacle is, the greater the influence it receives, and the more attention it invests.

[0102] Specifically, the degree to which the input motion features reflect the movement of obstacles determines the degree of interaction between them. For example, when the input is the positional features of obstacles at various historical moments, these features themselves indicate which obstacles interact. Since the extracted positional features of the obstacles are all located around the autonomous vehicle, there is interaction between them. The interaction weights determined by the attention sub-network can further measure the degree of interaction between pairs of obstacles based on their proximity. Generally, the closer two obstacles are, the more decisive their actions are for the obstacles; that is, the closer the interaction and the higher the attention level, the greater the determined interaction weights between them. For simplicity, the following explanation will use the positional features of obstacles at various historical moments as the input to the attention sub-network.

[0103] For each historical moment, the motion characteristics of each obstacle are weighted. The weighted motion characteristics reflect the degree of interaction between the obstacle and each other obstacle. Therefore, by weighting the motion characteristics of each obstacle at that historical moment, the resulting weighted motion characteristics of each obstacle can be considered to reflect the degree of interaction between each obstacle at that historical moment, that is, the spatial interaction characteristics of each obstacle at that historical moment.

[0104] S108: Input the spatial interaction features of each obstacle into the recurrent neural subnetwork of the first model to obtain the spatiotemporal interaction features of each obstacle.

[0105] At each historical moment, the spatial interaction features of each obstacle can reflect the interaction characteristics of each obstacle at that historical moment. However, it is difficult to accurately predict the future trajectory of each obstacle based solely on the isolated spatial interaction features of each obstacle at each historical moment. Therefore, in this specification, the spatial interaction features of each obstacle at each historical moment can be input into a recurrent neural network (RNN). Because the output of a recurrent neural network at the previous moment is fed back into the hidden layer of the recurrent neural network at the next moment, affecting the output at the next moment, this characteristic of the recurrent neural network can be used to obtain the spatiotemporal interaction features of each obstacle at each historical moment. Specifically, the spatial interaction features of each obstacle at each historical moment in the time series can be input into the recurrent neural network. The features output by the recurrent neural network at each historical moment actually contain the features from the spatial interaction information of all previous historical moments. That is, the features represent the relationship between the spatial interaction features in time sequence and can be used as the spatiotemporal interaction features of each obstacle before the corresponding historical moment. Any recurrent neural network can be selected as the recurrent neural network in this specification, such as a Long Short-Term Memory (LSTM) network; this specification does not impose any restrictions on this.

[0106] Furthermore, when the attention network is only input with positional features—that is, when the aforementioned spatiotemporal interaction features are determined solely based on the positional information of obstacles at various historical moments—other features extracted from the trajectory information can be used to enrich the information represented by these spatiotemporal interaction features. Specifically, other features contained in the historical motion trajectory, such as velocity features, can be extracted. The spatiotemporal interaction features output by the recurrent neural subnetwork can be used as coarse spatiotemporal interaction features. The information represented by these coarse features can be enriched based on the extracted other features, and the enriched coarse spatiotemporal interaction features can then be used as the spatiotemporal interaction features. Specifically, positional spatiotemporal interaction features can be determined based on the positional features and coarse spatiotemporal interaction features of each obstacle; velocity spatiotemporal interaction features can be determined based on the velocity features and coarse spatiotemporal interaction features of each obstacle; and spatiotemporal interaction features of each obstacle can be determined based on both positional and velocity spatiotemporal interaction features.

[0107] This recurrent neural network can output the spatiotemporal interaction features of the corresponding historical moment based on the spatial interaction features of each historical moment input. For each historical moment output, the spatiotemporal interaction features reflect the changes in the spatial interaction features of each obstacle over time before that historical moment.

[0108] S110: Input the spatiotemporal interaction features output by the first model into the pre-trained second model, and determine the predicted motion trajectory of each obstacle through the second model.

[0109] By inputting the spatiotemporal interaction features of each historical moment into a pre-trained second model, the predicted motion trajectory of each obstacle can be determined through this second model. The following explanation uses the spatiotemporal interaction features of each obstacle at each historical moment as an example, with the spatiotemporal interaction features of each obstacle output from the first model to the second model.

[0110] like Figure 1 As shown in the specification, this method provides a trajectory prediction method for obstacles. Based on the motion features extracted from the historical motion trajectories of each obstacle, the interaction weights between each pair of obstacles are determined. The motion features are then weighted according to the determined interaction weights to obtain weighted spatial interaction features. A recurrent neural subnetwork obtains the spatiotemporal interaction features of each obstacle based on the input spatial interaction features. A pre-trained second model determines the predicted motion trajectory of each obstacle based on the spatiotemporal interaction features. The interaction weights reflect the degree of influence between each obstacle and each other obstacle. Therefore, the spatial interaction features not only depict the motion features but also the interaction between obstacles, resulting in higher accuracy in predicting the future trajectories of obstacles.

[0111] Furthermore, one could adopt... Figure 3 The second model shown determines the predicted motion trajectory of each obstacle based on the spatiotemporal interaction characteristics of the input.

[0112] like Figure 3 As shown, the second model includes an encoder and a decoder. Spatiotemporal interaction features are input to the encoder, which encodes the received features and inputs the encoded features to the decoder. The decoder predicts the trajectory of the obstacle at the target time based on the input encoded features. The target time is a future time after the reference time; that is, the second model determines the predicted trajectory of each obstacle by predicting the trajectory at the target time. The output of this trajectory prediction model can be either the predicted trajectory at the target time or the predicted travel information (e.g., speed and direction) of each obstacle between the previous historical time and the target time. The trajectory at the target time is determined based on the acquired position information of each obstacle at the previous historical time. This specification does not limit the form of the output prediction results.

[0113] Specifically, the encoding end can be an LSTM, which encodes the spatiotemporal interaction features of each historical moment and obtains encoded features. The spatiotemporal interaction features output by the first model are the spatiotemporal interaction features of each obstacle at each historical moment in the time series. Specifically, when each historical moment is input into the encoding end sequentially according to the time series, the output of the encoding end, using the spatiotemporal interaction features of the historical moment closest to the reference moment as input, can be considered to contain information about the spatiotemporal interaction features of each historical moment. Therefore, this output can be used as the encoded features output by the encoding end. Of course, the hidden layer features output by the encoding end, using the spatiotemporal interaction features of the historical moment closest to the reference moment as input, can also be considered to carry information about the spatiotemporal interaction features of each historical moment; therefore, these hidden layer features can also be used as the encoded features output by the encoding end.

[0114] Generally speaking, in the execution of autonomous vehicles such as Figure 1 The method shown can predict the trajectory points of each obstacle at several target times, thereby obtaining the predicted motion trajectory of each obstacle over a future period. Specifically, after each prediction of the trajectory points of each obstacle at the target time at the decoding end, a new reference time can be determined, and the method can be applied... Figure 1 The method in steps S100 to S110 determines the spatiotemporal interaction characteristics of the previous historical moment of the redefined reference time based on the trajectory points of the previous historical moment of the redefined reference time, and predicts the trajectory points of each obstacle at the redefined target time based on the spatiotemporal interaction characteristics. This process is repeated, and the predicted trajectory of each obstacle passing through the predicted trajectory points is determined.

[0115] like Figure 4 As shown, the decoder first uses the encoded features output by the encoder and the spatiotemporal interaction features of historical time 2 as input to predict the trajectory points of each obstacle at target time 1. Then, it redetermines the reference time. After the reference time is redetermined, target time 1 becomes historical time 3. Figure 1 The method in steps S100 to S110 determines the spatiotemporal interaction characteristics of historical time 3, and predicts the trajectory points of each obstacle at the target time 2 based on the spatiotemporal interaction characteristics of historical time 3.

[0116] The aforementioned decoding end can be an LSTM, which, after executing once, ... Figure 1The method shown involves the decoder predicting the trajectory points at the first target time. It can use the hidden features output by the encoder as its initial hidden features and the spatiotemporal interaction features of the most recent historical time (the reference time) as input to predict the trajectory points of each obstacle at the first target time. After predicting the trajectory points at the first target time, the decoder can pass its previously output hidden features to the prediction of the trajectory points at the newly determined target time, using the spatiotemporal interaction features of the previous historical time (the reference time) as input to predict the trajectory points of each obstacle at the newly determined target time.

[0117] by Figure 5 For example, in Figure 5 In the process, the spatiotemporal interaction features x1 of the previous historical moment of the input reference time are input to the decoder. When the decoder predicts the trajectory points of the first target moment, the encoded features h0 output by the encoder (i.e., the hidden features output by the encoder) are used as the initial hidden features of the decoder, so that the decoder can predict the trajectory points of each obstacle at the first target moment based on h0 and x1 (the predicted trajectory points are in...). Figure 5 (not shown in the image), and outputs the hidden layer feature h1 at the first target time. After re-determining the reference time, when the decoder predicts the trajectory points of each obstacle at the second target time, the hidden layer feature is the hidden layer feature h1 output by the decoder itself at the first target time, and so on.

[0118] However, since obstacles in reality exhibit various movement tendencies, a single predicted trajectory cannot accurately depict the diverse motion tendencies of an obstacle when facing a complex environment with different paths. Therefore, the obstacle trajectory prediction method provided in this specification can predict several trajectories for each obstacle. Specifically, this can be achieved by concatenating the spatiotemporal interaction features input from the decoding end with random noise, enabling the decoding end to output several trajectory points based on a single input spatiotemporal interaction feature. Figure 6A and Figure 6B This illustrates a method that uses random noise to concatenate spatiotemporal interactive features, in the execution of a single... Figure 1 The methods shown are two approaches to predicting the trajectory points of an obstacle at two target times, thereby obtaining four predicted trajectories for the obstacle. Here, t0 is the previous historical time before the reference time, and t1 and t2 are the two target times for the predicted trajectory points, respectively.

[0119] exist Figure 6AIn the process, when the decoding end predicts the trajectory point of an obstacle at time t1 based on the spatiotemporal interaction characteristics of each obstacle at time t0, it can concatenate the spatiotemporal interaction characteristics at time t0 with four pre-determined random noises respectively, and predict the four trajectory points p1~p4 of the obstacle at time t1 based on the concatenated spatiotemporal interaction characteristics at time t0 respectively. When predicting the trajectory point of the obstacle at time t2, it can choose not to concatenate the spatiotemporal interaction characteristics at time t1 corresponding to the four trajectory points at time t1 with random noises, and thus obtain the four predicted trajectory points p5~p8 of the obstacle at time t2, thereby predicting the four predicted trajectories of the obstacle from t0 to t2.

[0120] And in Figure 6B In the process, when the decoding end predicts the trajectory point of an obstacle at time t1 based on the spatiotemporal interaction characteristics of each obstacle at time t0, it can concatenate the spatiotemporal interaction characteristics at time t0 with two pre-determined random noises respectively, and predict the two trajectory points p1 and p2 of the obstacle at time t1 based on the concatenated spatiotemporal interaction characteristics at time t0 respectively. When predicting the trajectory point of the obstacle at time t2, the spatiotemporal interaction characteristics at time t1 corresponding to p1 and p2 are concatenated with the two random noises respectively. Thus, using the spatiotemporal interaction characteristics at time t1 corresponding to p1 and p2 after concatenating random noises, the decoding end predicts the four trajectory points p3 to p6 of the obstacle at time t2, thereby predicting the four predicted trajectories of the obstacle from t0 to t2.

[0121] It can be seen that the number of predicted trajectories that can be determined for each obstacle is determined by the number and time of the spliced ​​random noise. Random noise can be spliced ​​into any spatiotemporal interaction feature using any existing method, which will not be elaborated here.

[0122] Example 2:

[0123] Figure 7 The training provided for the embodiments of this specification is as follows Figure 2 The flowcharts of the first and second models shown include:

[0124] S700: Determine the obstacles in each sample and the corresponding sample trajectory for each obstacle.

[0125] S702: Based on a pre-set reference time, for each sample obstacle, the trajectory before the reference time in the sample trajectory corresponding to the sample obstacle is taken as the initial trajectory of the sample obstacle, and the trajectory after the reference time is taken as the labeled trajectory of the sample obstacle.

[0126] Generally speaking, sample trajectories are the historical real trajectories of each obstacle collected in advance. Specifically, sensing devices can be deployed in the real environment beforehand, and the obstacles collected can be selected as sample obstacles. The real trajectories of the sample obstacles are then used as the sample trajectories for each sample obstacle. Since sample trajectories are usually the real trajectories collected within the same target time period, it is often unnecessary to extract them into trajectories within a preset time period after collection.

[0127] The sample trajectories of each obstacle within the target time period are divided by a pre-set reference time. The trajectory before the reference time is taken as the initial trajectory of the obstacle, and the trajectory after the reference time is taken as the labeled trajectory of the obstacle.

[0128] S704: For each sample obstacle, the initial trajectory of the sample obstacle is input into the first model, and the motion features of the sample obstacle are extracted through the feature extraction sub-network of the first model.

[0129] S706: Input the motion features of each sample obstacle into the attention subnetwork of the first model to obtain the pairwise interaction weights between each sample obstacle.

[0130] S708: The determined interaction weights are used to weight the motion characteristics of each sample obstacle, and the weighted motion characteristics of each sample obstacle are used as the spatial interaction characteristics of each sample obstacle.

[0131] S710: Input the spatial interaction features of each sample obstacle into the recurrent neural subnetwork of the first model to obtain the spatiotemporal interaction features of each sample obstacle.

[0132] S712: Input the spatiotemporal interaction features output by the first model into the second model, and determine the predicted trajectory of each sample obstacle through the second model.

[0133] Adopted and Figure 1 Using the same method, input the initial trajectory of each obstacle as follows: Figure 2 The trajectory prediction model shown outputs the predicted trajectory of each obstacle in the sample.

[0134] S714: Adjust the parameters in the first and second models with the goal of minimizing the difference between the predicted trajectory and the labeled trajectory of each obstacle.

[0135] With the goal of minimizing the difference between the predicted trajectory and the labeled trajectory of each sample obstacle, the parameters in the first and second models are adjusted. Furthermore, when the trajectory prediction model outputs several predicted trajectories for each sample obstacle, the parameters can be adjusted with the goal of minimizing the difference between all predicted trajectories and the labeled trajectory of each sample obstacle. Alternatively, for each sample obstacle, a predicted trajectory that is closest to the labeled trajectory of that sample obstacle can be selected from the predicted trajectories determined for that sample obstacle and used as the target trajectory. When adjusting the parameters, the goal is to minimize the difference between the target trajectory and the labeled trajectory of each sample obstacle, and the parameters in the first and second models are adjusted accordingly.

[0136] The above are exemplary methods for predicting the trajectory of obstacles provided in this specification. Based on the same idea, this specification also provides corresponding devices, storage media, and unmanned driving equipment.

[0137] Figure 8 This is a schematic diagram of the structure of an obstacle trajectory prediction device provided in an embodiment of this specification. The device includes:

[0138] The trajectory determination module 800 is used to determine the historical movement trajectory of each obstacle.

[0139] The feature extraction module 802 is used to input the historical motion trajectory of each obstacle within a preset time period into a pre-trained first model for each obstacle, and extract the motion features of the obstacle through the feature extraction sub-network of the first model.

[0140] The attention weight module 804 is used to input the motion features of each obstacle into the attention subnetwork of the first model to obtain the interaction weights between each pair of obstacles.

[0141] The spatial feature module 806 is used to weight the motion features of each obstacle with the determined interaction weights, and to use the weighted motion features of each obstacle as the spatial interaction features of each obstacle.

[0142] The spatiotemporal feature module 808 is used to input the spatial interaction features of each obstacle into the recurrent neural subnetwork of the first model to obtain the spatiotemporal interaction features of each obstacle.

[0143] The trajectory prediction module 810 is used to input the spatiotemporal interaction features output by the first model into a pre-trained second model, and through the second model, determine the predicted motion trajectory of each obstacle.

[0144] Optionally, the feature extraction module 802 is specifically used to extract the position and velocity features of the obstacle; the attention weight module 804 is specifically used to input the position features of each obstacle into the attention subnetwork of the first model; the spatiotemporal feature module 808 is specifically used to input the spatial interaction features of each obstacle into the recurrent neural subnetwork of the first model to obtain the coarse spatiotemporal interaction features of each obstacle; and the spatiotemporal interaction features of each obstacle are determined based on the position features, velocity features, and coarse spatiotemporal interaction features of each obstacle.

[0145] Optionally, the recurrent neural subnetwork is a first LSTM long short-term memory network; the spatiotemporal feature module 808 is specifically used to determine the position spatiotemporal interaction features based on the position features and coarse spatiotemporal interaction features of each obstacle; determine the velocity spatiotemporal interaction features based on the velocity features and coarse spatiotemporal interaction features of each obstacle; and determine the spatiotemporal interaction features of each obstacle based on the position spatiotemporal interaction features and the velocity spatiotemporal interaction features.

[0146] Optionally, the second model specifically includes an encoder and a decoder; the trajectory prediction module 810 is specifically used to input the spatiotemporal interaction features output by the first model into the encoder to obtain the encoded features output by the encoder; input the encoded features into the decoder, and predict the trajectory points of each obstacle at the target time through the decoder.

[0147] Optionally, the historical motion trajectory of each obstacle is the trajectory point of each obstacle at each historical moment before the reference moment; the target moment is the future moment after the reference moment; and the spatiotemporal interaction features output by the first model are the spatiotemporal interaction features of each historical moment.

[0148] Optionally, the encoding end is a second LSTM; the trajectory prediction module 810 is specifically used to input the spatiotemporal interaction features of each historical moment into the second LSTM, and obtain the hidden layer features output by the hidden layer in the second LSTM according to the spatiotemporal interaction features of each historical moment, and use them as the encoding features output by the encoding end.

[0149] Optionally, the trajectory prediction module 810 is specifically used to input the encoded features and the spatiotemporal interaction features of the previous historical moment of the reference moment into the decoding end, and predict the trajectory points of each obstacle at the target moment through the decoding end.

[0150] Optionally, the decoding end is a third LSTM; the feature extraction module 802 is specifically used to redetermine the reference time and redetermine the target time based on the redetermined reference time; the trajectory prediction module 810 is specifically used to input the hidden layer features output by the decoding end of the previous historical time of the redetermined reference time, and the spatiotemporal interaction features of the previous historical time of the redetermined reference time into the decoding end, so as to predict the trajectory points of each obstacle at the redetermined target time through the decoding end.

[0151] Optionally, the feature extraction module 802 is specifically configured to, for each obstacle, concatenate the spatiotemporal interaction vector corresponding to that obstacle in the spatiotemporal interaction features of the obstacle in the previous historical moment at the reference time with a predetermined number of random noises, and input the concatenated spatiotemporal interaction vectors into the decoding end; for each concatenated spatiotemporal interaction vector, based on the spatiotemporal interaction vector and the encoded features output by the encoding end, determine the trajectory point of the obstacle at the target time through the decoding end.

[0152] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The provided method for predicting the trajectory of obstacles.

[0153] This instruction manual also provides Figure 9 The diagram shows a schematic structural representation of an unmanned driving device. Figure 9 At the hardware level, the page loading device includes a processor, internal bus, network interface, memory, and non-volatile storage, and may also include other hardware required for the business logic. The processor reads the corresponding computer program from the non-volatile storage into memory and then runs it to achieve the above-mentioned functionality. Figure 1 The method for predicting the trajectory of obstacles described herein. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0154] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0155] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0156] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0157] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0158] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0159] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0160] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0161] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0162] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0163] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0164] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0165] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0166] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0168] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0169] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for predicting the trajectory of an obstacle, characterized in that, include: Determine the historical trajectory of each obstacle; For each obstacle, the historical motion trajectory of the obstacle within a preset time period is input into a pre-trained first model, and the motion features of the obstacle are extracted through the feature extraction sub-network of the first model. The motion features of each obstacle are input into the attention subnetwork of the first model to obtain the interaction weights between each pair of obstacles. The determined interaction weights are used to weight the motion characteristics of each obstacle, and the weighted motion characteristics of each obstacle are used as the spatial interaction characteristics of each obstacle. The spatial interaction features of each obstacle are input into the recurrent neural subnetwork of the first model to obtain the spatiotemporal interaction features of each obstacle. The recurrent neural subnetwork is a first LSTM long short-term memory network. The spatiotemporal interaction features output by the first model are input into the encoding end of the pre-trained second model to obtain the encoded features output by the encoding end, wherein the encoding end is a second LSTM; The encoded features are input into the decoding end of the second model, and the trajectory points of each obstacle at the target time are predicted through the decoding end. The historical trajectory of each obstacle is the trajectory point of each obstacle at each historical moment before the reference time; The target time is a future time after the reference time; The spatiotemporal interaction features output by the first model are the spatiotemporal interaction features at each historical moment. The spatiotemporal interaction features output by the first model are input into the encoding end to obtain the encoded features output by the encoding end, specifically including: The spatiotemporal interaction features of each historical moment are input into the second LSTM to obtain the hidden layer features output by the hidden layer of the second LSTM based on the spatiotemporal interaction features of each historical moment, which are then used as the output encoding features of the encoding end.

2. The method as described in claim 1, characterized in that, Extracting the motion characteristics of the obstacle, specifically including: Extract the position and velocity features of the obstacle; The motion features of each obstacle are input into the attention subnetwork of the first model, specifically including: The positional features of each obstacle are input into the attention subnetwork of the first model; The spatial interaction features of each obstacle are input into the recurrent neural subnetwork of the first model to obtain the spatiotemporal interaction features of each obstacle, specifically including: The spatial interaction features of each obstacle are input into the recurrent neural subnetwork of the first model to obtain the coarse spatiotemporal interaction features of each obstacle. Based on the positional characteristics, velocity characteristics, and coarse spatiotemporal interaction characteristics of each obstacle, the spatiotemporal interaction characteristics of each obstacle are determined.

3. The method as described in claim 2, characterized in that, Based on the positional characteristics, velocity characteristics, and coarse spatiotemporal interaction characteristics of each obstacle, the spatiotemporal interaction characteristics of each obstacle are determined, specifically including: Based on the positional characteristics and coarse spatiotemporal interaction characteristics of each obstacle, determine the positional spatiotemporal interaction characteristics; based on the velocity characteristics and coarse spatiotemporal interaction characteristics of each obstacle, determine the velocity spatiotemporal interaction characteristics. Based on the spatiotemporal interaction characteristics of location and velocity, the spatiotemporal interaction characteristics of each obstacle are determined.

4. The method as described in claim 1, characterized in that, The encoded features are input into the decoding terminal, and the decoding terminal predicts the trajectory points of each obstacle at the target time, specifically including: The encoded features and the spatiotemporal interaction features of the previous historical moment of the reference moment are input into the decoding end, and the trajectory points of each obstacle at the target moment are predicted through the decoding end.

5. The method as described in claim 4, characterized in that, The decoding end is a third LSTM; After predicting the trajectory points of each obstacle at the target time, the method further includes: The reference time is redefined, and the target time is redefined based on the redefined reference time. The hidden layer features output from the previous historical moment of the redefined reference time, and the spatiotemporal interaction features of the previous historical moment of the redefined reference time are input into the decoding end, so as to predict the trajectory points of each obstacle at the redefined target time through the decoding end.

6. The method as described in claim 1, characterized in that, The decoding end predicts the trajectory points of each obstacle at the target time, specifically including: For each obstacle, the spatiotemporal interaction vector corresponding to the obstacle in the spatiotemporal interaction features of the previous historical moment of the reference time is concatenated with a number of predetermined random noises, and the concatenated spatiotemporal interaction vectors are input into the decoding end. For each spliced ​​spatiotemporal interaction vector, based on the spatiotemporal interaction vector and the encoding features output by the encoding end, the trajectory point of the obstacle at the target time is determined by the decoding end.

7. The method as described in any one of claims 1 to 6, wherein pre-training the first model and the second model specifically includes: Identify the obstacles in each sample and the corresponding sample trajectory for each obstacle; Based on a pre-set reference time, for each sample obstacle, the trajectory before the reference time in the sample trajectory corresponding to the sample obstacle is taken as the initial trajectory of the sample obstacle, and the trajectory after the reference time is taken as the labeled trajectory of the sample obstacle. For each sample obstacle, the initial trajectory of the sample obstacle is input into the first model, and the motion features of the sample obstacle are extracted through the feature extraction sub-network of the first model. The motion features of each sample obstacle are input into the attention subnetwork of the first model to obtain the pairwise interaction weights between each sample obstacle. The determined interaction weights are used to weight the motion characteristics of each sample obstacle, and the weighted motion characteristics of each sample obstacle are used as the spatial interaction characteristics of each sample obstacle. The spatial interaction features of each sample obstacle are input into the recurrent neural subnetwork of the first model to obtain the spatiotemporal interaction features of each sample obstacle. The spatiotemporal interaction features output by the first model are input into the second model, and the predicted trajectory of each obstacle is determined through the second model. The parameters in the first and second models are adjusted with the goal of minimizing the difference between the predicted trajectory and the labeled trajectory of each obstacle.

8. The method as described in claim 7, characterized in that, The second model is used to determine the predicted trajectory of each obstacle sample, specifically including: For each sample obstacle, the second model is used to determine several predicted trajectories for that sample obstacle; With the goal of minimizing the difference between the predicted trajectory and the labeled trajectory of each obstacle, the parameters in the first and second models are adjusted, specifically including: For each sample obstacle, the predicted trajectory that differs least from the labeled trajectory of that sample obstacle is determined as the target trajectory of that sample obstacle; The parameters in the first and second models are adjusted with the goal of minimizing the difference between the target trajectory and the labeled trajectory of each obstacle.

9. A trajectory prediction device for obstacles, characterized in that, include: The trajectory determination module is used to determine the historical movement trajectory of each obstacle. The feature extraction module is used to input the historical motion trajectory of each obstacle within a preset time period into a pre-trained first model, and extract the motion features of the obstacle through the feature extraction sub-network of the first model. The attention weight module is used to input the motion features of each obstacle into the attention subnetwork of the first model to obtain the interaction weights between each pair of obstacles. The spatial feature module is used to weight the motion features of each obstacle based on the determined interaction weights, and then use the weighted motion features of each obstacle as the spatial interaction features of each obstacle. The spatiotemporal feature module is used to input the spatial interaction features of each obstacle into the recurrent neural subnetwork of the first model to obtain the spatiotemporal interaction features of each obstacle. The recurrent neural subnetwork is a first LSTM long short-term memory network. The trajectory prediction module is used to input the spatiotemporal interaction features output by the first model into the encoding end of a pre-trained second model to obtain the encoded features output by the encoding end, wherein the encoding end is a second LSTM; the encoded features are input into the decoding end of the second model, and the trajectory points of each obstacle at the target time are predicted through the decoding end; The historical trajectory of each obstacle is the trajectory point of each obstacle at each historical moment before the reference time; The target time is a future time after the reference time; The spatiotemporal interaction features output by the first model are the spatiotemporal interaction features at each historical moment. The trajectory prediction module is specifically used to input the spatiotemporal interaction features of each historical moment into the second LSTM, obtain the hidden layer features output by the hidden layer in the second LSTM according to the spatiotemporal interaction features of each historical moment, and use them as the encoding features output by the encoding end.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 8.

11. An unmanned driving device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 8.

Images