Autonomous Driving Pedestrian Trajectory Prediction Method and Device

Abstract

An autonomous driving pedestrian trajectory prediction method and device. First, obtain the scene features of the target pedestrian, the current walked historical trajectory features of the target pedestrian, and other pedestrians around the target pedestrian. Second, determine the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian according to the scene features of the target pedestrian, the current walked trajectory of the target pedestrian, and the current walked trajectories of other pedestrians. Then, perform a fusion process on the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian to obtain the fusion feature information. Finally, predict the future walking trajectory of the target pedestrian according to the fusion feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and particularly to an autonomous driving pedestrian trajectory prediction method and apparatus. Background Art

[0002] Accurately predicting pedestrian trajectories is very important in fields such as intelligent transportation and smart cities. It can provide safe passage areas for driverless vehicles, intelligent robots, etc., and also provide important reference information for their route planning, target detection, and obstacle avoidance.

[0003] Some existing early pedestrian trajectory prediction studies mainly rely on traditional methods of manually establishing motion models, which rely on manual design. However, due to the very subjective and flexible activities of pedestrians, and when pedestrians are walking, they are not only dominated by their own intentions, but also affected by surrounding environmental obstacles and other pedestrians, resulting in complex and abstract interactions between pedestrians and obstacles, and between pedestrians and pedestrians. It is often very difficult to establish a reasonable dynamic model for people.

[0004] In recent years, some studies have emerged that use deep neural networks for pedestrian trajectory prediction. Early methods have a strong dependence on theory, and many are based on specific scenarios. While methods based on deep neural networks reduce the dependence on scenarios to a certain extent. Although such methods can predict pedestrian trajectories well to a certain extent and reduce collisions between pedestrians, however, such methods often fail to fully consider the correlation information between multiple features. In space, multiple features often do not exist independently, but there is a certain correlation, and the information of multiple features is often not fully utilized. Especially in real life, the interactions between pedestrians and scenarios, and between pedestrians and pedestrians are very complex and abstract, and these factors may lead to not very good accuracy in pedestrian trajectory prediction.

[0005] Comprehensive analysis of relevant literature shows that existing methods cannot meet the requirements of accurately predicting pedestrian trajectories and reducing collisions, and their reliability and robustness also need to be further improved. Summary of the Invention

[0006] The main technical problem to be solved by the present invention is how to more accurately predict pedestrian trajectories.

[0007] According to a first aspect, in one embodiment, an autonomous driving pedestrian trajectory prediction method is provided, including:

[0008] Obtaining scene features of a target pedestrian according to a scene image of the current walked scene of the target pedestrian;

[0009] Obtaining the current walked trajectories of the target pedestrian and other pedestrians around the target pedestrian;

[0010] Determine the historical trajectory feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information of the target pedestrian according to the scene features of the target pedestrian, the current traveled trajectory of the target pedestrian, and the current traveled trajectories of the other pedestrians;

[0011] Perform fusion processing on the historical trajectory feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information of the target pedestrian to obtain fusion feature information;

[0012] Predict the future walking trajectory of the target pedestrian according to the fusion feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information.

[0013] According to a second aspect, an embodiment provides an automatic driving pedestrian trajectory prediction device, including:

[0014] A scene feature acquisition module, configured to acquire the scene features of the target pedestrian according to the scene image of the currently traveled scene of the target pedestrian;

[0015] A historical walking trajectory acquisition module, configured to acquire the current traveled trajectories of the target pedestrian and other pedestrians around the target pedestrian;

[0016] An interaction feature information acquisition module, which determines the historical trajectory feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information of the target pedestrian according to the scene features of the target pedestrian, the current traveled trajectory of the target pedestrian, and the current traveled trajectories of the other pedestrians;

[0017] A fusion feature information acquisition module, which performs fusion processing on the historical trajectory feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information of the target pedestrian to obtain fusion feature information;

[0018] A prediction module, which predicts the future walking trajectory of the target pedestrian according to the fusion feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information.

[0019] According to a third aspect, an embodiment provides a pedestrian trajectory prediction device, characterized by including:

[0020] A memory, configured to store programs;

[0021] A processor, configured to implement the method as described in the above embodiment by executing the programs stored in the memory.

[0022] According to a fourth aspect, in one embodiment, a computer-readable storage medium is provided, characterized in that a program is stored on the medium, and the program can be executed by a processor to implement the method as described in the above embodiments.

[0023] For the automatic driving pedestrian trajectory prediction method and device according to the above embodiments, first, obtain the scene features of the target pedestrian, the current traveled trajectories of the target pedestrian and other pedestrians around the target pedestrian. Secondly, determine the historical trajectory feature information, pedestrian-scene interaction feature information and pedestrian-pedestrian interaction feature information of the target pedestrian according to the scene features of the target pedestrian, the current traveled trajectory of the target pedestrian and the current traveled trajectories of other pedestrians. Then, perform a fusion process on the historical trajectory feature information, pedestrian-scene interaction feature information and pedestrian-pedestrian interaction feature information of the target pedestrian to obtain fusion feature information. Finally, predict the future traveled trajectory of the target pedestrian according to the fusion feature information, pedestrian-scene interaction feature information and pedestrian-pedestrian interaction feature information. Brief Description of the Drawings

[0024] Figure 1 It is a flowchart of the automatic driving pedestrian trajectory prediction method according to one embodiment;

[0025] Figure 2 For Figure 1 It is a flowchart of step 103 in the pedestrian trajectory prediction method shown;

[0026] Figure 3 For Figure 1 It is a flowchart of step 104 in the pedestrian trajectory prediction method shown;

[0027] Figure 4 It is a schematic structural diagram of a pedestrian trajectory prediction device based on the processing of uncertain information according to one embodiment. Detailed Embodiments

[0028] The present invention will be further described in detail below in conjunction with the accompanying drawings through specific embodiments. Similar elements in different embodiments are labeled with related similar element numbers. In the following embodiments, many details are described to enable a better understanding of the present application. However, those skilled in the art can easily recognize that some of the features can be omitted in different situations, or can be replaced by other elements, materials, or methods. In some cases, some operations related to the present application are not shown or described in the specification to avoid the core part of the present application being overwhelmed by excessive description. For those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0029] In addition, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can also be reordered or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and drawings are only for clearly describing a certain embodiment and do not mean a necessary sequence, unless it is stated otherwise that a certain sequence must be followed.

[0030] The serial numbers assigned to the components herein, such as "first", "second", etc., are only used to distinguish the objects described and do not have any sequential or technical meaning. And the "connection" and "coupling" mentioned in this application, unless otherwise specified, both include direct and indirect connection (coupling).

[0031] In the research of pedestrian trajectory prediction, due to the subjective flexibility of pedestrian activities, and the interaction information between pedestrians and the scene as well as the interaction information between pedestrians, which is very complex and abstract, there are often a lot of uncertain information. The research on the processing of uncertain information is not only a research hotspot in the fields of object recognition and information fusion, but also very important in the fields such as autonomous driving. The historical trajectory information features of the target pedestrian in the scene space, the interaction feature information between the pedestrian and the scene, and the interaction feature information between pedestrians often do not exist independently, but there is a certain correlation information. Effectively modeling and representing and utilizing the uncertain information therein can improve the accuracy of pedestrian trajectory prediction, and then can help driverless vehicles, intelligent service robots, etc. better master the state information of pedestrians, so as to better perform tasks such as obstacle avoidance and path planning.

[0032] Traditional methods rely on manually designed dynamic models and are highly dependent on the scene. The deep learning-based methods overcome this scene dependence to a certain extent. However, when fusing multiple features involved, they often adopt a direct concatenation method. This method often ignores the correlation information between multiple features and does not make full use of the correlation information between multiple features, which is often one of the factors leading to poor prediction trajectory results.

[0033] Through the investigation of the existing literature, the applicant found that the research on uncertainty processing can effectively represent and process the uncertain information caused by ignorance. Inspired by this, the present invention is based on the P-A operator, fully utilizes the information of multiple features and their correlation information in the process of pedestrian trajectory prediction, including the historical trajectory features of the target pedestrian, the interaction information features between the pedestrian and the scene, and the interaction information features between pedestrians. Then, in the fusion process, by concatenating the fusion result with the two interaction information features, the information loss is reduced, the interaction information features are fully utilized, and the uniqueness of the interaction information features is maintained.

[0034] Please refer to Figure 1, Figure 1 It is a flowchart of an autonomous driving pedestrian trajectory prediction method for an embodiment, hereinafter referred to as the pedestrian trajectory prediction method. The pedestrian trajectory prediction method provided in this embodiment includes the following steps.

[0035] Step 101: Obtain the scene features of the target pedestrian according to the scene image of the currently walked scene of the target pedestrian. The scene image in this embodiment refers to the image containing the target pedestrian collected at a historical moment, which can be a video image of continuous frames or a photo image of single-shot continuous moments. The scene image usually includes environmental factors such as streets and obstacles, as well as other pedestrians except the target pedestrian. In this embodiment, the scene image is input into the feature extraction model to extract the scene features at a certain moment.

[0036] Step 102: Obtain the currently walked trajectories of the target pedestrian and other pedestrians around the target pedestrian. The currently walked trajectories of the target pedestrian and other pedestrians around it refer to the position coordinate information of the target pedestrian and other pedestrians around it, which can be obtained by existing positioning methods or other existing means. In this embodiment, other pedestrians around the target pedestrian refer to other pedestrians in the scene image that may affect the walking trajectory of the target pedestrian, which may include all other pedestrians in the scene image except the target pedestrian, or may include some other pedestrians in the scene image except the target pedestrian.

[0037] Step 103: Determine the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian according to the scene features of the target pedestrian, the currently walked trajectory of the target pedestrian, and the currently walked trajectories of other pedestrians. Since in the research of pedestrian trajectory prediction, multiple feature information is often not independent, but there is certain associated information. These associated feature information usually includes: the interaction feature information between pedestrians and the interaction feature information between pedestrians and the scene. In order to better utilize these interaction feature information subsequently, this embodiment first obtains the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian according to the scene features of the target pedestrian, the currently walked trajectory of the target pedestrian, and the currently walked trajectories of other pedestrians. Among them, the pedestrian-scene interaction feature information refers to the interaction feature information between the target pedestrian and the scene, and the pedestrian-pedestrian interaction feature information refers to the interaction feature information between the target pedestrian and other pedestrians around it.

[0038] It should be noted that the above-mentioned pedestrian-scene interaction feature information and pedestrian-pedestrian interaction feature information may be multiple. For example, there may be multiple other pedestrians around the target pedestrian, then the interaction feature information between the target pedestrian and each other pedestrian is a pedestrian interaction feature information, and multiple pedestrian-pedestrian interaction feature information can be formed.

[0039] Step 104: Perform fusion processing on the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian to obtain fusion feature information.

[0040] Since there is a large amount of feature information involved in pedestrian trajectory prediction, the existing method of concatenating multiple feature information for prediction easily ignores the correlation information between multiple feature information. Moreover, when the number of feature information is very large, it will also increase the training difficulty of the neural network for pedestrian trajectory prediction, making it difficult for the neural network to converge. Therefore, in this embodiment, a fusion processing method is adopted to perform fusion processing on the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian obtained in step 103 to obtain a fusion result, that is, fusion feature information. Taking this fusion feature information as one feature information can significantly reduce the number of feature information while retaining the correlation information between multiple feature information.

[0041] Step 105: Predict the future walking trajectory of the target pedestrian according to the fusion feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information.

[0042] In this embodiment, the pedestrian-scene interaction feature information and the pedestrian-pedestrian interaction feature information are represented in series. In addition, white noise information is added in this embodiment. That is, the future walking trajectory of the target pedestrian is predicted by the fusion feature information, the series-connected pedestrian-scene interaction feature information and pedestrian-pedestrian interaction feature information, and the white noise information. Thus, not only the correlation information between the feature information is fully utilized, but also the uniqueness of the interaction feature information is maintained, making the prediction result of the final walking trajectory more accurate.

[0043] In one embodiment, please refer to Figure 2 , in step 103, according to the scene feature of the target pedestrian, the currently walked trajectory of the target pedestrian, and the currently walked trajectories of other pedestrians, determining the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian includes:

[0044] Step 1031: Input the currently walked trajectory of the target pedestrian and the currently walked trajectories of other pedestrians around the target pedestrian into the first long short-term memory model respectively to obtain the historical trajectory feature information of the target pedestrian and other pedestrians around the target pedestrian.

[0045] Step 1032: In this embodiment, the pedestrian-pedestrian interaction feature information is extracted through the social attention module. Specifically: Obtain the position information of other pedestrians around the target pedestrian relative to the target pedestrian at the current moment; and input the historical trajectory feature information of other pedestrians around the target pedestrian, the position information of other pedestrians around the target pedestrian relative to the target pedestrian at the current moment, and the historical trajectory feature of the target pedestrian at the current moment into the social attention module to obtain the pedestrian-pedestrian interaction feature information.

[0046] Step 1033: Input the scene feature of the target pedestrian and the trajectory feature information of the target pedestrian at the current moment into the physical attention module to obtain the pedestrian-scene interaction feature information. Similar to Step 1032, in this embodiment, the pedestrian-scene interaction feature information is extracted through the physical attention module.

[0047] In one embodiment, please refer to Figure 3 , in Step 104, the historical trajectory feature information of the target pedestrian, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information are fused to obtain the fused feature information, including:

[0048] Step 1041: Construct the P-A operator function. The P-A operator is the Power Average Operator, which can model the uncertain information between multiple feature information and then perform fusion representation.

[0049] In this embodiment, a new P-A operator function is constructed in combination with the principle of the existing P-A operator, which is described below.

[0050] The P-A operator function is obtained according to the following formula:

[0051]

[0052] where, (a1, a2,..., a n ) is a data set, a i represents the i-th data, a j represents the j-th data, represents the total support degree of other data for a i except a i , and sup(a i , a j ) represents the support degree function of a j for a i .

[0053] The sup(a i , a j ) function satisfies the condition: x and y are not equal to a i , aj Any number of

[0054] In the above P-A operator, the SUP(*) function is the support degree function in the P-A operator principle defined by the applicant, that is, the newly constructed function combined with the present application. It can realize the fusion of interactive feature information on the premise of meeting the function conditions.

[0055] Step 1042: Use the P-A operator function to fuse the historical trajectory feature information of the target pedestrian, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information to obtain the fused feature information.

[0056] Based on the P-A operator function constructed in the above step 1041, the historical trajectory feature information of the target pedestrian, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information are used as the data in the data set (a1, a2,..., a n ) and input into the P-A operator function to obtain the fused feature information.

[0057] In an embodiment, in step 105, according to the fused feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information, predicting the future walking trajectory of the target pedestrian includes:

[0058] Step 1051: Concatenate the pedestrian-scene interaction feature information and the pedestrian-pedestrian interaction feature information to form a concatenated set of interactive feature information.

[0059] Step 1052: Obtain a white noise information, and this white noise information can be any randomly generated white noise.

[0060] Step 1053: Input the fused feature information, the concatenated set of interactive feature information, and the white noise information into the second long short-term memory model to obtain the future walking trajectory of the target pedestrian. Since the interaction between pedestrians and scenes and the interaction between pedestrians are very complex and abstract, it is difficult to make full use of these interactive feature information. In order to reduce information loss and maintain the uniqueness of the two interactive feature information, the present application concatenates the pedestrian-scene interaction feature information and the pedestrian-pedestrian interaction feature information, makes full use of the interactive feature information, and compensates for some information losses.

[0061] The pedestrian trajectory prediction method provided by the embodiment of the present invention will be described in detail through an example below.

[0062] Step 201: Obtain the scene image ImgT at time T, and input the scene image ImgT into the feature extraction module to extract the scene features. In this embodiment, the Vgg-16 module in the CNN model is used as the feature extraction module to obtain the scene features at the current time T, as shown in the following formula.

[0063]

[0064] Among them, Img T is the scene image, and W CNN represents the model parameters of the CNN model, and CNN(*) represents the CNN network model. represents the scene feature at time T.

[0065] Step 202: Obtain the current traveled trajectory of each of the M pedestrians from the start time t = 1 to time T, as shown in the following formula.

[0066]

[0067] Among them, represents the walking trajectory (position information) of the I-th pedestrian at time t, represents the current traveled trajectory of the I-th pedestrian.

[0068] Step 203: Determine the pedestrian-scene interaction feature information and the pedestrian-pedestrian interaction feature information according to the following formula. In this embodiment, the I-th pedestrian is used as the target pedestrian.

[0069]

[0070]

[0071]

[0072]

[0073] Among them, represents the historical trajectory information feature of the target pedestrian I, represents the historical trajectory information of the J-th pedestrian, represents the relative position information between the target pedestrian I and the surrounding pedestrians and the historical trajectory information of the surrounding pedestrians. LSTM enc (*) is the first long short-term memory model, represents the current traveled trajectory of the target pedestrian I, represents the hidden layer feature information of the historical trajectory of the target pedestrian I input into the first long short-term memory model, and W enc represents the model parameters of the first long short-term memory model.

[0074] represents the pedestrian-scene interaction feature information, represents the pedestrian-pedestrian interaction feature information, and attention ph (*) represents the physical attention module, and attention so(*) represents the social attention module, represents the hidden layer information of the target pedestrian I in the second long short-term memory module, W ph represents the model parameters of the physical attention module, W so represents the model parameters of the social attention module.

[0075] Step 204: Construct the P-A operator function. This has been described in detail in the above embodiments, and the P-A operator function constructed in this embodiment will not be elaborated here.

[0076] Step 205: According to the following formula, use the P-A operator function to fuse the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian to obtain the fused feature information.

[0077]

[0078] Among them, is the fused feature information, is the hidden layer feature information of the first long short-term memory module of the historical trajectory of the target pedestrian I, is the pedestrian interaction feature information of the target pedestrian I, is the scene interaction feature information of the target pedestrian I;

[0079]

[0080] Among them,

[0081] Among them, D 3 (*) represents the spatial distance function between the historical trajectory feature information of the main pedestrian, pedestrian-pedestrian interaction feature information, and pedestrian-scene interaction feature information, sup(*) represents the mutual support degree between the historical trajectory feature information, pedestrian-pedestrian interaction feature information, and pedestrian-scene interaction feature information of the target pedestrian, and T(*) represents the total support degree of other features for the current feature except the current feature.

[0082] Step 206: According to the following formula, input the fused feature information, the concatenated set of interaction feature information, and the white noise information into the second long short-term memory model to obtain the future walking trajectory of the target pedestrian.

[0083]

[0084] Among them, is the concatenated set of interaction feature information, Z is the white noise information, W Geare the model parameters of the second long short-term memory model, LSTM Ge (*) represents the second long short-term memory model, represents the future walking trajectory of the predicted target pedestrian.

[0085] In this embodiment, as shown in the following formula, a randomly selected real trajectory or the future walking trajectory of the predicted target pedestrian output by the generator is input into the discriminator to determine whether the trajectory is the output trajectory of the generator or a real trajectory.

[0086]

[0087] Among them, is the discrimination result of the discriminator at the τ-th future moment, including two cases: 1 or 0, where 1 represents a real trajectory and 0 represents a trajectory generated by the generator; represents a randomly selected real trajectory or a trajectory generated by the generator. represents the hidden layer feature of the third long short-term memory module; LSTM Dis (*) represents the third long short-term memory model corresponding to the discriminator; W Dis are the model parameters of the third long short-term memory model.

[0088] The losses function adopted in the above method is the existing GAN network cross-entropy loss function and L2 loss function.

[0089] During the experiment, assuming that each pedestrian observes the movement of the past 8 time steps (3.2s) and predicts the position of the next 12 time steps (4.8s), we use 5 public datasets for the experiment. These datasets contain different situations of interactions between pedestrians in 5 scenarios (eth, hotel, univ, zara1, zara2), such as walking together, crossing or avoiding collisions. This algorithm trains and calculates the optimal model within 451 epochs, and the learning rate is 0.001. In the test phase, the test set is input into the optimal model to obtain the results of pedestrian trajectory prediction. The experimental results show that our method can effectively improve the accuracy of pedestrian trajectory prediction and actively reduce the collisions that occur during pedestrian walking, making pedestrians walk more in line with social norms.

[0090] Please refer to Figure 4 , this embodiment also provides an autonomous driving pedestrian trajectory prediction device, hereinafter referred to as the pedestrian trajectory prediction device. The pedestrian trajectory prediction device includes: a scene feature acquisition module 301, a historical walking trajectory acquisition module 302, an interaction feature information acquisition module 303, a fused feature information acquisition module 304, and a prediction module 305.

[0091] The scene feature acquisition module 301 is configured to acquire the scene features of the target pedestrian according to the scene image of the scene where the target pedestrian has walked currently.

[0092] The historical walking trajectory acquisition module 302 is configured to acquire the currently walked trajectories of the target pedestrian and other pedestrians around the target pedestrian.

[0093] The interaction feature information acquisition module 303 determines the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian according to the scene features of the target pedestrian, the currently walked trajectory of the target pedestrian, and the currently walked trajectories of other pedestrians.

[0094] The fused feature information acquisition module 304 performs a fusion process on the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian to obtain the fused feature information.

[0095] The prediction module 305 predicts the future walking trajectory of the target pedestrian according to the fused feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information.

[0096] It should be noted that the functional modules in this embodiment correspond to the above method steps, and their specific implementation manners have been described in detail in the above embodiments, and will not be elaborated here.

[0097] Those skilled in the art can understand that all or part of the functions of the above methods can be implemented in a hardware manner or in a computer program manner. When all or part of the functions in the above embodiments are implemented in a computer program manner, the program can be stored in a computer-readable storage medium. The storage medium may include: read-only memory, random access memory, magnetic disk, optical disk, hard disk, etc. The above functions are implemented by a computer executing the program. For example, the program is stored in the memory of the device, and when the processor executes the program in the memory, the above all or part of the functions can be implemented. In addition, when all or part of the functions in the above embodiments are implemented in a computer program manner, the program can also be stored in a storage medium such as a server, another computer, magnetic disk, optical disk, flash drive, or mobile hard disk, and is saved to the memory of the local device by downloading or copying, or the system of the local device is updated in version. When the processor executes the program in the memory, the above all or part of the functions in the above embodiments can be implemented.

[0098] The above uses specific examples to elaborate on the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art of the present invention, according to the idea of the present invention, several simple deductions, deformations, or substitutions can also be made.

Claims

1. An autonomous driving pedestrian trajectory prediction method, characterized in that, Including: Obtain the scene features of the target pedestrian according to the scene image of the current walked scene of the target pedestrian; Obtain the current walked trajectories of the target pedestrian and other pedestrians around the target pedestrian; Determine the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian according to the scene features of the target pedestrian, the current walked trajectory of the target pedestrian, and the current walked trajectories of the other pedestrians; Construct a P-A operator function, and use the P-A operator function to perform fusion processing on the historical trajectory feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information of the target pedestrian to obtain fusion feature information; The constructing of the P-A operator function includes: obtaining the P-A operator function according to the following formula: Among them, (a1, a2,..., a n ) is a data set, a i represents the i-th data, a j represents the j-th data, represents the total support degree of other data for a i except a i , where sup(a i , a j ) represents the support degree function of a j for a i ; sup(a i ,a j ) The function satisfies the condition: x and y are any numbers not equal to a i and a j ; Predict the future walking trajectory of the target pedestrian according to the fusion feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information.

2. The method according to claim 1, characterized in that, Determining the historical trajectory feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information of the target pedestrian according to the scene features of the target pedestrian, the current walked trajectory of the target pedestrian, and the current walked trajectories of the other pedestrians includes: Input the current walked trajectory of the target pedestrian and the current walked trajectories of the other pedestrians into a first long short-term memory model respectively to obtain the historical trajectory feature information of the target pedestrian and other pedestrians around the target pedestrian; Obtain the position information of other pedestrians around the target pedestrian relative to the target pedestrian at the current moment; and input the historical trajectory feature information of other pedestrians around the target pedestrian, the position information of other pedestrians around the target pedestrian relative to the target pedestrian at the current moment, and the historical trajectory feature of the target pedestrian at the current moment into a social attention module to obtain the pedestrian-pedestrian interaction feature information; Input the scene features of the target pedestrian and the historical trajectory feature of the target pedestrian at the current moment into a physical attention module to obtain the pedestrian-scene interaction feature information.

3. The method according to claim 1, characterized in that, Using the P-A operator function to fuse the historical trajectory feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information of the target pedestrian to obtain fusion feature information includes: Obtain the fusion feature information according to the following formula: Among them, is the fused feature information, is the historical trajectory feature information of target pedestrian I, is the pedestrian interaction feature information of target pedestrian I, is the scene interaction feature information of target pedestrian I; Among them, where sup(*) represents the mutual support degree among the historical trajectory feature information of the target pedestrian, the pedestrian-pedestrian interaction feature information, and the pedestrian-scene interaction feature information, and T(*) represents the total support degree of other feature information for the current feature information except the current feature information.

4. The method according to claim 1, wherein The predicting of the future walking trajectory of the target pedestrian according to the fusion feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information includes: Concatenate the pedestrian-scene interaction feature information and the pedestrian-pedestrian interaction feature information to form a concatenated set of interaction feature information; Obtain a white noise information; Input the fused feature information, the concatenated set of interaction feature information, and the white noise information into a second long short-term memory model to obtain the future walking trajectory of the target pedestrian.

5. An automatic driving pedestrian trajectory prediction device, characterized in that, It includes: A scene feature acquisition module, configured to acquire the scene features of the target pedestrian according to the scene image of the scene where the target pedestrian has walked currently. A historical walking trajectory acquisition module, configured to acquire the currently walked trajectories of the target pedestrian and other pedestrians around the target pedestrian. An interaction feature information acquisition module, which determines the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian according to the scene features of the target pedestrian, the currently walked trajectory of the target pedestrian, and the currently walked trajectories of the other pedestrians. A fused feature information acquisition module, which constructs a P-A operator function and uses the P-A operator function to perform fusion processing on the historical trajectory feature information, pedestrian-scene interaction feature information, and pedestrian-pedestrian interaction feature information of the target pedestrian to obtain fused feature information. The constructing of the P-A operator function includes: obtaining the P-A operator function according to the following formula: Among them, (a1, a2,..., a n ) is a data set, where a i represents the i-th data, and a j represents the j-th data. represents the total support degree of other data for a i except a i . Among them, sup(a i , a j ) represents the support degree function of a j for a i ; sup(a i ,a j ) The function satisfies the condition: x and y are any numbers not equal to a i and a j ; A prediction module, which predicts the future walking trajectory of the target pedestrian according to the fused feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information.

6. The device according to claim 5, characterized in that, The determining of the historical trajectory feature information, the pedestrian-scene interaction feature information, and the pedestrian-pedestrian interaction feature information of the target pedestrian according to the scene features of the target pedestrian, the currently walked trajectory of the target pedestrian, and the currently walked trajectories of the other pedestrians includes: Input the currently walked trajectories of the target pedestrian and the other pedestrians into a first long short-term memory model respectively to obtain the trajectory feature information of the target pedestrian and other pedestrians around the target pedestrian. Obtain the position information of other pedestrians around the target pedestrian relative to the target pedestrian at the current moment; and input the historical trajectory feature information of other pedestrians around the target pedestrian, the position information of other pedestrians around the target pedestrian relative to the target pedestrian at the current moment, and the historical trajectory feature of the target pedestrian at the current moment into a social attention module to obtain the pedestrian-pedestrian interaction feature information. Input the scene features of the target pedestrian and the historical trajectory feature of the target pedestrian at the current moment into a physical attention module to obtain the pedestrian-scene interaction feature information.

7. A pedestrian trajectory prediction device, characterized in that, It includes: A memory, configured to store programs. A processor, configured to implement the method according to any one of claims 1-4 by executing the programs stored in the memory.

8. A computer-readable storage medium, characterized in that, Programs are stored on the medium, and the programs can be executed by the processor to implement the method according to any one of claims 1-4.

Images