Pedestrian long-time trajectory prediction method and device

Abstract

This invention proposes a method, apparatus, and medium for predicting pedestrian trajectories. The method includes acquiring a historical location map of pedestrians, a global scene map, and historical observation trajectories of pedestrians; obtaining the scene location features of pedestrians based on the historical location map and the global scene map; modeling the historical observation trajectories of pedestrians to generate future movement trends of pedestrians; predicting the pedestrian's destination based on the future movement trend and scene location features of pedestrians; and combining the pedestrian's destination with the predicted future trajectory. This method combines a long-term trajectory prediction model that incorporates future movement trends of pedestrians, global scene layout, and multimodal targets, thereby improving the prediction accuracy of long-term pedestrian trajectory prediction.

Description

Technical Field

[0001] This invention relates to the field of crowd trajectory prediction technology, and in particular to a method and apparatus for predicting long-term pedestrian trajectories. Background Technology

[0002] Most existing pedestrian trajectory prediction methods focus on short-term trajectory prediction. Long-term trajectory prediction involves a longer prediction time, greater pedestrian movement distance, and more significant changes in the surrounding environment. Furthermore, the impact of short-term pedestrian interactions on the long-term future movement of pedestrians gradually weakens during the observation phase. Meanwhile, the pedestrian's target location, as a manifestation of the pedestrian's movement intention, determines the general trend of the pedestrian's movement and has a gradually increasing influence on the pedestrian's future route.

[0003] Most existing pedestrian trajectory prediction methods focus on short-term trajectory prediction. However, previous short-term prediction methods suffer from limitations due to the short distance pedestrians travel and limited scene changes, hindering practical applications. Long-term trajectory prediction, on the other hand, involves longer prediction times, greater pedestrian distances, and more significant scene changes. During the observation phase, the impact of short-term pedestrian interactions on long-term future movement gradually weakens, while the pedestrian's target location, reflecting their movement intention, increasingly influences their future route. While the existing YNet model achieves long-term trajectory prediction, it still has shortcomings in modeling the relationship between people and the scene, extracting pedestrian endpoints using scene layout, and extracting pedestrian movement patterns. This can lead to situations where the predicted target is located in trees, buildings, or where the future pedestrian trajectory does not match the actual movement trend. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a method and apparatus for predicting long-term pedestrian trajectories. This method combines a long-term trajectory prediction model that incorporates the future movement trend of pedestrians, global scene layout, and multimodal targets, thereby improving the accuracy of long-term pedestrian trajectory prediction.

[0005] To achieve the above objectives, the present invention provides a method for predicting long-term pedestrian trajectories, comprising:

[0006] Obtain historical location maps of pedestrians, global scene maps, and historical observation trajectories of pedestrians;

[0007] Based on the pedestrian historical location map and the global scene map, the scene location features of the pedestrians are obtained;

[0008] Model the historical observation trajectory of the pedestrian to generate the future movement trend of the pedestrian;

[0009] Based on the pedestrian's future movement trend and the pedestrian's scene location characteristics, predict the pedestrian's trajectory.

[0010] Optionally, obtaining the scene location features of pedestrians based on the pedestrian historical location map and the global scene map includes:

[0011] The pedestrian historical location map and the global scene map are stitched together and used as the input to the pedestrian scene location encoder.

[0012] By using a pedestrian scene location encoder, the scene location is encoded to obtain the scene location features of pedestrians.

[0013] Optionally, modeling the historical observation trajectory of the pedestrian to generate the pedestrian's future movement trend includes:

[0014] The pedestrian's historical observation trajectory is trend-encoded and decoded using a multilayer perceptron to generate the pedestrian's future movement trend;

[0015] The future movement trend of the pedestrians is represented as follows:

[0016] X tend =MLP dec (MLP enc (X obs W me );W md )

[0017] Among them, MLP enc X represents a motion trend encoder composed of a multilayer sensing mechanism. obs For the historical observation trajectory of pedestrians, W me The parameters representing the motion trend encoder, MLP dec W represents the motion trend decoder composed of a multilayer sensing mechanism. md X represents the parameters of the motion trend decoder. tend This indicates the future movement trend of the pedestrians generated in the final version.

[0018] Optionally, the pedestrian's scene location features include a first scene location feature related to the destination target and a second scene location feature related to the trajectory;

[0019] The prediction of pedestrian trajectories based on the pedestrian's future movement trends and scene location characteristics includes:

[0020] Based on the location features of the first scene, a destination map of the pedestrian is obtained;

[0021] The pedestrian trajectory is obtained based on the pedestrian destination map, the location features of the second scene, and the future movement trend of the pedestrian.

[0022] Optionally, an attention mechanism can be used to extract the first scene location features and the second scene location features from the scene location features of the pedestrian.

[0023] Optionally, the pedestrian destination map is represented as follows:

[0024] M des =Conv des (feature des W des )

[0025] Among them, M des Represents the generated pedestrian destination map, Conv des W represents the endpoint target decoder based on convolutional layers. des The parameters representing the endpoint target decoder, feature des Conv represents the first scene location features related to the endpoint target. des This indicates the endpoint target decoder.

[0026] Optionally, the method further includes:

[0027] The various regions in the global scene map are classified and each category is marked to construct a scene layout correction map;

[0028] The pedestrian destination target map is corrected using the scene layout correction map to obtain the corrected pedestrian destination target map.

[0029] Optionally, the various areas in the global scene map are divided into three categories: frequently traversed areas, occasionally traversed areas, and prohibited areas. The scene layout correction map is constructed by classifying the three categories of traversed areas according to the probability of pedestrians passing through them in descending order of probability.

[0030] The pedestrian destination map is modified using the scene layout correction map, discarding pedestrian destination targets in restricted areas, resulting in the modified pedestrian destination map. The modified pedestrian destination map includes pedestrian destination targets in frequently traversed areas and pedestrian destination targets in occasionally traversed areas.

[0031] Optionally, obtaining the pedestrian trajectory based on the pedestrian's destination map, the second scene location features, and the pedestrian's future movement trend includes:

[0032] The future movement trend of the pedestrian and the corrected destination map of the pedestrian are downsampled so that the future movement trend of the pedestrian and the destination map of the pedestrian correspond one-to-one with the dimensions of each layer of the scene location code;

[0033] The modified pedestrian destination map, the pedestrian's future movement trend, and the second scene location features are incorporated into each layer of the trajectory decoder to decode the pedestrian trajectory.

[0034] The pedestrian trajectory is represented as follows:

[0035] M traj =Conv traj (Concat(feature traj M tend M goal );W traj )

[0036] Among them, feature traj M represents the location features of the second scene. tend For the future movement trends of pedestrians, M goal For the pedestrian destination map, Concat represents the concatenation operation, and Conv... traj For trajectory decoder, W traj M is a parameter of the trajectory decoder. traj This represents the pedestrian's trajectory.

[0037] In another aspect, the present invention provides a pedestrian long-term trajectory prediction device, employing the above-described pedestrian long-term trajectory prediction method, wherein the device comprises at least:

[0038] The acquisition module is used to acquire pedestrian historical location maps, global scene maps, and pedestrian historical observation trajectories;

[0039] The scene location encoding module is used to obtain the scene location features of pedestrians based on the pedestrian historical location map and the global scene map;

[0040] The pedestrian movement trend modeling module is used to model the historical observation trajectory of the pedestrian and generate the future movement trend of the pedestrian.

[0041] The pedestrian trajectory generation module predicts pedestrian trajectories based on the pedestrian's future movement trends and scene location characteristics.

[0042] In another aspect, the present invention provides a storage medium for storing a computer program for performing the above-described pedestrian long-term trajectory prediction method.

[0043] In another aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method for predicting long-term pedestrian trajectories.

[0044] As can be seen from the above solutions, the advantages of the present invention are:

[0045] The pedestrian long-term trajectory prediction method provided by this invention obtains the scene location features of pedestrians based on historical pedestrian location maps and global scene maps; then, it models the historical pedestrian observation trajectories to generate future pedestrian movement trends; and finally, it predicts the pedestrian trajectory based on the future pedestrian movement trends and scene location features. This method comprehensively considers the movement trends contained in the historical pedestrian trajectories and the endpoint target information under scene constraints, combining the endpoint target and movement trends to predict future trajectories; thus, the predicted long-term trajectory conforms to both the constraints of the scene layout and the pedestrian's own movement trends, improving the prediction accuracy of long-term pedestrian trajectory prediction and generating future trajectories that conform to pedestrian movement and scene information. Attached Figure Description

[0046] Figure 1 A flowchart illustrating the long-term pedestrian trajectory prediction method provided in an embodiment of the present invention;

[0047] Figure 2 for Figure 1 A detailed flowchart of step S4 in the pedestrian long-term trajectory prediction method provided in the document;

[0048] Figure 3(a) is a comparison of the effects of the pedestrian long-term trajectory prediction method of the present invention and the Ynet prediction method (I);

[0049] Figure 3(b) is a comparison of the effects of the pedestrian long-term trajectory prediction method of the present invention and the Ynet prediction method (II);

[0050] Figure 3(c) is a comparison of the effects of the pedestrian long-term trajectory prediction method of the present invention and the Ynet prediction method (III);

[0051] Figure 4 This is a frame diagram of the pedestrian long-term trajectory prediction device of the present invention;

[0052] Figure 5 This is a schematic diagram of the structure of an electronic device;

[0053] in:

[0054] 400-Pedestrian Long-Term Trajectory Prediction Device;

[0055] 401 - Get Module;

[0056] 402 - Scene Location Encoding Module;

[0057] 403 - Pedestrian Movement Trend Modeling Module;

[0058] 404 - Pedestrian trajectory generation module;

[0059] 500 - Electronic Equipment;

[0060] 501-Processor;

[0061] 502 - Memory. Detailed Implementation

[0062] To make the above features and effects of the present invention clearer and easier to understand, specific embodiments are described below, and detailed descriptions are provided in conjunction with the accompanying drawings.

[0063] Existing pedestrian long-term trajectory prediction methods using the YNet model fully integrate scene information and the influence of destination intent. This approach proposes considering not only the pedestrian's destination intent but also the intent at intermediate moments for long-term trajectory prediction. This method argues that even with the same start and end points, the choice of intermediate trajectory points will differ, thus proposing intermediate moment points to obtain different multimodal trajectory predictions. This method fully integrates scene semantic information, using a multi-layer convolutional model to construct an encoder-decoder structure. First, it predicts the destination distribution map through encoding and decoding, sampling to obtain multimodal destinations. Then, it combines the destinations with multiple possible corresponding intermediate points. Finally, it combines the destinations, intermediate points, and encoded scene location information, using a trajectory decoder to obtain the location distribution map for each future prediction moment, statistically determining the pedestrian's position at that moment, and thus obtaining the final future trajectory. However, this method only considers the spatial relationship between people and the scene during encoding, but combines the same relational features when performing different tasks, and it does not effectively incorporate scene layout information during destination extraction. Furthermore, this method ignores the potential influence of the pedestrian's own motion state, leading to inaccurate pedestrian trajectory prediction.

[0064] To address this, the pedestrian long-term trajectory prediction method provided in this embodiment of the invention is an improvement on the Ynet long-term trajectory prediction method. It comprehensively considers the pedestrian's movement trend and the endpoint under external scene constraints, and combines the pedestrian's endpoint and movement trend to predict the future trajectory, so that the predicted trajectory conforms to both the constraints of the scene layout and the pedestrian's own movement trend.

[0065] For details, please refer to Figure 1 As shown, Figure 1 A flowchart illustrating the long-term pedestrian trajectory prediction method is shown.

[0066] A method for predicting long-term pedestrian trajectories includes:

[0067] S1. Obtain the historical location map of pedestrians, the global scene map, and the historical observation trajectory of pedestrians;

[0068] S2. Based on the pedestrian historical location map and the global scene map, obtain the scene location features of the pedestrians;

[0069] In the specific implementation, obtaining the scene location features of pedestrians based on the pedestrian historical location map and the global scene map includes:

[0070] The pedestrian historical location map and the global scene map are stitched together and used as input to the pedestrian scene location encoder. Specifically, in this embodiment, the pedestrian historical location map and the global scene map of each observed frame are stitched together and used as input to the pedestrian scene location encoder. If the global scene map has dimensions H×W and the semantic segmentation information M... scene The dimensions are H×W×C, where C is the number of semantic segmentation categories, and M is the pedestrian historical location map. pos The dimensions are H×W×T obs T obs This represents the historical duration of the observation. The pedestrian historical location map and the global scene map are semantically concatenated and used as the input to the pedestrian scene location encoder. The input dimension is H×W×(C+T). obs ),Right now:

[0071] input = Concat(M) pos M scene )

[0072] In the formula, M pos M represents a map showing the historical locations of pedestrians. scene This represents the global scene map, input represents the input of the pedestrian scene location encoder, and Concat represents the semantic concatenation operation.

[0073] Then, the scene location encoder is used to encode the scene location and obtain the scene location features of the pedestrian.

[0074] feature = Conv enc (input;W enc )

[0075] In the formula, Conv enc This represents a scene location encoder based on multiple convolutional layers. `input` represents the input to the pedestrian scene location encoder, `feature` represents the scene location features of the pedestrian output by the encoder, and `W` represents the input of the pedestrian scene location encoder. enc This represents the parameters of the scene position encoder.

[0076] In addition, the pedestrian's scene location features include a first scene location feature related to the destination target and a second scene location feature related to the trajectory;

[0077] The first scene location feature and the second scene location feature are extracted from the scene location features of the pedestrian using an attention mechanism.

[0078] In this embodiment, an attention mechanism is used to selectively extract the first scene location feature related to the destination target from the scene location features of the traveler.des and the second scene location features related to the trajectory traj This allows them to utilize different features to accomplish two different tasks: the distribution of the final destination and the distribution of the trajectory.

[0079] S3. Model the historical observation trajectory of the pedestrian to generate the future movement trend of the pedestrian.

[0080] The future movement trend of a pedestrian refers to the destination a pedestrian might reach while continuing their own movement, unaffected by external factors. Existing pedestrian trajectory prediction methods only model the pedestrian's movement sequence and then derive the pedestrian's long-term movement trend based on the modeled movement characteristics. This method only considers the pedestrian's own movement characteristics and does not take into account other external objective factors. In this embodiment, the future movement trend of a pedestrian is obtained by modeling historical pedestrian observation trajectories.

[0081] In the specific implementation, a multilayer perceptron is used to perform trend encoding and decoding on the historical observation trajectory of the pedestrian to generate the future movement trend of the pedestrian;

[0082] The future movement trend of the pedestrians is represented as follows:

[0083] X tend =MLP dec (MLP enc (X obs W me );W md )

[0084] Among them, MLP enc X represents a motion trend encoder composed of a multilayer sensing mechanism. obs For the historical observation trajectory of pedestrians, W me The parameters representing the motion trend encoder, MLP dec W represents the motion trend decoder composed of a multilayer sensing mechanism. md X represents the parameters of the motion trend decoder. tend This indicates the future movement trend of the pedestrians generated in the final version.

[0085] S4. Predict the pedestrian's trajectory based on the pedestrian's future movement trend and the pedestrian's scene location characteristics.

[0086] The final pedestrian destination, influenced by the pedestrian's location and the global scene layout, is the pedestrian destination target. In this embodiment, based on the extraction of first scene location features related to the destination target and second scene location features related to the trajectory from the pedestrian's scene location features using an attention mechanism, the pedestrian destination target map is determined using the first scene location features. Furthermore, to avoid long-term pedestrian destinations appearing in unwalkable areas, causing the predicted trajectory to completely deviate from the scene layout rules, this embodiment further refines the pedestrian destination target map by combining scene layout, dividing the distribution of walkable destinations, thereby filtering the pedestrian destination target map and restricting future multimodal destination target distributions to walkable scene layouts.

[0087] Finally, the pedestrian trajectory is obtained based on the corrected pedestrian destination map, the trajectory-related second scene location features, and the pedestrian's future movement trend.

[0088] like Figure 2 As shown, Figure 2 The diagram below shows a detailed flowchart of step S4.

[0089] In a specific implementation, predicting the pedestrian's trajectory based on the pedestrian's future movement trend and the pedestrian's scene location characteristics includes:

[0090] S41. Based on the location features of the first scene, a pedestrian destination target map is obtained, that is, the pedestrian destination target map is represented as:

[0091] W des =Conv des (feature des W des )

[0092] Among them, M des Represents the generated pedestrian destination map, Conv des W represents the endpoint target decoder based on convolutional layers. des The parameters representing the endpoint target decoder, feature des Conv represents the first scene location features related to the endpoint target. des This indicates the endpoint target decoder.

[0093] S42. Classify the various regions in the global scene map and mark each classified region to construct a scene layout correction map;

[0094] The pedestrian destination target map is corrected using the scene layout correction map to obtain the corrected pedestrian destination target map.

[0095] In this embodiment, the various areas in the global scene map are divided into three categories: frequently traversed areas, occasionally traversed areas, and prohibited areas. The scene layout correction map is constructed by classifying the three categories of traversed areas according to the probability of pedestrians passing through them in descending order of probability.

[0096] The pedestrian destination map is modified using the scene layout correction map, discarding pedestrian destination targets in restricted areas, resulting in the modified pedestrian destination map. The modified pedestrian destination map includes pedestrian destination targets in frequently traversed areas and pedestrian destination targets in occasionally traversed areas.

[0097] The specific method for obtaining the distribution of passable targets by modifying the map using scene layout is as follows:

[0098]

[0099] In the formula M sample For the final generated sampled map, M des S represents the endpoint distribution generated by the model, where S denotes the traversal probability of locations included in the scenario feasibility correction map. This represents element-wise multiplication.

[0100] Finally, multimodal sampling is performed on the walkable endpoint distribution to obtain the final extracted multimodal long-term endpoint information, which is the multimodal pedestrian endpoint target X. goal .

[0101] S43. Based on the pedestrian's destination target map, the location features of the second scene, and the pedestrian's future movement trend, the pedestrian's trajectory is obtained.

[0102] Pedestrian trajectory generation is closely related to their destination. The pedestrian's destination location, as the pedestrian's movement goal, attracts pedestrians to choose suitable routes to reach it. Simultaneously, the semantic influence of the scene must be considered during pedestrian movement to avoid difficult-to-pass locations and choose appropriate paths. In this embodiment, pedestrian trajectory generation integrates the pedestrian's future movement trend, the pedestrian's destination map, and second scene location features related to the trajectory to jointly decode the pedestrian's destination trajectory.

[0103] In its implementation, this method constructs a pedestrian trajectory decoder similar to the endpoint target decoder. The decoder consists of multiple convolutional layers and incorporates an attention module to extract second-scene location features related to trajectory decoding during the encoding process. trajIn the trajectory decoding process, not only scene location features are combined, but also information on the pedestrian's future movement trend and destination target is incorporated. Therefore, the trajectory decoder structure has two more dimensions per layer compared to the destination target decoder structure. To ensure that the vector dimensions of the pedestrian's future movement trend and destination target correspond to the encoded scene location features, the two-dimensional coordinates X of the pedestrian's future movement trend are first... tend and the two-dimensional coordinates X of the multimodal endpoint target goal Represented by the location map, the corresponding future movement trend M of the pedestrian is obtained. tend And pedestrian destination map M goal .

[0104] Then, the future movement trend of the pedestrian and the corrected destination map of the pedestrian are downsampled so that the future movement trend of the pedestrian and the destination map of the pedestrian correspond one-to-one with the dimensions of each layer of the scene location code.

[0105] The modified pedestrian destination map, the pedestrian's future movement trend, and the second scene location features are incorporated into each layer of the trajectory decoder to decode the pedestrian trajectory.

[0106] The pedestrian trajectory is represented as follows:

[0107] M traj =Conv traj (Concat(feature traj M tend M goal );W traj )

[0108] Among them, feature traj M represents the location features of the second scene. tend For the future movement trends of pedestrians, M goal For the pedestrian destination map, Concat represents the concatenation operation, and Conv... traj For trajectory decoder, W traj M is a parameter of the trajectory decoder. traj For pedestrian trajectory, M traj The dimensions are H×W×T pred T pred The prediction duration represents the predicted trajectory.

[0109] Then, through the statistical operation of Softargmax, the two-dimensional coordinates X of the corresponding pedestrian trajectory are obtained. traj ,Right now:

[0110] X traj =Softargmax(M traj ), where Xtraj The dimension is 2×T pred .

[0111] In summary, the pedestrian long-term trajectory prediction method provided in this embodiment obtains the scene location features of pedestrians based on historical pedestrian location maps and global scene maps; then, it models the historical pedestrian observation trajectories to generate future pedestrian movement trends; and finally, it predicts pedestrian trajectories based on these future movement trends and the pedestrians' scene location features. This method comprehensively considers the movement trends contained in the pedestrian's historical trajectories and the endpoint target information under scene constraints, combining the endpoint target and movement trends to predict future trajectories. This ensures that the predicted long-term trajectory conforms to both the constraints of the scene layout and the pedestrian's own movement trends, improving the prediction accuracy in long-term pedestrian trajectory prediction and generating future trajectories that conform to both pedestrian movement and scene information.

[0112] Furthermore, this embodiment proposes a scene layout correction module to limit the walkable area, reduce the probability of invalid predictions, and ensure that the predicted target is located within a reasonable range that conforms to the scene layout, thereby improving the accuracy of pedestrian destination target prediction. Simultaneously, for the two different tasks of predicting the destination and predicting the future trajectory, an attention module is used to selectively extract first scene location features related to the destination target and second scene location features related to the trajectory. These extracted differential features are then combined to complete the two different prediction tasks, fully realizing the modeling of both destination prediction and trajectory prediction, and improving prediction accuracy.

[0113] The following specific example illustrates the effectiveness of the pedestrian long-term trajectory prediction method provided by this invention.

[0114] Table 1 shows the ADE / FDE values ​​of the pedestrian long-term trajectory prediction method of this embodiment compared with other baseline models on the Stanford Drone Dataset. This embodiment compares the best trajectory among 20 multimodal trajectories generated by this model and the baseline model. This model first acquires 20 multimodal endpoint targets and predicts a corresponding multimodal trajectory for each endpoint target. The evaluation metric is ADE / FDE, where ADE is the sum of squared distance differences between the predicted and actual locations over the next 30 frames (30 seconds), and FDE is the squared distance difference between the predicted and actual endpoint locations. Smaller ADE and FDE values ​​indicate better performance. The results show that among competitive comparative methods for long-term trajectory prediction on the Stanford Drone Dataset, the method of this invention outperforms similar methods. Compared with the Ynet model prediction, the method of this invention improves ADE by 8.2% and FDE by 12.8%. These experimental results demonstrate that the method of this invention is capable of making accurate long-term trajectory predictions.

[0115] Table 1 Comparison of long-term trajectory prediction results between the method of this invention and other methods

[0116] method SGAN PECNet Ynet Method of the present invention ADE 155.32 72.22 47.94 44.03 FDE 307.88 118.13 66.71 58.17

[0117] Simultaneously, the pedestrian long-term trajectory prediction method of the present invention is compared with the qualitative experiments of Ynet to prove the effectiveness of the present invention, such as... Figures 3(a)-3(c) As shown in Figures 3(a), 3(b), and 3(c), a comparison of the long-term pedestrian trajectory prediction results under three different working conditions is presented. Figures 3(a)-3(c) The upper half of the figure shows the trajectory distribution predicted by the method of this invention, and the lower half shows the trajectory distribution predicted by YNet. In the figure, the dashed lines represent the historical trajectories of pedestrians, the solid lines represent the future actual trajectories, and the shaded areas represent the distribution of the predicted pedestrian trajectories for 20 trials. Figures 3(a)-3(c) As can be seen from the present invention, the pedestrian trajectory prediction effect of the method is better, the overlap with the actual trajectory is higher, and it can predict behaviors such as going straight and turning.

[0118] The above embodiments of the present invention can be applied to terminal devices that implement the pedestrian long-term trajectory prediction method. These terminal devices may include personal terminals and host computer terminals, etc., and the embodiments of the present invention do not impose any limitations on them. The terminal can support operating systems such as Windows, Android, iOS, and Windows Phone.

[0119] Reference Figure 4 , Figure 4 A pedestrian long-term trajectory prediction device 400 is shown. This device, applied to pedestrian long-term trajectory prediction methods, can be used in personal terminals and host computer terminal devices. It can achieve prediction through methods such as... Figure 1 , Figure 2 The pedestrian long-term trajectory prediction method shown in this application and the pedestrian long-term trajectory prediction device provided in this embodiment can realize each process of the above-mentioned pedestrian long-term trajectory prediction method.

[0120] A pedestrian long-term trajectory prediction device 400 includes at least an acquisition module 401, a scene location encoding module 402, a pedestrian movement trend modeling module 403, and a pedestrian trajectory generation module 404.

[0121] The acquisition module 401 is used to acquire the historical location map of pedestrians, the global scene map, and the historical observation trajectory of pedestrians.

[0122] The scene location encoding module 402 is used to obtain the scene location features of pedestrians based on the pedestrian historical location map and the global scene map;

[0123] The pedestrian movement trend modeling module 403 is used to model the historical observation trajectory of the pedestrian and generate the future movement trend of the pedestrian.

[0124] The pedestrian trajectory generation module 404 is used to predict the pedestrian trajectory based on the pedestrian's future movement trend and the pedestrian's scene location characteristics.

[0125] The pedestrian long-term trajectory prediction device provided in this embodiment comprehensively considers the movement trend contained in the pedestrian's historical trajectory and the destination target information under scene constraints, and combines the destination target and movement trend to predict the future trajectory; so that the predicted long-term trajectory conforms to both the constraints of the scene layout and the pedestrian's own movement trend, thereby improving the prediction accuracy of pedestrian long-term trajectory prediction, and can generate future trajectories that conform to pedestrian movement and scene information.

[0126] It should be understood that the descriptions of the pedestrian long-term trajectory prediction method also apply to the pedestrian long-term trajectory prediction device 400 according to the embodiments of this application, and will not be described in detail again to avoid repetition.

[0127] Furthermore, it should be understood that the pedestrian long-term trajectory prediction device 400 according to the embodiments of this application is only illustrated by the division of the above-described functional modules. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the pedestrian long-term trajectory prediction device can be divided into functional modules that are different from the modules shown above, in order to complete all or part of the functions described above.

[0128] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application.

[0129] like Figure 5 As shown in the figure, this application embodiment also provides an electronic device 500, including a processor 501, a memory 502, and a program or instructions stored in the memory 502 and executable on the processor 501. When the program or instructions are executed by the processor 501, they implement the steps of the above-mentioned pedestrian long-term trajectory prediction method and achieve the same technical effect.

[0130] Therefore, the electronic device 500 according to the embodiments of this application comprehensively considers the motion trend contained in the pedestrian's historical trajectory and the destination target information under the scene constraints, and combines the destination target and motion trend to predict the future trajectory; so that the predicted long-term trajectory conforms to both the constraints of the scene layout and the pedestrian's own motion trend, thereby improving the prediction accuracy in predicting the long-term trajectory of pedestrians, and can generate a future trajectory that conforms to the pedestrian's motion and scene information.

[0131] Other technical effects of the electronic device 500 according to the embodiments of this application will not be described in detail here to avoid repetition.

[0132] It should be noted that the electronic devices in the embodiments of this application may include mobile electronic devices and non-mobile electronic devices.

[0133] This application also provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the above-described pedestrian long-term trajectory prediction method and achieve the same technical effect.

[0134] Therefore, according to the readable storage medium of the present application embodiment, by comprehensively considering the motion trend contained in the pedestrian's historical trajectory and the destination target information under the scene constraints, the prediction of the future trajectory is realized by combining the destination target and the motion trend; so that the predicted long-term trajectory conforms to both the constraints of the scene layout and the pedestrian's own motion trend, the prediction accuracy of the long-term trajectory prediction of pedestrians is improved, and a future trajectory that conforms to the pedestrian's motion and scene information can be generated.

[0135] Other technical effects of the readable storage medium according to the embodiments of this application will not be described in detail here to avoid repetition.

[0136] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0137] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be applied, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0139] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for predicting long-term pedestrian trajectories, characterized in that, include: Obtain historical location maps of pedestrians, global scene maps, and historical observation trajectories of pedestrians; Based on the pedestrian historical location map and the global scene map, the scene location features of the pedestrians are obtained. The scene location features of the pedestrians include a first scene location feature related to the destination target and a second scene location feature related to the trajectory. Model the historical observation trajectory of the pedestrian to generate the future movement trend of the pedestrian; Predicting pedestrian trajectories based on the pedestrian's future movement trends and scene location characteristics includes: Based on the location features of the first scene, a destination map of the pedestrian is obtained; The various areas in the global scene map are divided into three categories: frequently traversed areas, occasionally traversed areas, and prohibited areas. The scene layout correction map is constructed by classifying the three categories of traversed areas according to the probability of pedestrian passage from high to low. The pedestrian destination target map is modified using the scene layout correction map, discarding pedestrian destination targets in restricted areas, resulting in the modified pedestrian destination target map. The modified pedestrian destination target map includes pedestrian destination targets in frequently traversed areas and pedestrian destination targets in occasionally traversed areas. The future movement trend of the pedestrian and the corrected destination map of the pedestrian are downsampled so that the future movement trend of the pedestrian and the destination map of the pedestrian correspond one-to-one with the dimensions of each layer of the scene location code; The modified pedestrian destination map, the pedestrian's future movement trend, and the second scene location features are incorporated into each layer of the trajectory decoder to decode the pedestrian trajectory.

2. The method according to claim 1, characterized in that, The process of obtaining pedestrian scene location features based on pedestrian historical location maps and global scene maps includes: The pedestrian historical location map and the global scene map are stitched together and used as the input to the pedestrian scene location encoder. By using a pedestrian scene location encoder, the scene location is encoded to obtain the scene location features of pedestrians.

3. The method according to claim 1, characterized in that, The process of modeling the historical observation trajectory of the pedestrian to generate the pedestrian's future movement trend includes: The pedestrian's historical observation trajectory is trend-encoded and decoded using a multilayer perceptron to generate the pedestrian's future movement trend; The future movement trend of the pedestrians is represented as follows: in, This represents a motion trend encoder composed of a multi-layer sensing mechanism. For the historical observation trajectory of pedestrians, Parameters representing the motion trend encoder, This represents a motion trend decoder composed of a multi-layer sensing mechanism. The parameters represent the motion trend decoder. This indicates the future movement trend of the pedestrians generated in the final version.

4. The method according to claim 1, characterized in that, The first scene location feature and the second scene location feature are extracted from the scene location features of the pedestrian using an attention mechanism.

5. The method according to claim 1, characterized in that, The pedestrian destination map is represented as follows: in, This represents the generated pedestrian destination map. Represents the endpoint target decoder based on convolutional layers. The parameters represent the endpoint target decoder. This represents the first scene location features related to the endpoint target.

6. The method according to claim 1, characterized in that, The pedestrian trajectory is represented as follows: in, Indicates the location features of the second scene For the future movement trends of pedestrians, A destination map for pedestrians. This indicates a splicing operation. For trajectory decoder, These are the parameters for the trajectory decoder. This represents the pedestrian's trajectory.

7. A pedestrian long-term trajectory prediction device, characterized in that, The pedestrian long-term trajectory prediction method according to any one of claims 1-6, wherein the apparatus comprises at least: The acquisition module is used to acquire pedestrian historical location maps, global scene maps, and pedestrian historical observation trajectories; The scene location encoding module is used to obtain the scene location features of pedestrians based on the pedestrian historical location map and the global scene map. The scene location features of pedestrians include a first scene location feature related to the destination target and a second scene location feature related to the trajectory. The pedestrian movement trend modeling module is used to model the historical observation trajectory of the pedestrian and generate the future movement trend of the pedestrian. The pedestrian trajectory generation module predicts pedestrian trajectories based on the pedestrian's future movement trends and scene location features, and includes: Based on the location features of the first scene, a destination map of the pedestrian is obtained; The various areas in the global scene map are divided into three categories: frequently traversed areas, occasionally traversed areas, and prohibited areas. The scene layout correction map is constructed by classifying the three categories of traversed areas according to the probability of pedestrian passage from high to low. The pedestrian destination target map is modified using the scene layout correction map, discarding pedestrian destination targets in restricted areas, resulting in the modified pedestrian destination target map. The modified pedestrian destination target map includes pedestrian destination targets in frequently traversed areas and pedestrian destination targets in occasionally traversed areas. The future movement trend of the pedestrian and the corrected destination map of the pedestrian are downsampled so that the future movement trend of the pedestrian and the destination map of the pedestrian correspond one-to-one with the dimensions of each layer of the scene location code; The modified pedestrian destination map, the pedestrian's future movement trend, and the second scene location features are incorporated into each layer of the trajectory decoder to decode the pedestrian trajectory.

Images