Dynamic target trajectory prediction method, system and device, and storage medium
Through the dynamic target trajectory prediction method, the elliptical model and generative adversarial network are used to predict pedestrian trajectories in complex crowd environments, which solves the problems of path planning freezing and high computing resource consumption in mobile robot navigation, and improves the efficiency and security of the navigation system.
Patent Information
- Application Number
- CN202411425067.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2025-01-07
AI Technical Summary
In dynamic environments, it is difficult for mobile robot navigation systems to efficiently and safely carry out path planning, especially in complex pedestrian environments. The existing technology faces the problems of large computing resources, long training time and high hardware requirements.
Using dynamic target trajectory prediction method, by obtaining pedestrian motion data, building an elliptical model to characterize the motion comfort zone, perform feature clustering, use variational automatic encoder and attention pooling module to generate pedestrian trajectory prediction results, and combine it with the generation adversarial network. Make predictions.
It improves the accuracy of pedestrian trajectory prediction in dynamic and complex scenarios, takes into account the relationship between different pedestrians in complex populations, and enhances the efficiency and safety of the navigation system.
Smart Images
Figure CN119273001A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of trajectory prediction, and in particular to a method, system, device, and storage medium for dynamic target trajectory prediction. Background Art
[0002] With the rapid development of science and technology, mobile robots are increasingly used in daily life, industrial production, medical care, military exploration and other fields. Among them, efficient and safe navigation system is one of the key technologies to realize autonomous movement of robots.
[0003] At present, the navigation system of mobile robots mainly relies on a variety of sensors (such as lidar, cameras, ultrasonic sensors, etc.) to collect information about the surrounding environment, and processes this information through algorithms for path planning. However, in actual applications, when facing dynamic environments, path planning through sensors is prone to freezing problems or requires sharp turns, which affects navigation efficiency and safety. In addition, in order to cope with dynamic environments, deep reinforcement learning (DRL), as a powerful machine learning technology, has also been widely used in mobile robot navigation tasks. However, long-term planning methods, including complex strategies based on DRL, often face challenges such as high consumption of computing resources, long training time, and high hardware requirements.
[0004] Therefore, how to predict trajectories in a dynamic and complex pedestrian environment so as to navigate a mobile robot has become a technical problem that needs to be solved urgently in this field.
[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention
[0006] The main purpose of this application is to provide a dynamic target trajectory prediction method, system, device, and storage medium, aiming to solve the technical problem of how to perform trajectory prediction for a dynamic and complex pedestrian environment.
[0007] To achieve the above objectives, the present application proposes a dynamic target trajectory prediction method, the method comprising:
[0008] Obtain motion data of multiple pedestrians;
[0009] For each of the pedestrians, constructing an elliptical model according to the motion data, wherein the elliptical model is used to characterize the motion comfort zone of the pedestrian;
[0010] Performing feature clustering based on the motion data to obtain pedestrian groups among the pedestrians;
[0011] The historical trajectory of each target pedestrian in the pedestrian group is encoded by a variational autoencoder to obtain a feature code of each target pedestrian;
[0012] The influence weight of each target pedestrian is calculated by the attention pooling module;
[0013] The pedestrian group, the ellipse models corresponding to the pedestrian group, the feature codes and the influence weights are input into a preset generative adversarial network model to obtain a pedestrian trajectory prediction result of the pedestrian group.
[0014] In one embodiment, after the step of obtaining the pedestrian trajectory prediction result of the pedestrian group, the method further includes:
[0015] For each of the target pedestrians, when the target pedestrian is detected, initializing an intention hypothesis particle;
[0016] Extracting an implicit feature vector of the historical trajectory of the target pedestrian through a preset deep learning model;
[0017] Predicting the target pedestrian's movement intention according to the implicit feature vector, wherein the movement intention includes: movement position, movement direction and movement speed;
[0018] The pedestrian trajectory prediction result is updated according to the movement intention of each target pedestrian.
[0019] In one embodiment, the step of initializing the intention hypothesis particle includes:
[0020] Initialize the motion state of the intended assumed particle through Gaussian distribution;
[0021] The weights of the intention hypothesis particles are evenly distributed so that the initial weights of the intention hypothesis particles are the same.
[0022] In one embodiment, the step of obtaining motion data of multiple pedestrians includes:
[0023] Acquire environmental images in real time through cameras;
[0024] Calculating motion vectors of multiple pedestrians in the environment image in two consecutive frames of environment images by an optical flow method;
[0025] The motion speed and the motion direction in the motion vector are used as the motion data of each of the pedestrians.
[0026] In one embodiment, the step of constructing an elliptical model according to the motion data includes:
[0027] Setting the motion direction in the motion data as the major axis direction of the ellipse model;
[0028] The ellipse model is constructed based on the motion speed in the motion data.
[0029] In one embodiment, the step of calculating the influence weight of each target pedestrian by the attention pooling module includes:
[0030] For each first target pedestrian among the target pedestrians, calculating the trajectory influence weights of other target pedestrians in the pedestrian group compared with the first target pedestrian;
[0031] The influence weight of each trajectory is used as the influence weight of each target pedestrian.
[0032] In addition, to achieve the above purpose, the present application also proposes a dynamic target trajectory prediction system, the dynamic target trajectory prediction system comprising:
[0033] A data acquisition module, used to acquire motion data of multiple pedestrians;
[0034] A personal modeling module, configured to construct an elliptical model for each of the pedestrians according to the motion data, wherein the elliptical model is used to characterize the motion comfort zone of the pedestrian;
[0035] A group modeling module, used for performing feature clustering based on the motion data to obtain a pedestrian group among the pedestrians;
[0036] An encoding module, used for encoding the historical trajectory of each target pedestrian in the pedestrian group by a variational autoencoder to obtain a feature code of each target pedestrian;
[0037] An attention pooling module, used for calculating the influence weight of each target pedestrian in the attention pooling module;
[0038] The trajectory prediction module is used to input the pedestrian group, the elliptical models corresponding to the pedestrian group, the feature codes and the influence weights into a preset generative adversarial network model to obtain the pedestrian trajectory prediction result of the pedestrian group.
[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a dynamic target trajectory prediction device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the dynamic target trajectory prediction method as described above.
[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the dynamic target trajectory prediction method described above are implemented.
[0041] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the dynamic target trajectory prediction method as described above are implemented.
[0042] The present application provides a dynamic target trajectory prediction method. The present application constructs an elliptical model for each detected pedestrian, determines the motion comfort zone of each pedestrian, then finds the clustered pedestrian group through feature clustering, and then predicts the pedestrian trajectory based on the historical motion trajectory and influence weight of each pedestrian in the pedestrian group and the elliptical model.
[0043] In summary, the present application can dynamically model the personal comfort zone of pedestrians through an adaptive elliptical model, and effectively predict pedestrian trajectories by combining generative adversarial networks (GANs) and attention mechanisms. Compared with traditional dynamic target trajectory prediction methods, the present application takes into account the relationships between different pedestrians in a complex crowd, and can model the motion trajectory of each pedestrian based on the analysis of the group, thereby improving the accuracy of pedestrian trajectory prediction in dynamic and complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0046] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the method for predicting dynamic target trajectory of the present application;
[0047] Figure 2 This is a schematic diagram of the module structure of the dynamic target trajectory prediction system according to an embodiment of the present application;
[0048] Figure 3 Schematic diagram of the device structure of the hardware operating environment involved in the dynamic target trajectory prediction method in the embodiment of the present application.
[0049] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0050] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.
[0051] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.
[0052] The main solution of the embodiment of the present application is: obtaining motion data of multiple pedestrians; for each pedestrian, constructing an elliptical model according to the motion data, wherein the elliptical model is used to characterize the motion comfort zone of the pedestrian; performing feature clustering based on the motion data to obtain pedestrian groups among the pedestrians; encoding the historical trajectory of each target pedestrian in the pedestrian group through a variational autoencoder to obtain the feature code of each target pedestrian; calculating the influence weight of each target pedestrian through an attention pooling module; inputting the pedestrian group, the elliptical models corresponding to the pedestrian group, the feature code and the influence weight into a preset generative adversarial network model to obtain the pedestrian trajectory prediction result of the pedestrian group.
[0053] In this embodiment, for ease of description, the following description is made with the dynamic target trajectory prediction device as the execution subject.
[0054] With the rapid development of science and technology, mobile robots are increasingly used in daily life, industrial production, medical care, military exploration and other fields. Among them, efficient and safe navigation system is one of the key technologies to realize autonomous movement of robots.
[0055] At present, the navigation system of mobile robots mainly relies on a variety of sensors (such as lidar, cameras, ultrasonic sensors, etc.) to collect information about the surrounding environment, and processes this information through algorithms for path planning. However, in actual applications, when facing dynamic environments, path planning through sensors is prone to freezing problems or requires sharp turns, which affects navigation efficiency and safety. In addition, in order to cope with dynamic environments, deep reinforcement learning (DRL), as a powerful machine learning technology, has also been widely used in mobile robot navigation tasks. However, long-term planning methods, including complex strategies based on DRL, often face challenges such as high consumption of computing resources, long training time, and high hardware requirements.
[0056] Therefore, how to predict trajectories in a dynamic and complex pedestrian environment so as to navigate a mobile robot has become a technical problem that needs to be solved urgently in this field.
[0057] In response to the above problems, the present application provides a dynamic target trajectory prediction method. The present application constructs an elliptical model for each detected pedestrian, determines the motion comfort zone of each pedestrian, and then finds the clustered pedestrian groups through feature clustering. The pedestrian trajectory is then predicted based on the historical motion trajectory and influence weight of each pedestrian in the pedestrian group and the elliptical model.
[0058] In summary, the present application can dynamically model the personal comfort zone of pedestrians through an adaptive elliptical model, and effectively predict pedestrian trajectories by combining generative adversarial networks (GANs) and attention mechanisms. Compared with traditional dynamic target trajectory prediction methods, the present application takes into account the relationships between different pedestrians in a complex crowd, and can model the motion trajectory of each pedestrian based on the analysis of the group, thereby improving the accuracy of pedestrian trajectory prediction in dynamic and complex scenarios.
[0059] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a dynamic target trajectory prediction device, etc. The following takes the dynamic target trajectory prediction device as an example to illustrate this embodiment and the following embodiments.
[0060] Based on this, the present application embodiment provides a dynamic target trajectory prediction method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the dynamic target trajectory prediction method of the present application.
[0061] In this embodiment, the dynamic target trajectory prediction method includes steps S10 to S60:
[0062] Step S10, obtaining motion data of multiple pedestrians;
[0063] In this embodiment, various sensors (such as cameras, radars, GPS devices, etc.) may be used to collect pedestrian motion data, which may include the location (such as longitude and latitude), speed, acceleration, and possible direction information of the pedestrian.
[0064] In a feasible implementation manner, the above step S10 may include steps S11 to S13:
[0065] Step S11, acquiring an environmental image in real time through a camera;
[0066] In this embodiment, in the application, it is necessary to capture the environment image containing pedestrians in real time through the camera.
[0067] Step S12, calculating motion vectors of a plurality of pedestrians in the environment image in two consecutive frames of environment images by using an optical flow method;
[0068] In this embodiment, the optical flow method is an image processing technology used to calculate the instantaneous speed of the movement of pixels in an image, i.e., the optical flow. In two consecutive frames of environmental images, the optical flow method can identify and calculate the motion vector of pedestrians (or other objects in the image), including the magnitude and direction of the speed.
[0069] As an example, the calculation formula of the optical flow method is: I(x, y, t)≈I(x+Δx, y+Δy, t+Δt), where I(x, y, t) is the motion vector, and Δx, Δy, and Δt are the changes in coordinates and vectors.
[0070] Step S13: Using the motion speed and motion direction in the motion vector as motion data of each of the pedestrians.
[0071] In this embodiment, after the motion vector of the pedestrian is obtained, the motion speed and motion direction thereof are extracted as the key motion data of the pedestrian.
[0072] Step S20, constructing an ellipse model according to the motion data for each pedestrian, wherein the ellipse model is used to characterize the motion comfort zone of the pedestrian;
[0073] In this embodiment, an ellipse model is constructed for each pedestrian using the pedestrian's motion data (such as position, speed, etc.). This ellipse model is used to characterize the pedestrian's "comfort zone" during walking, that is, the range of the area where the pedestrian may walk. This range is usually determined based on the pedestrian's historical motion trajectory and current motion state.
[0074] Furthermore, in a feasible implementation manner, the above step S20 may include steps S21 to S22:
[0075] Step S21, setting the motion direction in the motion data as the major axis direction of the ellipse model;
[0076] In this embodiment, the movement direction of the pedestrian (i.e., the direction of the velocity vector) is set to the major axis direction of the ellipse according to the extracted motion data. This means that the major axis of the ellipse will extend along the main motion path of the pedestrian. Setting the movement direction to the major axis direction of the ellipse can intuitively reflect the main motion trend of the pedestrian. This is of great significance for understanding the movement pattern of pedestrians, predicting future motion paths, and analyzing the interactive behaviors between pedestrians. In addition, this setting also makes the ellipse model more consistent with the actual motion characteristics of pedestrians, improving the accuracy and practicality of the model.
[0077] Step S22: constructing the ellipse model based on the motion speed in the motion data.
[0078] In this embodiment, after determining the direction of the major axis of the ellipse, it is necessary to further construct the shape and size of the ellipse model. Specifically, the magnitude of the movement speed can be used as a reference factor for the semi-major axis of the ellipse, and the size of the semi-minor axis can be set by considering the comfort of the pedestrian and possible walking deviation. In addition, the size and shape of the ellipse can also be dynamically adjusted according to dynamic information such as the acceleration and deceleration of the pedestrian.
[0079] Specifically, the ellipse model can be expressed by the following formula:
[0080] σ x =(f 0 +v·f v )·σ 0 ;
[0081] Among them, σ x and σ y are the standard deviations of the ellipse model in the x and y directions, f 0 is a constant, v is the speed of the pedestrian, and f v is the speed influence factor, σ 0 is the basic standard deviation.
[0082] Step S30, performing feature clustering based on the motion data to obtain pedestrian groups among the pedestrians;
[0083] In this embodiment, a clustering algorithm (such as K-means, DBSCAN, etc.) is used to perform feature clustering on the motion data of pedestrians to identify groups of pedestrians with similar motion features. These groups can be divided based on features such as walking speed, direction, and position.
[0084] As an example, by inputting pedestrian position and motion information into an asymmetric Gaussian function, the strength of the relationship between pedestrians is calculated, and a social relationship graph is generated to capture the dynamic interactive relationship of the pedestrian group.
[0085] Calculation formula:
[0086]
[0087] Among them, (x i ,y i ) and (x j ,y j ) are the positions of pedestrians i and j respectively, and σ is the standard deviation.
[0088] Step S40, encoding the historical trajectory of each target pedestrian in the pedestrian group by a variational autoencoder to obtain a feature code of each target pedestrian;
[0089] In this embodiment, a variational autoencoder (VAE) is used to encode the historical trajectory of each target pedestrian in the pedestrian group to generate a low-dimensional feature encoding. This encoding contains the potential information and features of the pedestrian trajectory.
[0090] Step S50, calculating the influence weight of each target pedestrian through an attention pooling module;
[0091] In this embodiment, the attention pooling module calculates the influence weight of each target pedestrian in the pedestrian group based on the feature encoding of the target pedestrian and possible context information (such as the position and speed of other pedestrians). This weight reflects the importance of the pedestrian in predicting future trajectories.
[0092] Furthermore, in a feasible implementation manner, the above step S50 may include steps S51 to S52:
[0093] Step S51, for each first target pedestrian among the target pedestrians, calculating the trajectory influence weights of other target pedestrians in the pedestrian group compared with the first target pedestrian;
[0094] In this embodiment, in the dynamic target trajectory prediction system, for each individual identified as a target pedestrian (referred to as the "first target pedestrian"), the system calculates the potential influence weight of other target pedestrians in the pedestrian group on their trajectory. This calculation process is based on a variety of factors, including but not limited to the physical distance between other pedestrians and the first target pedestrian, speed difference, consistency of travel direction, and historical interaction patterns.
[0095] Step S52: using the influence weights of the trajectories as the influence weights of the target pedestrians.
[0096] In this embodiment, after obtaining the influence weights of each trajectory, the system will directly use the calculated trajectory influence weights as the influence weights of each target pedestrian. This means that for each target pedestrian, the system will generate a weight set associated with it, which includes the influence weights of each other pedestrian in the group on him.
[0097] Specifically, the attention pooling module effectively extracts the interaction information between pedestrians by calculating the influence weight of each pedestrian on the future trajectory of the target pedestrian, thereby better describing the dynamic relationship between pedestrians. This module helps the model generate more accurate trajectory predictions by paying attention to the degree of influence of different pedestrians on the target pedestrian. In order to better describe the influence between pedestrians, we designed an attention pooling module that combines social characteristics. This module calculates the influence weight of each pedestrian on the future trajectory of the target pedestrian through a multi-layer fully connected network and a softmax activation function. The attention mechanism is used to obtain the weight of the influence of all people in the scene on the future trajectory of the target pedestrian, and summarize all the information required for the target pedestrian to make a decision, thereby achieving the purpose of pedestrian interaction modeling. The specific formula is calculated as follows:
[0098] q ij =[v j ,d ij ,cosθ ij ,cosω ij ];
[0099] q i =[q i1 ,q i2 ,...,q ij ,...,q iN ];
[0100] M = s(q i ; W);
[0101] F i =maxpool(M,H i );
[0102] Where v represents the pedestrian velocity vector, d ij represents the distance vector between pedestrians, θ ij represents the cosine value of the angle between the velocity vector of pedestrian i and the distance vector between pedestrians i and j, ω ij represents the cosine value of the angle between the velocity vectors of pedestrians i and j, q ij represents the eigenvector of the influence of j on i, H i represents the set of hidden feature vectors of i, s(·) represents softmax, and W is the learning weight value.
[0103] Step S60: input the pedestrian group, the ellipse models corresponding to the pedestrian group, the feature codes and the influence weights into a preset generative adversarial network model to obtain a pedestrian trajectory prediction result of the pedestrian group.
[0104] In this embodiment, the pedestrian group, ellipse model, feature encoding and influence weight obtained in the previous steps are input into a preset generative adversarial network (GAN) model. The GAN model generates predicted pedestrian trajectories through adversarial training of the generator and the discriminator.
[0105] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction, and will not be repeated later. On this basis, after the above step S60, the method can also include steps A10 to A40:
[0106] Step A10, for each of the target pedestrians, when the target pedestrian is detected, initializing the intention hypothesis particle;
[0107] In this embodiment, in the dynamic target trajectory prediction system, whenever a new target pedestrian is detected, the system will immediately initialize the intention hypothesis particles related to the pedestrian. These particles represent the system's initial assumptions about the target pedestrian's possible future motion intentions, providing a basis for subsequent motion intention predictions.
[0108] Further, in a feasible implementation manner, the step of initializing the intended hypothetical particles in the above step A10 may include steps A11 to A12:
[0109] Step A11, initializing the motion state of the intended assumed particle through Gaussian distribution;
[0110] In this embodiment, in the dynamic target trajectory prediction system, when a new target pedestrian is detected, the system will first initialize the motion state of the intention hypothesis particles related to the pedestrian through Gaussian distribution. These motion states include key parameters such as the position, speed and direction of the particles, which together constitute the particles' initial assumptions about the target pedestrian's future motion intentions.
[0111] The Gaussian distribution is chosen because it can simulate the probability distribution of many random phenomena in nature and is easy to implement and calculate. Through Gaussian distribution initialization, the system can give the intended particle a reasonable initial motion state range, making the subsequent prediction and update process more stable and reliable.
[0112] Step A12, uniformly distributing the weights of the intention hypothesis particles so that the initial weights of the intention hypothesis particles are the same.
[0113] In this embodiment, after initializing the motion state of the intention hypothesis particles, the system will evenly distribute the weights of these particles. This means that each particle has the same weight in the initial stage, reflecting that the system has the same degree of confidence in the initial assumption of each particle's possible future motion intention.
[0114] The uniform distribution of weights helps the system to fairly consider the contribution of each particle in the subsequent prediction and update process, avoiding the deviation of prediction results due to the imbalance of initial weights.
[0115] Step A20, extracting an implicit feature vector of the historical trajectory of the target pedestrian through a preset deep learning model;
[0116] In this embodiment, the system uses a preset deep learning model to extract features from the historical trajectory of the target pedestrian and obtain an implicit feature vector.
[0117] Step A30, predicting the target pedestrian's movement intention according to the implicit feature vector, wherein the movement intention includes: movement position, movement direction and movement speed;
[0118] In this embodiment, based on the extracted implicit feature vectors, the system further predicts the target pedestrian's movement intentions, which include information such as movement position, movement direction, and movement speed.
[0119] Step A40: updating the pedestrian trajectory prediction result according to the movement intention of each target pedestrian.
[0120] In this embodiment, after obtaining the movement intention of each target pedestrian, the system will update the pedestrian trajectory prediction results according to these intentions. This includes adjusting parameters such as the shape, direction and speed of the predicted trajectory to more accurately reflect the future movement trend of the target pedestrian.
[0121] Specifically, as an example, in order to meet the requirements of social navigation for long-term multimodal trajectory prediction of pedestrians, pedestrian intentions can be introduced, and a variable intention filter based on deep learning and particle filtering is proposed. The filter performs weighted sampling of particles of intention assumptions and intention adjustment of the deep learning model, so that the generator can maintain accurate inference of pedestrian intentions over a long time span.
[0122] 1) Particle initialization: When a pedestrian is detected, the intention hypothesis particles are initialized, and each particle represents a possible intention hypothesis. A deep learning model (such as LSTM) is used to extract the pedestrian's historical trajectory features and initialize the particle weights.
[0123] Particle initialization: Use Gaussian distribution to initialize particle state (position, speed, direction).
[0124] Particle weight: Initially, the weights of all particles are evenly distributed.
[0125] 2) Potential intention generation: Combined with the prediction results of the deep learning model, the weights of the particles are dynamically adjusted. At each time step, the possible intention of the next step is predicted by the deep learning model, and the particle filtering method is used to update the particles.
[0126] Historical trajectory encoding: Use LSTM to encode pedestrian historical trajectories and generate implicit feature vectors.
[0127] Intention prediction: Based on the implicit feature vector, the pedestrian’s future movement intention (such as position, speed, and direction) is predicted.
[0128] 3) Particle filtering: Particle filtering is performed on the entire observation history to maintain belief in all potential intention hypotheses and prevent premature convergence of motion prediction due to intention variation.
[0129] Prediction step: According to the pedestrian motion model, the state of each particle is predicted to generate possible future states.
[0130] Update step: Based on the observed data (such as sensor data), the weight of each particle is updated to reflect the degree of match between the predicted trajectory and the true trajectory.
[0131] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the dynamic target trajectory prediction method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.
[0132] This application also provides a dynamic target trajectory prediction system, please refer to Figure 2 , the dynamic target trajectory prediction system comprises:
[0133] A data acquisition module 10, used to acquire motion data of multiple pedestrians;
[0134] A personal modeling module 20, configured to construct an elliptical model for each of the pedestrians according to the motion data, wherein the elliptical model is used to characterize the motion comfort zone of the pedestrian;
[0135] A group modeling module 30, configured to perform feature clustering based on the motion data to obtain a pedestrian group among the pedestrians;
[0136] An encoding module 40, configured to encode the historical trajectory of each target pedestrian in the pedestrian group by using a variational autoencoder to obtain a feature code of each target pedestrian;
[0137] An attention pooling module 50, used for calculating the influence weight of each target pedestrian in the attention pooling module calculation;
[0138] The trajectory prediction module 60 is used to input the pedestrian group, the elliptical models corresponding to the pedestrian group, the feature codes and the influence weights into a preset generative adversarial network model to obtain a pedestrian trajectory prediction result of the pedestrian group.
[0139] Optionally, the dynamic target trajectory prediction system is also used to:
[0140] For each of the target pedestrians, when the target pedestrian is detected, initializing an intention hypothesis particle;
[0141] Extracting an implicit feature vector of the historical trajectory of the target pedestrian through a preset deep learning model;
[0142] Predicting the target pedestrian's movement intention according to the implicit feature vector, wherein the movement intention includes: movement position, movement direction and movement speed;
[0143] The pedestrian trajectory prediction result is updated according to the movement intention of each target pedestrian.
[0144] Optionally, the dynamic target trajectory prediction system is also used to:
[0145] Initialize the motion state of the intended assumed particle through Gaussian distribution;
[0146] The weights of the intention hypothesis particles are evenly distributed so that the initial weights of the intention hypothesis particles are the same.
[0147] Optionally, the data acquisition module is also used to:
[0148] Acquire environmental images in real time through cameras;
[0149] Calculating motion vectors of multiple pedestrians in the environment image in two consecutive frames of environment images by an optical flow method;
[0150] The motion speed and the motion direction in the motion vector are used as the motion data of each of the pedestrians.
[0151] Optionally, the Personal Modeling Module is also used to:
[0152] Setting the motion direction in the motion data as the major axis direction of the ellipse model;
[0153] The ellipse model is constructed based on the motion speed in the motion data.
[0154] Optionally, the attention pooling module is also used to:
[0155] For each first target pedestrian among the target pedestrians, calculating the trajectory influence weights of other target pedestrians in the pedestrian group compared with the first target pedestrian;
[0156] The influence weight of each trajectory is used as the influence weight of each target pedestrian.
[0157] The dynamic target trajectory prediction system provided by the present application adopts the dynamic target trajectory prediction method in the above embodiment, which can solve the technical problem of how to perform trajectory prediction for a dynamic and complex pedestrian environment. Compared with the prior art, the beneficial effects of the dynamic target trajectory prediction system provided by the present application are the same as the beneficial effects of the dynamic target trajectory prediction method provided by the above embodiment, and other technical features of the dynamic target trajectory prediction system are the same as the features disclosed in the above embodiment method, which will not be repeated here.
[0158] The present application provides a dynamic target trajectory prediction device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the dynamic target trajectory prediction method in the above-mentioned embodiment 1.
[0159] Reference below Figure 3 , which shows a schematic diagram of the structure of a dynamic target trajectory prediction device suitable for implementing the embodiment of the present application. The dynamic target trajectory prediction device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The dynamic target trajectory prediction device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0160] like Figure 3As shown, the dynamic target trajectory prediction device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the dynamic target trajectory prediction device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the dynamic target trajectory prediction device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a dynamic target trajectory prediction device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.
[0161] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0162] The dynamic target trajectory prediction device provided by the present application adopts the dynamic target trajectory prediction method in the above embodiment, which can solve the technical problem of how to perform trajectory prediction for a dynamic and complex pedestrian environment. Compared with the prior art, the beneficial effects of the dynamic target trajectory prediction device provided by the present application are the same as the beneficial effects of the dynamic target trajectory prediction method provided by the above embodiment, and the other technical features in the dynamic target trajectory prediction device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0163] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0164] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0165] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the dynamic target trajectory prediction method in the above-mentioned embodiment.
[0166] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.
[0167] The computer-readable storage medium may be included in the dynamic target trajectory prediction device; or may exist independently without being assembled into the dynamic target trajectory prediction device.
[0168] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the dynamic target trajectory prediction device, the dynamic target trajectory prediction device: obtains motion data of multiple pedestrians; for each of the pedestrians, constructs an elliptical model according to the motion data, wherein the elliptical model is used to characterize the motion comfort zone of the pedestrian; performs feature clustering based on the motion data to obtain pedestrian groups among the pedestrians; encodes the historical trajectory of each target pedestrian in the pedestrian group through a variational autoencoder to obtain the feature code of each target pedestrian; calculates the influence weight of each target pedestrian through an attention pooling module; inputs the pedestrian group, the elliptical models corresponding to the pedestrian group, the feature code and the influence weight into a preset generative adversarial network model to obtain the pedestrian trajectory prediction result of the pedestrian group.
[0169] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0170] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0171] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0172] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned dynamic target trajectory prediction method, and can solve the technical problem of how to perform trajectory prediction for a dynamic and complex pedestrian environment. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the dynamic target trajectory prediction method provided by the above-mentioned embodiment, and will not be repeated here.
[0173] The present application also provides a computer program product, including a computer program, which implements the steps of the dynamic target trajectory prediction method as described above when executed by a processor.
[0174] The computer program product provided by the present application can solve the technical problem of how to perform trajectory prediction for a dynamic and complex pedestrian environment. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the dynamic target trajectory prediction method provided by the above embodiment, which will not be repeated here.
[0175] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.
Claims
1. A dynamic target trajectory prediction method, characterized in that: The method comprises: Obtain motion data of multiple pedestrians; For each of the pedestrians, constructing an elliptical model according to the motion data, wherein the elliptical model is used to characterize the motion comfort zone of the pedestrian; Performing feature clustering based on the motion data to obtain pedestrian groups among the pedestrians; The historical trajectory of each target pedestrian in the pedestrian group is encoded by a variational autoencoder to obtain a feature code of each target pedestrian; The influence weight of each target pedestrian is calculated by the attention pooling module; The pedestrian group, the ellipse models corresponding to the pedestrian group, the feature codes and the influence weights are input into a preset generative adversarial network model to obtain a pedestrian trajectory prediction result of the pedestrian group.
2. The dynamic target trajectory prediction method according to claim 1, characterized in that: After the step of obtaining the pedestrian trajectory prediction result of the pedestrian group, the method further includes: For each of the target pedestrians, when the target pedestrian is detected, initializing an intention hypothesis particle; Extracting an implicit feature vector of the historical trajectory of the target pedestrian through a preset deep learning model; Predicting the target pedestrian's movement intention according to the implicit feature vector, wherein the movement intention includes: movement position, movement direction and movement speed; The pedestrian trajectory prediction result is updated according to the movement intention of each target pedestrian.
3. The dynamic target trajectory prediction method according to claim 2, characterized in that: The steps of initializing the intention hypothesis particles include: Initialize the motion state of the intended assumed particle through Gaussian distribution; The weights of the intention hypothesis particles are evenly distributed so that the initial weights of the intention hypothesis particles are the same.
4. The dynamic target trajectory prediction method according to claim 1, characterized in that: The step of obtaining motion data of multiple pedestrians includes: Acquire environmental images in real time through cameras; Calculating motion vectors of multiple pedestrians in the environment image in two consecutive frames of environment images by an optical flow method; The motion speed and the motion direction in the motion vector are used as the motion data of each of the pedestrians.
5. The dynamic target trajectory prediction method according to claim 1, characterized in that: The step of constructing an ellipse model according to the motion data comprises: Setting the motion direction in the motion data as the major axis direction of the ellipse model; The ellipse model is constructed based on the motion speed in the motion data.
6. The dynamic target trajectory prediction method according to claim 1, characterized in that: The step of calculating the influence weight of each target pedestrian by the attention pooling module includes: For each first target pedestrian among the target pedestrians, calculating the trajectory influence weights of other target pedestrians in the pedestrian group compared with the first target pedestrian; The influence weight of each trajectory is used as the influence weight of each target pedestrian.
7. A dynamic target trajectory prediction system, characterized in that: The system comprises: A data acquisition module, used to acquire motion data of multiple pedestrians; A personal modeling module, configured to construct an elliptical model for each of the pedestrians according to the motion data, wherein the elliptical model is used to characterize the motion comfort zone of the pedestrian; A group modeling module, used for performing feature clustering based on the motion data to obtain a pedestrian group among the pedestrians; An encoding module, used for encoding the historical trajectory of each target pedestrian in the pedestrian group by a variational autoencoder to obtain a feature code of each target pedestrian; An attention pooling module, used for calculating the influence weight of each target pedestrian in the attention pooling module; The trajectory prediction module is used to input the pedestrian group, the elliptical models corresponding to the pedestrian group, the feature codes and the influence weights into a preset generative adversarial network model to obtain the pedestrian trajectory prediction result of the pedestrian group.
8. A dynamic target trajectory prediction device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the dynamic target trajectory prediction method according to any one of claims 1 to 6.
9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the dynamic target trajectory prediction method according to any one of claims 1 to 6 are implemented.
10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the dynamic target trajectory prediction method according to any one of claims 1 to 6 are implemented.