Motion mode guided pedestrian trajectory prediction method and device based on conditional diffusion
By using conditional diffusion models in pedestrian trajectory prediction, combining clustering and frequency domain transformation to generate multimodal future trajectory, the problems of trajectory consistency and high computational cost in the existing methods are solved, and efficient and accurate pedestrian trajectory prediction is achieved.
Patent Information
- Application Number
- CN202510589269.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2025-08-26
AI Technical Summary
The existing pedestrian trajectory prediction methods are difficult to ensure the consistency between the trajectory and the real motion pattern, and the traditional diffusion model is expensive to calculate.
By obtaining pedestrian historical trajectory data, using clustering algorithms to label motion mode pseudo-labels, combining frequency domain transformation and conditional diffusion models, a condition-guided diffusion model is constructed for reverse denoising, and a multimodal future trajectory is generated.
Improve the accuracy and diversity of trajectory prediction, maintain consistency between the generated trajectory and the real motion pattern, reduce calculation costs, and is suitable for real-time systems.
Smart Images
Figure CN120541518A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer vision and artificial intelligence security technology, and in particular to a method, device, storage medium, and electronic device for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion. Background Art
[0002] Pedestrian trajectory prediction has important application value in fields such as autonomous driving and video surveillance.
[0003] While existing methods for predicting pedestrian trajectories (such as GANs and CVAEs) can generate multimodal trajectories, they struggle to ensure that the trajectories are consistent with actual motion patterns. While diffusion models excel in generation tasks, their unconditional nature can cause trajectories to deviate from actual behavior patterns. Furthermore, traditional diffusion models require extensive denoising steps, resulting in high computational costs. Therefore, incorporating prior knowledge of motion patterns and improving generation efficiency remains a challenge in current research. Summary of the Invention
[0004] The embodiments of the present application provide a method, device, storage medium, and electronic device for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion, which can effectively improve the accuracy and diversity of trajectory prediction while maintaining the consistency of the generated trajectory with the actual motion pattern.
[0005] The present application provides a method for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion, including: Obtain a pedestrian historical trajectory data set; the pedestrian historical trajectory data set includes a plurality of pedestrian historical trajectory data and pedestrian future trajectory data; The pedestrian historical trajectory data is labeled by a clustering algorithm to obtain a motion pattern pseudo label; Convert the pedestrian historical trajectory data from the time domain to the frequency domain to extract low-frequency motion trend features; Inputting the low-frequency motion trend features into a pedestrian motion pattern classifier to predict the probability of the motion pattern corresponding to the pedestrian historical trajectory data; calculating a loss function based on the probability of the motion pattern and the motion pattern pseudo-label, and iteratively training the pedestrian motion pattern classifier; Constructing a conditional guided diffusion model based on the motion pattern, performing reverse denoising based on the conditional guided diffusion model, and generating a multimodal future trajectory; calculating a joint loss function based on the multimodal future trajectory and the pedestrian's future trajectory data, and training the conditional guided diffusion model; The trajectory of the pedestrian to be detected is predicted based on the trained pedestrian motion pattern classifier and the conditional guided diffusion model.
[0006] Furthermore, in the above-mentioned motion pattern guided pedestrian trajectory prediction method based on conditional diffusion, the step of labeling the pedestrian historical trajectory data by a clustering algorithm to obtain motion pattern pseudo labels includes: Extracting the speed, acceleration, direction angle and angular velocity of the pedestrian historical trajectory data to form a feature vector; performing normalization processing on the feature vector; The K-Means algorithm is used to cluster the standardized feature vectors to obtain motion mode pseudo labels.
[0007] Furthermore, in the above-mentioned motion pattern guided pedestrian trajectory prediction method based on conditional diffusion, the method of clustering the normalized feature vectors using the K-Means algorithm to obtain motion pattern pseudo labels includes: Initialize the cluster center and randomly select K trajectories from the normalized feature vector as the initial cluster center, where K is the number of modes you want to cluster; Assign trajectories to the nearest cluster. For each trajectory, calculate its distance to all cluster centers and assign it to the nearest cluster. Update the cluster center and calculate the trajectory centroid of each cluster; Convergence is judged. If the change of cluster center is less than the threshold, the clustering is stopped; or if the maximum number of iterations is reached, the clustering is stopped; Returns the motion mode category label and cluster center for each trajectory.
[0008] Furthermore, in the above-mentioned motion pattern-guided pedestrian trajectory prediction method based on conditional diffusion, the step of converting the pedestrian historical trajectory data from the time domain to the frequency domain and extracting low-frequency motion trend features includes: Past observed trajectory coordinates corresponding to the pedestrian historical trajectory data Perform one-dimensional discrete cosine transform respectively and calculate the frequency domain coefficients:
[0009] in, represents discrete cosine transform, represents the future time step, represents the past time step, represents the Kronecker function, Indicates the number of DCT coefficients; Before Retention Low-frequency coefficients , filter out high-frequency noise; Reconstruct the time domain trajectory from the low-frequency coefficients via the inverse discrete cosine transform:
[0010] in, represents the coefficients after discrete cosine transform, The coefficient is the low-frequency motion trend characteristic.
[0011] Furthermore, in the above-mentioned motion pattern-guided pedestrian trajectory prediction method based on conditional diffusion, the low-frequency motion trend features are input into the pedestrian motion pattern classifier to predict the probability of the motion pattern corresponding to the pedestrian historical trajectory data, including: The noisy pedestrian historical trajectory data is input into the multi-layer perceptron and attention mechanism for feature extraction:
[0012]
[0013]
[0014] in, represents the trajectory data of the function noise, is a multi-layer perceptron, is the attention mechanism, is the activation function, is the probability of the predicted motion pattern, are learnable parameters.
[0015] Furthermore, the above-mentioned motion pattern-guided pedestrian trajectory prediction method based on conditional diffusion, wherein the step of constructing a conditional guided diffusion model based on the motion pattern, performing reverse denoising based on the conditional guided diffusion model, and generating a multimodal future trajectory, includes: calculating a probability gradient based on the probability of the motion pattern; A conditional guided diffusion model is generated based on the probability gradient, and reverse denoising is performed based on the conditional guided diffusion model; the conditional guided diffusion model is:
[0016] in, represents the denoising result of step t, represents the denoising result of step t-1, , and represents the hyperparameters of the diffusion model, represents the noise prediction network, represents the conditional guide step size, represents the standard deviation, represents the classifier gradient, Represents a classifier network.
[0017] Furthermore, in the above-mentioned motion pattern guided pedestrian trajectory prediction method based on conditional diffusion, the joint loss function is:
[0018]
[0019]
[0020] in, is the joint loss function, is the cross entropy loss function, is the end-to-end diffusion loss function, is the real motion modality label, is the predicted motion modality label, is the number of trajectories, is the number of multimodal trajectory predictions, For the real future trajectory, To predict future trajectories.
[0021] Furthermore, the above-mentioned motion pattern guided pedestrian trajectory prediction method based on conditional diffusion further includes: The original noisy trajectory data Input to the trajectory predictor to predict the Step denoising results , expressed by the following formula:
[0022]
[0023] in, For under conditions The probability distribution of is the mean, is the standard deviation, is the identity matrix, is the trajectory predictor, is the noise reduction step size.
[0024] The present application also provides a device for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion, comprising: Data acquisition module, used to obtain pedestrian historical trajectory data; A motion pattern labeling module is used to obtain a pedestrian historical trajectory data set; the pedestrian historical trajectory data set includes a plurality of pedestrian historical trajectory data and pedestrian future trajectory data; A frequency domain conversion module is used to convert the pedestrian historical trajectory data from the time domain to the frequency domain to extract low-frequency motion trend features; a motion pattern prediction module, configured to input the low-frequency motion trend features into a pedestrian motion pattern classifier to predict the probability of the motion pattern corresponding to the pedestrian historical trajectory data; calculate a loss function based on the probability of the motion pattern and the motion pattern pseudo-label, and iteratively train the pedestrian motion pattern classifier; A conditional diffusion generation module is configured to construct a conditional guided diffusion model based on the motion pattern, perform reverse denoising based on the conditional guided diffusion model, and generate a multimodal future trajectory; calculate a joint loss function based on the multimodal future trajectory and the pedestrian's future trajectory data, and train the conditional guided diffusion model.
[0025] The future trajectory prediction module predicts the trajectory of pedestrians to be detected based on the trained pedestrian motion pattern classifier and conditional guided diffusion model.
[0026] An embodiment of the present application also provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for being loaded by a processor to execute any of the above-mentioned motion pattern-guided pedestrian trajectory prediction methods based on conditional diffusion.
[0027] An embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used to perform any of the steps in the above-mentioned method for guiding pedestrian trajectory prediction based on motion patterns based on conditional diffusion.
[0028] The present application provides a motion pattern-guided pedestrian trajectory prediction method, device, storage medium, and electronic device based on conditional diffusion. This application explicitly integrates pedestrian motion patterns (such as turning and speed changes) into the diffusion model for the first time, and achieves controllable generation through clustering and classifier guidance. The discrete cosine transform (DCT) transform is introduced to extract low-frequency motion trends, effectively filter out noise, and improve trajectory smoothness and physical rationality. The optimized noise density and reduced diffusion steps solve the problem of high computational cost of traditional diffusion models and promote their application in real-time systems. Through conditional diffusion guided by motion patterns, the model can generate diverse and reasonable future trajectories, covering various behaviors such as straight driving, turning, and stationary, avoiding the limitations of a single prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] The following detailed description of the specific embodiments of the present application in conjunction with the accompanying drawings will make the technical solutions and other beneficial effects of the present application apparent.
[0030] Figure 1 Flowchart of the motion pattern-guided pedestrian trajectory prediction method based on conditional diffusion provided in an embodiment of the present application.
[0031] Figure 2This is a visualization result diagram of pedestrian trajectory prediction provided in an embodiment of the present application.
[0032] Figure 3 This is a visualization result diagram of the multimodal pedestrian trajectory prediction provided in an embodiment of the present application.
[0033] Figure 4 A schematic diagram of the structure of a motion pattern-guided pedestrian trajectory prediction device based on conditional diffusion provided in an embodiment of the present application.
[0034] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0035] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.
[0036] Pedestrian behavior is inherently multimodal and diverse, necessitating a balance between accuracy and diversity in prediction. However, real-world scenarios present complex dynamic environments, requiring predictions to consider multiple possible outcomes, such as going straight, turning left, turning right, or staying still. To address this complexity, multimodal prediction has become a widely adopted paradigm in research. Previously proposed generative models, such as GANs and CVAEs, have demonstrated the potential to mitigate uncertainty through multimodal prediction. In recent years, diffusion models have garnered significant attention due to their impressive performance in various generative tasks, including trajectory prediction. Diffusion models offer stable training and interpretability, making them an effective framework for modeling generative tasks. With advances in diffusion models, conditional diffusion models have been proposed to achieve specific generative goals, resulting in controllable and high-quality results. However, the application of such models to trajectory prediction remains relatively rare. Identifying key control conditions in trajectory prediction remains an important research direction.
[0037] This application rethinks multimodal prediction and finds that it implicitly decouples pedestrian motion patterns. These patterns describe typical behaviors such as walking speed, path selection, and social interactions. Learning motion patterns can help reduce the uncertainty of random trajectory predictions. Pedestrian trajectories are not completely random, but are influenced by consistent motion patterns that reflect the typical navigation behavior of individuals in the environment. For example, pedestrians generally walk in a straight line when there are no obstacles, or move along the edge of the sidewalk. These predictable patterns provide a basis for predicting future movements. By capturing and analyzing such patterns, models can improve prediction accuracy, which benefits applications such as traffic management, pedestrian safety, and autonomous driving systems.
[0038] The present invention provides a method, device, storage medium, and electronic device for predicting pedestrian trajectories using motion patterns guided by conditional diffusion. The present invention provides a device for predicting pedestrian trajectories using motion patterns guided by conditional diffusion. The device can be integrated into an electronic device, such as a terminal or server. The terminal can include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other device.
[0039] See also Figure 1 , Figure 1 A flowchart of a method for predicting pedestrian trajectories using motion patterns guided by conditional diffusion provided in an embodiment of the present application, which is applied to an electronic device, includes the following steps: S1, obtain pedestrian historical trajectory data; the pedestrian historical trajectory data set includes several pedestrian historical trajectory data and pedestrian future trajectory data.
[0040] Step S1 specifically includes: S11, based on the video captured by a fixed camera in different scenes, divide it into several frames, select a certain number of frames as a group, identify the position of pedestrians in each frame and convert it from the image coordinate system to the world coordinate system; obtain the pedestrian trajectory in each group, that is, the world coordinate of the pedestrian at each moment ; S112, divide the pedestrian's historical trajectory and future trajectory, for pedestrians For example, its historical trajectory is expressed as , Indicates the time step of the pedestrian’s past observation trajectory; The future trajectory of , Indicates the time step of the pedestrian's future trajectory; assuming a scenario with pedestrians, then the historical observation trajectory set of all pedestrians in this scene is , the future trajectory set of all pedestrians in this scene is .
[0041] S2, annotate pedestrian historical trajectory data through clustering algorithm to obtain motion pattern pseudo labels.
[0042] In one embodiment, step S2 includes the following steps: S21, extracting the speed, acceleration, direction angle and angular velocity of the pedestrian's historical trajectory data to form a feature vector.
[0043] Specifically, the velocity is obtained by calculating the displacement and time between adjacent trajectory points , the formula is defined as follows:
[0044] Acceleration is obtained by calculating the velocity change between adjacent moments , the formula is defined as follows:
[0045] The direction angle is obtained by calculating the arc tangent of the position transformation between adjacent trajectory points , the formula is defined as follows:
[0046] The angular velocity is obtained by calculating the direction angle transformation at adjacent times , the formula is defined as follows:
[0047] Aggregate all trajectory features to form a feature vector .
[0048] S22, normalize the feature vector.
[0049] Since there are differences in magnitude between different features, in order to reduce the impact of such differences on the clustering results, all feature vectors are Perform normalization processing, and the formula is defined as follows:
[0050] in, is any trajectory feature element in the feature vector, is the mean of the corresponding trajectory feature, is the standard deviation of the corresponding trajectory feature.
[0051] S23, using the K-Means algorithm to cluster the standardized feature vectors to obtain motion mode pseudo labels.
[0052] In one embodiment, step S23 includes the following steps: S231, initializing the cluster centers, randomly selecting K trajectories in the normalized feature vector as the initial cluster centers, where K is the number of modes to be clustered; S232, assign the trajectory to the nearest cluster. For each trajectory, calculate its distance to all cluster centers and assign it to the nearest cluster; S233, update the cluster center and calculate the trajectory centroid of each cluster; S234, judging convergence, if the change of the cluster center is less than the threshold, then stop; or if the maximum number of iterations is reached, then stop clustering; S235, returning the motion mode category label and cluster center of each trajectory.
[0053] S3, converts the pedestrian historical trajectory data from the time domain to the frequency domain and extracts low-frequency motion trend features.
[0054] Specifically, the past observation trajectory coordinates corresponding to the pedestrian historical trajectory data Perform one-dimensional discrete cosine transform respectively and calculate the frequency domain coefficients:
[0055] in, represents discrete cosine transform, represents the future time step, represents the past time step, represents the Kronecker function, Indicates the number of DCT coefficients; Before Retention Low-frequency coefficients , filter out high-frequency noise; Reconstruct the time domain trajectory from the low-frequency coefficients via the inverse discrete cosine transform:
[0056] in, Represents the DCT coefficient after discrete cosine transform, The coefficient is the low-frequency motion trend characteristic.
[0057] S4, inputs the low-frequency motion trend features into the pedestrian motion pattern classifier to predict the probability of the motion pattern corresponding to the pedestrian's historical trajectory data; calculates the loss function based on the probability of the motion pattern and the motion pattern pseudo-label, and iteratively trains the pedestrian motion pattern classifier.
[0058] In one embodiment, the pedestrian motion pattern classifier includes a multi-layer perceptron and an attention mechanism. Low-frequency motion trend features are input into the pedestrian motion pattern classifier to predict the probability of the motion pattern corresponding to the pedestrian's historical trajectory data, including: The noisy pedestrian historical trajectory data is input into the multi-layer perceptron and attention mechanism for feature extraction:
[0059]
[0060]
[0061] in, represents the trajectory data of the function noise, is a multi-layer perceptron, is the attention mechanism, is the activation function, is the probability of the predicted motion pattern, are learnable parameters.
[0062] S5: Build a conditional guided diffusion model based on the motion pattern, perform reverse denoising based on the conditional guided diffusion model, and generate multimodal future trajectories; calculate a joint loss function based on the multimodal future trajectories and the pedestrian's future trajectory data, and train the conditional guided diffusion model.
[0063] In one embodiment, step S5 of constructing a conditional guided diffusion model based on the motion pattern, performing reverse denoising based on the conditional guided diffusion model, and generating a multimodal future trajectory includes the following steps: S51, calculating a probability gradient based on the probability of the motion pattern; S52, generating a conditional guided diffusion model based on the probability gradient, and performing reverse denoising based on the conditional guided diffusion model; the conditional guided diffusion model is:
[0064] in, represents the denoising result of step t, represents the denoising result of step t-1, , and represents the hyperparameters of the diffusion model, represents the noise prediction network, represents the conditional guide step size, represents the standard deviation, represents the classifier gradient, Represents a classifier network.
[0065] In one embodiment, calculating a joint loss function based on the multimodal future trajectory and the pedestrian future trajectory data in step S5 to train the conditional guided diffusion model includes the following steps: In the first stage, the classifier is trained using the pseudo-labels of motion patterns generated by K-Means clustering as supervisory signals. The cross-entropy loss function is used to minimize the difference between the predicted probability distribution and the true pseudo-label distribution:
[0066] S54, the second stage of end-to-end diffusion model training, directly optimizes the quality of the generated trajectory rather than the intermediate noise prediction, jointly optimizes the marginal loss and the joint loss, ensures the minimum distance between each predicted trajectory and the true trajectory, and ensures the matching of the overall multimodal prediction with the true trajectory:
[0067] in, is the joint loss function, is the cross entropy loss function, is the end-to-end diffusion loss function, is the pseudo label of motion modality obtained by clustering, is the predicted motion modality label, is the number of trajectories, is the number of multimodal trajectory predictions, For the real future trajectory, To predict future trajectories.
[0068] Furthermore, after step S5, the method further includes: The original noisy trajectory data Input to the trajectory predictor to predict the Step denoising results The standard diffusion model paradigm usually requires a large number of steps Recovering the real data in the back diffusion will cause serious computational consumption problems in multimodal trajectory generation. To reduce the number of denoising steps, the present invention designs a trajectory predictor based on the model structure of the classifier, which inputs the original noisy trajectory Prediction Step denoising results Optimized trajectory samples with a small amount of noise is considered as the starting point of back diffusion and only needs to execute Step denoising ( ), rather than a large number of denoising steps , this process can be expressed by the following formula:
[0069]
[0070] in, For under conditions The probability distribution of is the mean, is the standard deviation, is the identity matrix, is the trajectory predictor, is the noise reduction step size.
[0071] A specific embodiment is shown below: Datasets The present invention is trained and tested on three public pedestrian trajectory prediction datasets ETH-UCY, SDD and NBA.
[0072] Experimental setup: This paper processes the three datasets differently to divide the length of the observed and future trajectories. For ETH-UCY and SDD, this paper uses 3.2s (8 frames) of observed trajectories to predict the future 4.8s (12 frames) of trajectories. For the NBA dataset, this paper uses 2.0s (10 frames) of past historical trajectories to predict the future 4.0s (20 frames) of trajectories. To improve computational efficiency, this paper retains the previous DCT coefficients. The DCT and iDCT operations are approximated by ETH-UCY and SDD datasets. , NBA data set The present invention sets the number of denoising steps for each reverse diffusion to The model is implemented in PyTorch and trained using the AdamW optimizer with a batch size of 12 and an initial learning rate of , trained for 100 epochs on a single 4090 GPU.
[0073] This paper uses two widely used metrics to evaluate the performance of trajectory prediction: Average Displacement Error (ADE) and Final Displacement Error (FDE). ADE measures the average deviation between the predicted trajectory and the true trajectory at all time steps, while FDE specifically evaluates the accuracy of the predicted trajectory at the final time step. The formula is defined as follows:
[0074]
[0075] Quantitative Experimental Results 4.1 As shown in Table 1, our method significantly outperforms existing methods on all subsets of the ETH-UCY dataset, demonstrating superior pedestrian trajectory prediction accuracy. It is particularly noteworthy that, compared to diffusion-based methods such as MID and LED, our method, through a conditionally guided diffusion mechanism, ensures that the generated trajectories are highly consistent with the true motion patterns.
[0076] Table 1 Performance comparison results of different methods and the proposed method on the ETH-UCY dataset
[0077] 4.2 The comparison results in Table 2 show that this method significantly improves the average minADE20 / minFDE20 indicators from 7.03 / 10.65 to 6.42 / 8.09, achieving performance improvements of 8.7% and 24.0% respectively. Figure 2) further verified that the trajectories generated by this method have better social compliance and movement rationality.
[0078] Table 2 Performance comparison results of different methods and the proposed method on the SDD dataset
[0079] 4.3 As shown in Table 3, within a 4-second prediction duration, our method reduces the average ADE / FDE from 0.73 / 0.90 to 0.61 / 0.76, achieving relative performance improvements of 16.4% and 15.6%, respectively. This fully demonstrates the strong generalization capability of our method in complex motion scenes.
[0080] Table 3 Performance comparison results of different methods and the proposed method on NBA dataset
[0081] The qualitative experimental results are to more intuitively demonstrate the performance advantages of the method of the present invention. The prediction results of the current optimal method PPT (green track) are compared with the prediction results of the present method (red track) through visualization. Figure 2 The pedestrian trajectory prediction visualization result diagram provided in the embodiment of this application is as follows: Figure 2 As shown in the figure, given a historical observed trajectory (sky blue), the optimal prediction result (the yellow star represents the end point) from 20 samplings is selected and compared with the actual future trajectory (blue). The experimental results show that in group walking scenarios, traditional methods are prone to trajectory collisions (green trajectories) due to the continuous modeling of interactive relationships. However, the predicted trajectory generated by this method (red) not only accurately reflects the pedestrian's true intentions, but also conforms to social norms. Figure 3 The multimodal pedestrian trajectory prediction visualization result diagram provided in the embodiment of this application is as follows: Figure 3 The multimodal prediction visualization shown proves that the present invention can effectively capture the multimodal characteristics of trajectory prediction while maintaining the pedestrian's intrinsic motion pattern; in the tests of all five subsets of ETH-UCY, the spatial deviation between the predicted endpoint position (yellow star) and the true endpoint (blue star) of this method is significantly smaller than that of the comparison method.
[0082] This paper explores a pedestrian motion pattern-guided trajectory prediction method based on a conditional diffusion model and proposes a motion pattern-guided conditional diffusion model that achieves accelerated sampling in the frequency domain. Specifically, a clustering algorithm automatically labels trajectory motion patterns, and a classifier is trained to predict the trajectory's pattern attribution probability. Simultaneously, the discrete cosine transform (DCT) is used to convert the trajectory to the frequency domain to learn more representative pattern features. By optimizing the noise distribution and reducing the number of denoising steps, the back-diffusion process is effectively guided to generate accurate trajectories. Experiments on three public datasets (ETH-UCY, SDD, and NBA) demonstrate that this method significantly outperforms existing state-of-the-art algorithms. Quantitative and qualitative results confirm that the motion pattern-guided conditional diffusion mechanism not only improves model robustness but also enhances multimodal prediction performance.
[0083] According to the method described in the above embodiment, this embodiment will be further described from the perspective of a motion pattern-guided pedestrian trajectory prediction device based on conditional diffusion. The motion pattern-guided pedestrian trajectory prediction device based on conditional diffusion can be implemented as an independent entity or integrated into an electronic device, which can be a terminal, server, or other device. The terminal may include a tablet computer, a laptop computer, a personal computer (PC), a micro processing box, or other devices.
[0084] See also Figure 4 , Figure 4 The present invention specifically describes a device for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion, which is applied to electronic devices. The device may include: Data acquisition module, used to obtain pedestrian historical trajectory data; A motion pattern labeling module is used to obtain a pedestrian historical trajectory data set; the pedestrian historical trajectory data set includes a plurality of pedestrian historical trajectory data and pedestrian future trajectory data; A frequency domain conversion module is used to convert the pedestrian historical trajectory data from the time domain to the frequency domain to extract low-frequency motion trend features; a motion pattern prediction module, configured to input the low-frequency motion trend features into a pedestrian motion pattern classifier to predict the probability of the motion pattern corresponding to the pedestrian historical trajectory data; calculate a loss function based on the probability of the motion pattern and the motion pattern pseudo-label, and iteratively train the pedestrian motion pattern classifier; A conditional diffusion generation module is configured to construct a conditional guided diffusion model based on the motion pattern, perform reverse denoising based on the conditional guided diffusion model, and generate a multimodal future trajectory; calculate a joint loss function based on the multimodal future trajectory and the pedestrian's future trajectory data, and train the conditional guided diffusion model.
[0085] The future trajectory prediction module predicts the trajectory of pedestrians to be detected based on the trained pedestrian motion pattern classifier and conditional guided diffusion model.
[0086] During specific implementation, the above modules and / or units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above modules and / or units can refer to the previous method embodiments. The specific beneficial effects that can be achieved can also be found in the beneficial effects in the previous method embodiments, which will not be repeated here.
[0087] In addition, embodiments of the present application further provide an electronic device, which may be a computer, tablet computer, or other device. This electronic device can implement the steps of any embodiment of the motion pattern-guided pedestrian trajectory prediction method based on conditional diffusion provided in the embodiments of the present application, and thus can achieve the beneficial effects achieved by any of the motion pattern-guided pedestrian trajectory prediction methods based on conditional diffusion provided in the embodiments of the present application. For details, please refer to the previous embodiments and will not be repeated here.
[0088] Figure 5 The following figure shows a block diagram of the specific structure of an electronic device provided by an embodiment of the present invention. This electronic device can be used to implement the motion pattern-guided pedestrian trajectory prediction method based on conditional diffusion provided in the above embodiments. The electronic device 500 can be a terminal, server, or other device. The terminal can include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other device.
[0089] RF circuit 510 is used to receive and transmit electromagnetic waves, converting them into electrical signals, thereby enabling communication with a communications network or other devices. RF circuit 510 may include various existing circuit components for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption / decryption chip, a subscriber identity module (SIM) card, memory, and the like. RF circuit 510 can communicate with various networks, such as the Internet, an intranet, or a wireless network, or with other devices via a wireless network. These wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The wireless networks may utilize various communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE802.11g, and / or IEEE802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging, and short messaging, and any other suitable communication protocols, including those currently undeveloped.
[0090] The memory 520 can be used to store software programs and modules, such as the corresponding program instructions / modules in the above-mentioned embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520, that is, realizing functions such as taking pictures with the front camera, processing the captured images, and switching the display color of the displayed content on the display screen. The memory 520 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 520 may further include a memory remotely located relative to the processor 580, and these remote memories may be connected to the electronic device 500 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0091] The input unit 530 may be used to receive input digital or character information, and generate a keyboard and a mouse related to user settings and function control. The display unit 540 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces. These graphical user interfaces can be composed of graphics, text, icons, videos, or any combination thereof. The display unit 540 may include a display panel 541. Optionally, the display panel 541 can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like.
[0092] Audio circuit 560, speaker 561, and microphone 562 provide an audio interface between the user and electronic device 500. Audio circuit 560 converts received audio data into electrical signals and transmits them to speaker 561, which then converts them into sound signals for output. Microphone 562, on the other hand, converts collected sound signals into electrical signals, which are then received by audio circuit 560 and converted into audio data. The audio data is then processed by output processor 580 and transmitted via RF circuit 510 to, for example, another terminal. Alternatively, the audio data may be output to memory 520 for further processing. Audio circuit 560 may also include an earphone jack to allow communication between external headphones and electronic device 500.
[0093] Electronic device 500, through a transmission module 570 (e.g., a Wi-Fi module), can help users receive requests, send information, and so on, providing users with wireless broadband Internet access. Although the figure shows transmission module 570, it is understood that it is not a required component of electronic device 500 and can be omitted as needed without changing the essence of the invention.
[0094] Processor 580 is the control center of electronic device 500. It connects all components of the phone using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 520 and accessing data stored in memory 520, it executes various functions of electronic device 500 and processes data, thereby providing overall monitoring of the electronic device. Optionally, processor 580 may include one or more processing cores. In some embodiments, processor 580 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 580.
[0095] Electronic device 500 also includes a power supply 590 (e.g., a battery) for powering various components. In some embodiments, the power supply can be logically connected to processor 580 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. Power supply 590 can also include any components, such as one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, and a power status indicator.
[0096] Although not shown, the electronic device 500 also includes a camera (such as a front camera and a rear camera), a Bluetooth module, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations: Obtain a pedestrian historical trajectory data set; the pedestrian historical trajectory data set includes a plurality of pedestrian historical trajectory data and pedestrian future trajectory data; The pedestrian historical trajectory data is labeled by a clustering algorithm to obtain a motion pattern pseudo label; Convert the pedestrian historical trajectory data from the time domain to the frequency domain to extract low-frequency motion trend features; Inputting the low-frequency motion trend features into a pedestrian motion pattern classifier to predict the probability of the motion pattern corresponding to the pedestrian historical trajectory data; calculating a loss function based on the probability of the motion pattern and the motion pattern pseudo-label, and iteratively training the pedestrian motion pattern classifier; Constructing a conditional guided diffusion model based on the motion pattern, performing reverse denoising based on the conditional guided diffusion model, and generating a multimodal future trajectory; calculating a joint loss function based on the multimodal future trajectory and the pedestrian's future trajectory data, and training the conditional guided diffusion model; The trajectory of the pedestrian to be detected is predicted based on the trained pedestrian motion pattern classifier and the conditional guided diffusion model.
[0097] During specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments and will not be repeated here.
[0098] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be accomplished through instructions, or by controlling related hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any embodiment of the motion pattern-guided pedestrian trajectory prediction method based on conditional diffusion provided by the embodiment of the present invention.
[0099] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0100] Since the instructions stored in the storage medium can execute the steps in any embodiment of the method for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion provided in the embodiments of the present invention, the beneficial effects that can be achieved by any method for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion provided in the embodiments of the present invention can be achieved. For details, please refer to the previous embodiments and will not be repeated here.
[0101] The above is a detailed introduction to the motion pattern-guided pedestrian trajectory prediction method, device, storage medium and electronic device provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A motion pattern guided pedestrian trajectory prediction method based on conditional diffusion, characterized in that: The method comprises: Obtain a pedestrian historical trajectory data set; the pedestrian historical trajectory data set includes a plurality of pedestrian historical trajectory data and pedestrian future trajectory data; The pedestrian historical trajectory data is labeled by a clustering algorithm to obtain a motion pattern pseudo label; Convert the pedestrian historical trajectory data from the time domain to the frequency domain to extract low-frequency motion trend features; Inputting the low-frequency motion trend features into a pedestrian motion pattern classifier to predict the probability of the motion pattern corresponding to the pedestrian historical trajectory data; calculating a loss function based on the probability of the motion pattern and the motion pattern pseudo-label, and iteratively training the pedestrian motion pattern classifier; Constructing a conditional guided diffusion model based on the motion pattern, performing reverse denoising based on the conditional guided diffusion model, and generating a multimodal future trajectory; calculating a joint loss function based on the multimodal future trajectory and the pedestrian's future trajectory data, and training the conditional guided diffusion model; The trajectory of the pedestrian to be detected is predicted based on the trained pedestrian motion pattern classifier and the conditional guided diffusion model.
2. The method for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion according to claim 1, characterized in that: The method of labeling the pedestrian historical trajectory data by a clustering algorithm to obtain a motion pattern pseudo label includes: Extracting the speed, acceleration, direction angle and angular velocity of the pedestrian historical trajectory data to form a feature vector; performing normalization processing on the feature vector; The K-Means algorithm is used to cluster the standardized feature vectors to obtain motion mode pseudo labels.
3. The method for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion according to claim 2, characterized in that: The K-Means algorithm is used to cluster the normalized feature vectors to obtain motion pattern pseudo labels, including: Initialize the cluster center and randomly select K trajectories from the normalized feature vector as the initial cluster center, where K is the number of modes you want to cluster; Assign trajectories to the nearest cluster. For each trajectory, calculate its distance to all cluster centers and assign it to the nearest cluster. Update the cluster center and calculate the trajectory centroid of each cluster; Convergence is judged. If the change of cluster center is less than the threshold, the clustering is stopped; or if the maximum number of iterations is reached, the clustering is stopped; Returns the motion mode category label and cluster center for each trajectory.
4. The method for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion according to claim 1, characterized in that: The step of converting the pedestrian historical trajectory data from the time domain to the frequency domain and extracting low-frequency motion trend features includes: Past observed trajectory coordinates corresponding to the pedestrian historical trajectory data Perform one-dimensional discrete cosine transform respectively and calculate the frequency domain coefficients: in, represents discrete cosine transform, represents the future time step, represents the past time step, represents the Kronecker function, which is used to determine whether two variables are equal. Indicates the number of DCT coefficients; Before Retention Low-frequency coefficients , filter out high-frequency noise; Reconstruct the time domain trajectory from the low-frequency coefficients via the inverse discrete cosine transform: in, represents the coefficients after discrete cosine transform, The coefficient is the low-frequency motion trend characteristic.
5. The method for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion according to claim 1, characterized in that: Inputting the low-frequency motion trend features into the pedestrian motion pattern classifier to predict the probability of the motion pattern corresponding to the pedestrian historical trajectory data includes: The noisy pedestrian historical trajectory data is input into the multi-layer perceptron and attention mechanism for feature extraction: in, represents the trajectory data of the function noise, is a multi-layer perceptron, is the attention mechanism, is the activation function, is the probability of the predicted motion pattern, are learnable parameters.
6. The method for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion according to claim 1, characterized in that: The step of constructing a conditional guided diffusion model based on the motion pattern, performing reverse denoising based on the conditional guided diffusion model, and generating a multimodal future trajectory includes: calculating a probability gradient based on the probability of the motion pattern; A conditional guided diffusion model is generated based on the probability gradient, and reverse denoising is performed based on the conditional guided diffusion model; the conditional guided diffusion model is: in, represents the t-th step denoising result, represents the denoising result of step t-1, , and represents the hyperparameters in the diffusion model, represents the noise prediction network, represents the conditional guide step size, represents the standard deviation, represents the classifier gradient, Represents a classifier network.
7. The method for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion according to claim 5, characterized in that: The joint loss function is: in, is the joint loss function, is the cross entropy loss function, is the end-to-end diffusion loss function, is the real motion modality label, To predict the motion modality label, is the number of trajectories, is the number of multimodal trajectory predictions, For the real future trajectory, To predict future trajectories.
8. The method for predicting pedestrian trajectories guided by motion patterns based on conditional diffusion according to claim 1, characterized in that: The method further comprises: The original noisy trajectory data Input to the trajectory predictor to predict the Step denoising results , expressed by the following formula: in, For under conditions The probability distribution of is the mean, is the standard deviation, is the identity matrix, is the trajectory predictor, is the noise reduction step size.
9. A motion pattern-guided pedestrian trajectory prediction device based on conditional diffusion, characterized in that: include: Data acquisition module, used to obtain pedestrian historical trajectory data; Movement pattern annotation module, used to obtain pedestrian historical trajectory dataset; The pedestrian historical trajectory data set includes a plurality of pedestrian historical trajectory data and pedestrian future trajectory data; A frequency domain conversion module is used to convert the pedestrian historical trajectory data from the time domain to the frequency domain to extract low-frequency motion trend features; a motion pattern prediction module, configured to input the low-frequency motion trend features into a pedestrian motion pattern classifier to predict the probability of the motion pattern corresponding to the pedestrian historical trajectory data; Calculating a loss function based on the probability of the motion pattern and the motion pattern pseudo-label, and iteratively training the pedestrian motion pattern classifier; A conditional diffusion generation module is configured to construct a conditional guided diffusion model based on the motion pattern, perform reverse denoising based on the conditional guided diffusion model, and generate a multimodal future trajectory; calculate a joint loss function based on the multimodal future trajectory and the pedestrian's future trajectory data, and train the conditional guided diffusion model. The future trajectory prediction module predicts the trajectory of pedestrians to be detected based on the trained pedestrian motion pattern classifier and conditional guided diffusion model.
10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, which are suitable for being loaded by a processor to execute the motion pattern-guided pedestrian trajectory prediction method based on conditional diffusion according to any one of claims 1 to 8.