A pedestrian intention recognition method and system based on hierarchical spatiotemporal causal graphs and state diffusion

Abstract

The application discloses a pedestrian intention recognition method and system based on hierarchical space-time causal graph and state diffusion, and belongs to the cross technical field of artificial intelligence and automatic driving. In view of the defects that the prior art is shallow in scene understanding, single in prediction and lacks microscopic clues, the application first pre-processes a video stream to extract pedestrian posture, dynamic target and environmental element features; then constructs an HSTCG containing three-layer nodes of 'posture-entity-environment' and a learnable causal edge, generates a context vector through a graph evolution encoder; injects a PFSDM to generate a multi-modal future state through reverse denoising, and combines a joint loss to optimize end to end. Through the integrated graph causal reasoning and probability generation framework, the application realizes multi-modal and probabilistic prediction of the future behavior of pedestrians, and significantly improves the accuracy, robustness and safety of the intention recognition system.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of artificial intelligence and autonomous driving, specifically relating to a pedestrian intent recognition method and system based on hierarchical spatiotemporal causal graphs and state diffusion. Background Technology

[0002] With the rapid development of Advanced Driver Assistance Systems (ADAS) and autonomous driving technologies, accurate perception of the road environment, scene understanding, and behavior prediction have become crucial. Among these, accurately and promptly predicting the behavioral intentions of vulnerable road users (VRUs) such as pedestrians is one of the core challenges in ensuring driving safety and improving traffic efficiency. A system capable of predicting whether a pedestrian is about to "cross the road," "wait," or "turn back" can buy valuable reaction time for the vehicle's decision-making and planning modules, thereby effectively avoiding potential collision risks.

[0003] Existing pedestrian intent recognition technologies are mainly divided into two categories: 1) Traditional machine learning methods: These methods rely heavily on manually designed features (such as pedestrian speed, orientation, and distance from crosswalks) combined with classifiers like Support Vector Machines (SVM) and Random Forests to determine intent. These methods have weak generalization ability and struggle to adapt to complex urban scenarios, such as mixed pedestrian and vehicle traffic, occlusion, and traffic light switching. They also exhibit low accuracy in dynamic interactive scenarios. 2) Deep learning methods: These methods employ a "detection-tracking-prediction" paradigm. They locate pedestrians using convolutional neural networks and model trajectory features using temporal models such as Long Short-Term Memory (LSTM), Gated Recurrent Units (GRU), or Transformers. The final output is a single intent classification or trajectory prediction result. While this approach offers improvements over traditional methods, it still suffers from three major drawbacks:

[0004] 1. Superficial understanding of scene context: Pedestrians are treated as isolated individuals, or global visual features are simply pieced together with pedestrian features, ignoring the structured interaction relationships between "pedestrian-traffic light", "pedestrian-vehicle", and "pedestrian-crossing". For example, existing methods cannot distinguish the difference in intent between "pedestrian facing the crosswalk and the traffic light is green" and "pedestrian facing the crosswalk but the traffic light is red", leading to a sharp drop in accuracy at complex intersections.

[0005] 2. Contradictions between prediction models and reality: By outputting only a single deterministic result, these methods fail to capture the randomness and multimodal characteristics of pedestrian behavior. At decision-making critical points (such as when a pedestrian hesitates about whether to cross the road), these methods may miss high-risk events of "low-probability crossings," posing serious safety hazards.

[0006] 3. Lack of micro-behavioral cues: Simplifying pedestrians into a bounding box or center point fails to utilize postural details such as head rotation, limb swing, and body tilt, thus missing early signals of changing intentions. Summary of the Invention

[0007] The purpose of this invention is to provide a pedestrian intent recognition method and system based on hierarchical spatiotemporal causal graphs and state diffusion, thereby solving the aforementioned technical problems.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0009] A pedestrian intent recognition method based on hierarchical spatiotemporal causal graphs and state diffusion includes the following steps:

[0010] Step S1: Acquisition and preprocessing of temporal observation data: Acquire continuous temporal observation data containing the target pedestrian, perform target detection, pose estimation, scene segmentation and feature normalization to obtain pedestrian pose features (including human key point coordinates and confidence), dynamic target features (including pedestrian / vehicle coordinates, speed and acceleration) and scene environment features (including traffic lights, pedestrian crossing position and status) to form structured initial features.

[0011] Step S2, Hierarchical Spatiotemporal Causal Graph Construction: Based on the initial features, a hierarchical spatiotemporal causal graph (HSTCG) sequence that evolves over time is constructed. The hierarchical spatiotemporal causal graph uses multi-layer nodes and learnable causal edges to structurally represent the dynamic interaction relationships within the scene.

[0012] Step S3, Graph Evolution Encoding: Input the hierarchical spatiotemporal causal graph sequence into the graph evolution encoder, and output the context condition vector containing spatiotemporal dynamics and causal relationships.

[0013] Step S4, Probabilistic Future State Diffusion Model Prediction: Input the context condition vector into the Probabilistic Future State Diffusion Model (PFSDM), and generate one or more latent future state variables containing future trajectory sequences and intention classifications from the prior noise distribution through a condition-guided inverse denoising process.

[0014] Step S5, Joint Loss Calculation and Parameter Optimization: Based on the generated future state and the real future state labels, construct a joint total loss function that includes diffusion loss, intent classification loss and trajectory regression loss; adjust the parameters of the graph evolution encoder and the probabilistic future state diffusion model end-to-end through the backpropagation algorithm until the model converges.

[0015] Step S6, Multimodal Inference Output: Perform 50-100 parallel reverse denoising processes, perform cluster analysis on the generated future state latent variables, and output multimodal prediction results containing intention categories, average trajectories, probability distributions, and uncertainty assessments.

[0016] Further optimization involves acquiring a continuous video stream from a sensor (such as a high-resolution camera) in step S1, performing the following processing on each frame of data, and outputting structured initial features:

[0017] Multi-target detection and tracking: The DETR or YOLOv8 detector is used to obtain 2D bounding boxes and class labels of targets such as pedestrians and vehicles. The ByteTrack algorithm is used to assign unique IDs to achieve cross-frame tracking and output the historical trajectory of the target.

[0018] Pedestrian pose estimation: Within the pedestrian bounding box, the coordinates and confidence scores of 17 human key points (head, shoulder, elbow, hip, knee, ankle) are extracted using the HRNet or ViTPose model. Normalization is performed based on "the center of the bounding box as the origin and the diagonal length as the scale" to eliminate translation and scaling differences.

[0019] Scene segmentation: The SegFormer semantic segmentation model is used to identify and locate environmental elements such as pedestrian crossings, traffic lights, curbs, and stop lines, and outputs position coordinates (such as the center of a traffic light), status codes (such as red light = 0, green light = 1) and category codes.

[0020] Feature vectorization: Dynamic target (pedestrian / vehicle) features (coordinates, velocity, acceleration, bounding box size change rate), pose features (normalized key points), and environmental features (relative position, state encoding) are mapped to a 256-dimensional unified feature space to form an initial feature matrix.

[0021] Further optimization involves, in step S2, constructing a hierarchical spatiotemporal causal graph HSTCG sequence {G1,G2,...,G} that evolves over time. Tobs}, each G t =(V t A t ,F t Specifically, this includes:

[0022] Step S2.1: Define a three-level node set V t The bottom layer consists of posture nodes representing 17 key body points of a pedestrian; the middle layer consists of entity nodes representing dynamic traffic participants, with one node corresponding to each dynamic target (target pedestrian, surrounding vehicles, other pedestrians) in the scene; the top layer consists of environmental nodes representing key elements such as traffic lights and crosswalks, with one node corresponding to each key environmental element, and the feature is an environmental element vector.

[0023] Step S2.2: Define the set A of three types of edges. t Intra-layer edges connect pose nodes to represent the skeletal structure, simulating the human skeletal structure, with a fixed weight of 1. Cross-layer edges connect pose nodes to entity nodes to aggregate pose information, with a fixed weight of 1. Inter-layer edges connect entity nodes to environment nodes or different entity nodes to represent interaction relationships; the weights of inter-layer edges are learnable causal weights. w ij , w ij∈[0,1], quantifying the causal influence between nodes, such as the pedestrian-traffic light relationship when the traffic light is green. w ij Increase.

[0024] Step S2.3: Define the node feature matrix F t The original features of each node are mapped to a unified feature space through a multilayer perceptron, thus completing the node feature initialization.

[0025] Step S2.4: Using a learnable causal reasoning module, based on the dynamic interaction information between nodes, the causal weights of the inter-layer edges are dynamically calculated to quantify the potential causal influence between nodes.

[0026] Further optimization involves step S2.4, where the causal weight calculation is implemented through a causal relationship inference module. This inference module is a two-layer MLP: the input is node pair features (… h i , h j The impact on the score is calculated using the following formula. e ij = a T ⋅ LeakyReLU W [ h i || h j ]),

[0027] Then through w ij =sigmoid(LayerNorm( e ij The causal weights are normalized to the interval [0,1], where a For attention parameter vector, W It is a linear projection matrix.

[0028] Further optimization involves step S3, where the graph evolution encoder is a key connecting module between HSTCG and PFSDM. Through a two-stage architecture of causal modulation GAT and Transformer, it extracts contextual condition vectors that contain causal relationships and spatiotemporal dynamics. c The processing steps of a graph evolution encoder include:

[0029] Step S3.1: At each time step, a causal modulated graph attention network (GAT) is used, based on the causal weights of the inter-layer edges. w ij Modulate the attention coefficient, aggregate neighborhood information for the current hierarchical spatiotemporal causal graph, and update the node representation.

[0030] The calculation of attention coefficients in the Causal Modulation Graph Attention Network (GAT) includes:

[0031] Raw attention score: ;

[0032] Modulation coefficients: , N i For nodes i The set of neighboring nodes;

[0033] Node update: ;

[0034] Add a dropout layer with a probability of 0.1-0.2 after each GAT layer.

[0035] Step S3.2: Extract the spatial aggregation features of the target pedestrian entity nodes at each time step to form a temporal sequence, input it into the Transformer encoder to capture long-term dependencies, and obtain the context condition vector through the label vector or average pooling.

[0036] The Transformer encoder includes:

[0037] Multi-head self-attention sublayer: 4-8 heads, each head has a dimension of D model / Number of heads, using scaled dot product attention and future time step mask;

[0038] Feedforward Neural Network (FFN) Sublayer: Hidden layer dimension is 4× D model The activation function is GELU;

[0039] Residual connections and layer normalization: Output = LayerNorm(Input + Sublayer(Input)), the input sequence needs to be added with sinusoidal position encoding; Input is the original input feature sequence / feature vector of the multi-head self-attention sub-layer of the Transformer encoder and the feedforward neural network FFN sub-layer.

[0040] Further optimization, step S4 specifically includes:

[0041] Step S4.1: Define the latent variables of the future state z 0, from the future T pred The pedestrian two-dimensional relative coordinate sequence at each time step and N class The vector is formed by concatenating one-hot vectors, z0∈R (T pred ×2)+N class );

[0042] T pred The time step for future prediction is set to 20-50 frames (corresponding to 2-5 seconds @ 10fps). Each frame contains two dimensions: the pedestrian's horizontal position coordinate X and vertical position coordinate Y relative to the reference point.

[0043] N class The number of pedestrian intent categories is 3-5 (including at least crossing the road, waiting on the side of the road, and walking straight along the road), encoded in one-hot vector form.

[0044] Step S4.2: Design the forward diffusion process, in T Dispatching via monotonically increasing noise within each time step β t Gaussian noise is gradually injected into z0 to generate arbitrary time intervals. t Noisy latent variables z t ,

[0045] ;

[0046] in, Let I be the standard Gaussian noise vector, and let α be the identity matrix. t =1-β t , α t To retain a proportion of true information for each step. The percentage of true information retained from the previous t steps;

[0047] Set the total number of diffusion steps T = 800-1200, and define the noise variance sequence. β t , t∈[1,T], the β t It increases linearly from β1 = 0.0001 to 0.001 to β T =0.01-0.02, ensuring that early slow noise addition preserves details, and later rapid noise addition approaches noise.

[0048] Step S4.3: Construct a U-Net-based denoising network Input noisy latent variables z t Time step coding emb ( t and context condition vector c Predict the noise added during the forward process. The time-step encoding is implemented using sinusoidal positional encoding, and the context condition vector is injected into each residual block of the U-Net through a cross-attention mechanism. The denoising network architecture design includes:

[0049] Step S4.3.1: The U-Net basic skeleton of downsampling-intermediate feature processing-upsampling is adopted. The downsampling stage and the upsampling stage each have 4-6 layers. Downsampling is achieved by extracting global spatiotemporal features through convolutional layers (3×3 kernel size, stride 2). Upsampling is achieved by restoring local trajectory details through transposed convolutional layers (3×3 kernel size, stride 2).

[0050] Step S4.3.2: In the intermediate feature processing stage of U-Net, 1-3 layers of self-attention modules are introduced. The number of attention heads in the self-attention modules is 4-8, which are used to capture the long-term dependencies of pedestrian future trajectory sequences.

[0051] Step S4.3.3: Encoding the time step emb ( t Generated through sinusoidal positional encoding, with dimensions 256-512, and related to the context condition vector. c (The dimensions are consistent with emb(t)) After concatenation, each residual block of U-Net is injected through the cross-attention module. The number of attention heads in the cross-attention module is the same as that in the self-attention module, which is used to guide the network to focus on key causal interaction information in the scene.

[0052] Further optimization includes the following iterative calculations in the reverse denoising process:

[0053] Initial noise vector is sampled from a standard normal distribution. z T ~N(0,I), iterate in reverse from t=T to t=1, and perform the denoising step:

[0054] Each iteration calculates the noisy latent variable of the previous time step using the following formula. z t-1 :

[0055] ;

[0056] in, w t For random noise, when t>1, w t ~N(0,I); at t=1, w t =0;

[0057] , σ t This is a parameter for controlling the randomness of sampling;

[0058] The reverse denoising process is executed in parallel N times, and cluster analysis is performed on the generated N future state latent variables to obtain multimodal prediction results and corresponding probabilities.

[0059] Further optimization, step S5 specifically includes:

[0060] Step S5.1: Construct a structure including diffusion loss L diff Intent Classification Loss L intent With trajectory regression loss L traj Joint total loss function L total ;

[0061] L total =λ diff L diff +λ intent L intent +λ traj L traj ;

[0062] Where, λ diff , λ intent , λ traj These are the loss weighting coefficients;

[0063] Diffusion loss L diff To calculate the mean square error between the predicted noise and the actual noise, i.e. E represents the... z 0, c, Calculation of the expected value of t, ||·|| 2 It is an L2 norm;

[0064] Intended classification loss L intent = CrossEntropy ( softmax ( Int pred ), Int true ),in Int pred To predict z The intent logic value separated in 0, Int true Labeling as true intent softmax The function is used to transform logical values ​​into a probability distribution. CrossEntropy This is the function for calculating cross-entropy;

[0065] Trajectory Regression Loss L traj = SmoothL1( Tr pred , Tr true ),in Tr pred To predict z The future trajectory sequence separated from 0, Tr true The predicted trajectory is the true trajectory sequence, and SmoothL1 is the smoothing L1 loss calculation function, which is used to measure the point-by-point coordinate error between the predicted trajectory and the true trajectory.

[0066] Step S5.2: Based on the joint total loss function, update the graph evolution encoder and PFSDM parameters through backpropagation using the AdamW optimizer for end-to-end training; the training batch size is set to 16-64, and the number of training rounds is set to 300-600 rounds.

[0067] Mixed precision training is used to accelerate convergence. The early stopping strategy is set as follows: if the pedestrian trajectory endpoint error (FDE) on the validation set does not decrease for 15-25 consecutive rounds, then training is stopped and the current optimal model weights are saved.

[0068] Further optimization is achieved in step S6, where the cluster analysis employs the DBSCAN algorithm: eps=0.4-0.6, min_samples=3-5, based on trajectory similarity and intention. Figure 1 Consistent clustering is used, and the probability of each cluster is: number of samples in the cluster / total number of samples. The top-3 clusters with the highest probabilities are output as the multimodal results.

[0069] A pedestrian intent recognition system based on hierarchical spatiotemporal causal graphs and state diffusion, used to implement the above method, includes:

[0070] Data acquisition and preprocessing module: includes a high-resolution camera and feature processing unit, performs target detection, pose estimation, and scene segmentation, and outputs structured initial features;

[0071] Graph Construction and Encoding Module: Includes HSTCG construction unit and graph evolution encoding unit. The construction unit dynamically generates a three-layer causal graph sequence, and the encoding unit outputs a context condition vector through causal modulation GAT and Transformer. c ;

[0072] Probability prediction module: Loaded with the PFSDM, including forward diffusion unit, reverse denoising unit and clustering unit, performs multiple rounds of sampling and clustering, and outputs multimodal future state prediction results;

[0073] Results output module: Includes formatting unit and communication interface, which converts intent category, trajectory sequence, probability distribution and uncertainty evaluation results into standardized format and transmits them to downstream decision planning module via CAN / Ethernet;

[0074] Computing platform: It adopts a dedicated computing platform for autonomous driving, which supports mixed-precision training and real-time inference.

[0075] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described pedestrian intent recognition method based on hierarchical spatiotemporal causal graphs and state diffusion.

[0076] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0077] 1. Deep Structured Scene Understanding Ability: This invention designs a hierarchical spatiotemporal causal graph (HSTCG), which no longer treats the scene as a flat set of features, but explicitly models the internal and external relationships of the three levels of "attitude-entity-environment", thus understanding complex traffic scenes from the perspectives of structure and causality.

[0078] 2. Probabilistic and Multimodal Prediction of the Future: This invention fundamentally overcomes the limitations of traditional deterministic prediction by designing the Probabilistic Future State Diffusion Model (PFSDM). It refactors the intention prediction problem into a generative task, capable of generating a distribution of future possibilities from noise. During inference, multiple samplings can yield several different but reasonable combinations of future trajectories and intentions, and their respective probabilities can be calculated.

[0079] 3. Integrated and End-to-End Learning Framework: The HSTCG and PFSDM proposed in this invention are not simply concatenated, but jointly optimized within a unified framework. The graph encoder learns how to extract the most effective conditional information for the diffusion model, while the goal of the diffusion model, in turn, guides the graph encoder to focus on the spatiotemporal causal patterns most relevant to future states. This end-to-end training approach ensures deep collaboration among modules, avoiding the problems of information loss and suboptimal solutions found in multi-stage methods.

[0080] 4. Enhanced Interpretability and Robustness: By analyzing the dynamically changing causal edge weights in HSTCG, we can intuitively understand which key interactions the model "focuses on" when making predictions (e.g., the causal weight between pedestrian head turning and vehicle nodes increases). Simultaneously, since the model learns a complete future state distribution, the entropy of its predicted distribution increases accordingly for ambiguous or adversarial inputs, thus providing the system with an intrinsic measure of uncertainty and enhancing the model's robustness and reliability. Attached Figure Description

[0081] Figure 1The flowchart shows a pedestrian intent recognition method based on hierarchical spatiotemporal causal graphs and state diffusion.

[0082] Figure 2 The model training convergence curve;

[0083] Figure 3 A graph showing the error growth trend under different prediction durations;

[0084] Figure 4 The results of the module validity verification;

[0085] Figure 5 The feature space distribution of the context vectors (visualized using T-SNE);

[0086] Figure 6 For traffic scenarios, (a) shows the intermediate perception results interface during system operation, and (b) shows that in a real traffic scenario, the system outputs three possible future behavior modalities for the target pedestrian.

[0087] Figure 7 A real-time spatiotemporal causal attention heatmap;

[0088] Figure 8 This demonstrates the inverse denoising process of the Probabilistic Future State Diffusion Model (PFSDM). Detailed Implementation

[0089] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0090] Example 1: This example is designed for autonomous driving scenarios on urban roads. It targets areas with dense pedestrian traffic and complex behavior, such as intersections, crosswalks, and areas around residential communities. It enables multimodal probability prediction of the behavioral intentions of target pedestrians in the next 2-3 seconds (corresponding to 20-30 frames), providing a safe and reliable predictive basis for the decision-making and planning module of autonomous vehicles and avoiding collision risks.

[0091] This embodiment details the complete training process of the pedestrian intent recognition model described in this invention, such as... Figure 1 As shown, it specifically includes:

[0092] Step S1: Acquisition and Characterization Preprocessing of Time-Series Observation Data

[0093] 1) Data acquisition: The camera acquires a continuous video stream in real time, and extracts the first 16 frames (Tobs=16 frames, corresponding to 1.6 seconds of historical data) as time series observation data.

[0094] 2) Multi-target detection and tracking: The YOLOv8 detector is used to detect targets in each frame of the image, and outputs 2D bounding boxes and category labels of targets such as pedestrians and vehicles. The ByteTrack algorithm is used to assign unique IDs to achieve cross-frame tracking and obtain the historical trajectory of the target pedestrian. The target pixel coordinate positioning accuracy is ≤1 pixel.

[0095] 3) Pedestrian pose estimation: Run the HRNet model within the target pedestrian bounding box to extract the coordinates and confidence scores of 17 human key points (head, shoulder, elbow, hip, knee, ankle, etc.). Normalize the data with the center of the bounding box as the origin and the diagonal length as the scale to eliminate translation and scaling differences.

[0096] 4) Scene segmentation: The SegFormer semantic segmentation model is used to identify environmental elements such as pedestrian crossings, traffic lights, and curbs, and outputs the center coordinates of traffic lights, status codes (red light = 0, green light = 1, yellow light = 2), and the vertex coordinates of the pedestrian crossing polygon.

[0097] 5) Feature vectorization: Dynamic target features (coordinates, velocity, acceleration), attitude features (normalized key points), and environmental features (relative position, state encoding) are mapped to a 256-dimensional unified feature space through MLP to form an initial feature matrix.

[0098] Step S2: Dynamic construction and encoding of hierarchical spatiotemporal causal graph HSTCG, which transforms unstructured scene information into a dynamic graph structure that can be efficiently processed by graph neural networks.

[0099] 1) Graph sequence definition: Construct the HSTCG sequence {G1, G2, ..., G} corresponding to 16 frames. 16}, each graph G t =(V t A t ,F t );

[0100] 2) Three-layer node set V t Bottom layer: 17 pose nodes, corresponding to 17 key points of the human body, with normalized coordinates and confidence scores as features; Middle layer: Entity nodes, including the target pedestrian, 3 surrounding vehicles, and 2 other pedestrians, totaling 6 nodes, with dynamic target vectors as features; Top layer: Environment nodes, including 1 traffic light, 1 pedestrian crossing, and 1 curb, totaling 3 nodes, with environmental element vectors as features.

[0101] 3) Set A of three types of edges tIntra-layer edges: connect 17 pose nodes (simulating skeletal structure), with a fixed weight of 1; Cross-layer edges: each pose node connects to the target pedestrian entity node, with a fixed weight of 1; Inter-layer edges: connect entity nodes to environment nodes (e.g., target pedestrian - traffic light) and different entity nodes (e.g., target pedestrian - surrounding vehicles), totaling 12 inter-layer edges, with learnable causal weights. w ij ∈[0,1].

[0102] 4) Node feature initialization: The original features of each node are mapped to a 512-dimensional feature space through a 2-layer MLP to obtain the node feature matrix F. t .

[0103] 5) Causal weight calculation: A two-layer MLP is used as the causal relationship inference module, and the input node pairs features ( h i , h j ), through formula e ij = a T ⋅ LeakyReLU W [ h i || h j The influence score is calculated, normalized using LayerNorm, and then mapped to the [0,1] interval using the sigmoid function to obtain the dynamic causal weights. For example, when the traffic light is green, the target pedestrian-traffic light weights are calculated. w ij =0.85.

[0104] Step S3: Graph Evolution Coding

[0105] 1) Causal Modulation Graph Attention Network (GAT) Spatial Aggregation: Each time step employs a 2-layer Causal Modulation Graph Attention Network (GAT), based on the causal weights w of the inter-layer edges. ij The modulation attention coefficient is calculated by using node feature similarity to determine the original attention score and the modulation coefficient α. ij =w ij • softmax (raw score), w ij ∈[0,1], with an initial value of 0.5, dynamically updated during model training; the node update formula is: A dropout layer with a probability of 0.15 is added after each GAT layer to enhance generalization ability.

[0106] 2) Transformer Temporal Aggregation: Extract updated features of target pedestrian entity nodes from 16 frames to form a temporal sequence with a length of 16 and a dimension of 512; input to the Transformer encoder with 6 heads, Dmodel=512, FFN hidden layer dimension of 2048, GELU activation function, sinusoidal position encoding, and future time step mask to avoid information leakage; extract global features through the [CLS] marker vector to obtain a 512-dimensional context condition vector c.

[0107] Step S4: Probabilistic Future State Diffusion Model (PFSDM) Prediction

[0108] 1) Definition of latent variables for future states: z 0 from the future 30 frames (T pred The pedestrian's two-dimensional relative coordinate sequence (30×2=60 dimensions) is formed by concatenating it with three types of intent one-hot vectors (crossing = [1,0,0], waiting = [0,1,0], and going straight = [0,0,1]). z 0∈R 63 .

[0109] 2) Forward diffusion process: Set the total number of diffusion steps T=1000, and the noise variance sequence β t It grows linearly from β1=0.0005 to β t =0.015; Calculate the derived parameter α t =1-β t , Through formula Towards z 0. Gradually inject noise.

[0110] 3) Denoising Network Construction: The U-Net architecture is adopted as the denoising network. The downsampling stage consists of 4 layers (3×3 convolutional kernels, stride 2), and the upsampling stage consists of 4 layers (3×3 transposed convolutional kernels, stride 2). In the intermediate feature processing stage, a 2-layer self-attention module (6 heads) is introduced to capture long-term trajectory dependencies. The time-step encoding emb(t) generates a 512-dimensional vector through sinusoidal position encoding, which is concatenated with the context condition vector c and then injected into each residual block of U-Net through the cross-attention module.

[0111] 4) Inverse denoising iteration: Sample the initial noise vector from the standard normal distribution. z 1000 ~N(0,I), iterate backwards from t=1000 to t=1; the formula for each iteration is:

[0112] ;

[0113] in When t>1, w t ~N(0,I), w at t=1 t =0.

[0114] Step S5: Design and end-to-end optimization of the joint loss function

[0115] 1) To effectively train the entire model, a joint loss function is designed:

[0116] Diffusion loss (MSE loss), weight λ diff =1.0;

[0117] Intended classification loss L intent = CrossEntropy ( softmax ( Int pred ), Int true ), weight λ intent =0.8;

[0118] Trajectory Regression Loss L traj = SmoothL 1( Tr pred , Tr true ), weight λ traj =0.5;

[0119] Total loss L total =1.0・L diff +0.8·L intent +0.5·L traj ;

[0120] 2) Model Training: The batch size was set to 32, and the number of training epochs was 400. The AdamW optimizer was used (learning rate 1e-4, weight decay 1e-5). Mixed precision training (FP16) was enabled to accelerate convergence. The early stopping strategy was set as follows: if the pedestrian trajectory endpoint error (FDE) in the validation set does not decrease for 20 consecutive epochs, training is stopped and the optimal model weights are saved. The training convergence curve is shown below. Figure 2 As shown, the total loss rapidly decreased from the initial 5.2 to 0.55 and stabilized, with the diffusion loss dominating the optimization process, proving the effectiveness of the end-to-end framework.

[0121] Step S6: Multimodal Inference Output

[0122] 1) Parallel sampling: Execute 80 reverse denoising processes in parallel to generate 80 samples of future state latent variables;

[0123] 2) Cluster analysis: The DBSCAN algorithm was used for clustering, with parameters eps=0.5 and min_samples=4. Clustering was performed based on trajectory similarity and meaning. Figure 1 Consistency was used to cluster 80 samples into 3 clusters;

[0124] 3) Result formatting: Cluster 1, sample size 52: Intent = crossing the road, probability = 52 / 80 = 65%, average trajectory is crossing the pedestrian crossing from the current position, endpoint error ≤ 0.8 meters; Cluster 2, sample size 24: Intent = waiting at the roadside, probability = 24 / 80 = 30%, average trajectory is staying at the current position, lateral displacement ≤ 0.2 meters; Cluster 3, sample size 4: Intent = going straight along the road, probability = 4 / 80 = 5%, average trajectory is moving forward along the roadside, longitudinal displacement 3.5 meters.

[0125] 4) Output Results: The multimodal prediction results (intent category, average trajectory, probability distribution, uncertainty assessment) are transmitted to the decision planning module via the CAN bus. The visualization effect is as follows: Figure 6 The right figure shows the prediction results for different intentions, displayed as a superimposed layer of colored trajectory pipes. Among them, Figure 6 (a) shows the intermediate perception results interface during system operation. The system performs object detection (green / yellow bounding boxes), human pose estimation (blue skeleton connecting lines), scene semantic segmentation (purple road surface mask), and traffic light status recognition (red boxes) on the input video stream in real time. Figure 6 (b) shows that in a real traffic scenario, the system outputs three possible future behavior modalities for the target pedestrian, which are superimposed as colored trajectory pipes with probability labels: the red pipe represents the intention to "cross" (probability 65%), the blue pipe represents the intention to "wait" (probability 30%), and the green pipe represents the intention to "return" (probability 5%).

[0126] Experimental verification results:

[0127] 1) Robustness verification: such as Figure 3 As shown, under different prediction durations, the trajectory endpoint error (FDE) of the method in this invention (HSTCG-PFSDM) increases most gradually: FDE = 0.21 meters at 0.5 seconds, FDE = 0.38 meters at 1.0 seconds, FDE = 0.65 meters at 2.0 seconds, and FDE = 0.9 meters at 3.0 seconds, significantly outperforming LSTM (FDE = 1.8 meters at 3.0 seconds) and ST-GCN (FDE = 1.3 meters at 3.0 seconds), demonstrating its long-term prediction capability and robustness. The diffusion model, through its progressive denoising generation method, is more resistant to the accumulation of uncertainty in long-term predictions than the direct coordinate regression method. This is crucial for the safety of autonomous driving, because the earlier the accurate prediction (e.g., 3 seconds in advance), the more time the vehicle has to react.

[0128] For the traditional LSTM method, the error increases exponentially as the prediction time increases (from 0.5 seconds to 3.0 seconds), indicating that the traditional method is difficult to predict the long-term future. The existing ST-GCN technology performs slightly better, but the error still increases significantly after 2.0 seconds.

[0129] 2) Module validity verification:

[0130] like Figure 4 As shown, the ablation experiment results indicate that:

[0131] Basic diffusion model (without HSTCG): FDE = 0.88 m;

[0132] Remove causal reasoning module (w / o causal reasoning): FDE = 0.72 m;

[0133] Remove hierarchical structure (w / o hierarchy): FDE = 0.61 meters;

[0134] Complete model (HSTCG+PFSDM): FDE = 0.48 meters.

[0135] Basic diffusion model (without HSTCG): Average displacement error ADE = 0.45 meters;

[0136] Remove the causal reasoning module (without causal reasoning): ADE = 0.38 m;

[0137] Remove hierarchical structure (w / o hierarchy): ADE = 0.32 meters;

[0138] Complete model (HSTCG+PFSDM): FDE=0.24 meters, verifying the core role of hierarchical structure and causal reasoning module.

[0139] 3) Feature discrimination verification: such as Figure 5 As shown, by using the T-SNE algorithm to reduce the dimensionality of the context condition vector c to a 2D space, samples of three intentions—crossing the road (red), waiting on the roadside (blue), and going straight along the road (green)—form a clearly defined cluster with no obvious mixing, proving that HSTCG can effectively extract scene features with distinctiveness.

[0140] Causal attention visualization: Figure 7This is a real-time spatiotemporal causal attention heatmap, where the horizontal axis represents the temporal evolution from the historical frame (t-15) to the current frame (t=0), and the vertical axis corresponds to the "pose-entity-environment" hierarchical nodes described in this invention. The color intensity in the heatmap directly reflects the modulation effect of causal weights on the attention coefficients: when a target pedestrian approaches the intersection, the heatmap values ​​corresponding to environmental nodes (traffic lights, pedestrian crossings) increase significantly (tending towards dark red), indicating that the causal modulation GAT automatically enhances the attention to key environmental elements, verifying the dynamic modulation effect of causal weights.

[0141] Verification of the reverse denoising process: such as Figure 8 As shown, from t=1000 (pure noise) to t=0 (generation state), the noisy point cloud gradually condenses under the guidance of the context vector. Through gradual condensation and denoising, it eventually converges to generate three clear and independent future trajectory sequences, which correspond to the three intentions of traversing, waiting, and going straight, respectively. This intuitively demonstrates the multimodal generation capability of PFSDM.

[0142] In actual urban road tests, the pedestrian intent recognition system of this embodiment demonstrated that the model inference real-time performance meets the requirements of autonomous driving engineering (end-to-end latency of single-frame processing ≤80ms), the intent recognition accuracy reaches 92.3%, and the multimodal prediction covers more than 95% of real pedestrian behavior. Among them, the low probability intent prediction accuracy in high-risk scenarios (such as pedestrians hesitating to cross the road) reaches 87%, providing sufficient reaction time (≥2 seconds) for autonomous vehicles and effectively reducing the risk of collision.

[0143] Example 2: A pedestrian intent recognition system based on hierarchical spatiotemporal causal graphs and state diffusion, used to implement the above method. This system can be integrated into autonomous vehicles, intelligent surveillance cameras, or mobile robots, and specifically includes:

[0144] Data acquisition and preprocessing module: Includes a high-resolution camera and feature processing unit, performs target detection, pose estimation and scene segmentation, and outputs structured initial features;

[0145] Graph Construction and Encoding Module: Includes HSTCG construction unit and graph evolution encoding unit. The construction unit dynamically generates a three-layer causal graph sequence, and the encoding unit outputs a context condition vector through causal modulation GAT and Transformer. c ;

[0146] Probability prediction module: Loaded with the PFSDM, including forward diffusion unit, reverse denoising unit and clustering unit, performs multiple rounds of sampling and clustering, and outputs multimodal future state prediction results;

[0147] Results output module: Includes formatting unit and communication interface, which converts intent category, trajectory sequence, probability distribution and uncertainty evaluation results into standardized format and transmits them to downstream decision planning module via CAN / Ethernet;

[0148] Computing platform: NVIDIA Drive AGX Orin, supporting mixed-precision training and real-time inference.

[0149] Example 3: A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described pedestrian intent recognition method based on hierarchical spatiotemporal causal graphs and state diffusion.

[0150] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A pedestrian intent recognition method based on hierarchical spatiotemporal causal graphs and state diffusion, characterized in that, Includes the following steps: Step S1: Acquisition and preprocessing of temporal observation data: Acquire continuous temporal observation data containing the target pedestrian, perform target detection, pose estimation, scene segmentation and feature normalization to obtain pedestrian pose features, dynamic target features and scene environment features, and form structured initial features; Step S2, Hierarchical Spatiotemporal Causal Graph Construction: Based on the initial features, a hierarchical spatiotemporal causal graph HSTCG sequence that evolves over time is constructed. The hierarchical spatiotemporal causal graph uses multi-layer nodes and learnable causal edges to structurally represent the dynamic interaction relationships within the scene. Step S3, Graph Evolution Encoding: Input the hierarchical spatiotemporal causal graph HSTCG sequence into the graph evolution encoder, and output the context condition vector containing spatiotemporal dynamics and causal relationships; Step S4, Probabilistic Future State Diffusion Model Prediction: Input the context condition vector into the Probabilistic Future State Diffusion Model (PFSDM), and generate one or more latent future state variables containing future trajectory sequences and intention classifications from the prior noise distribution through a condition-guided inverse denoising process. Step S5, Joint Loss Calculation and Parameter Optimization: Based on the generated future state and the real future state labels, construct a joint total loss function that includes diffusion loss, intent classification loss and trajectory regression loss; adjust the parameters of the graph evolution encoder and the probabilistic future state diffusion model end-to-end through the backpropagation algorithm until the model converges; Step S6, Multimodal Inference Output: Perform 50-100 parallel reverse denoising processes, perform cluster analysis on the generated future state latent variables, and output multimodal prediction results containing intention categories, average trajectories, probability distributions, and uncertainty assessments.

2. The pedestrian intent recognition method based on hierarchical spatiotemporal causal graph and state diffusion according to claim 1, characterized in that, In step S2, a hierarchical spatiotemporal causal graph HSTCG sequence {G1,G2,...,G} that evolves over time is constructed. Tobs }, each G t =(V t A t ,F t Specifically, this includes: Step S2.1: Define a three-level node set V t The bottom layer consists of posture nodes representing key points of a pedestrian's body; the middle layer consists of entity nodes representing dynamic traffic participants; and the top layer consists of environmental nodes representing key elements such as traffic lights and pedestrian crossings. Step S2.2: Define the set A of three types of edges. t Intra-layer edges connect pose nodes to represent the skeletal structure; cross-layer edges connect pose nodes and entity nodes to aggregate pose information; inter-layer edges connect entity nodes and environment nodes or different entity nodes to represent interaction relationships; and the weights of inter-layer edges are learnable causal weights. w ij , w ij ∈[0,1], i and j are the index identifiers of nodes in the hierarchical spatiotemporal causal graph HSTCG; Step S2.3: Define the node feature matrix F t The original features of each node are mapped to a unified feature space through a multilayer perceptron, thus completing the node feature initialization. Step S2.4: Using a learnable causal reasoning module, based on the dynamic interaction information between nodes, the causal weights of the inter-layer edges are dynamically calculated to quantify the potential causal influence between nodes.

3. The pedestrian intent recognition method based on hierarchical spatiotemporal causal graph and state diffusion according to claim 2, characterized in that, In step S2.4, the causal weight calculation is implemented through a causal relationship reasoning module, which is a two-layer MLP: the input is node pair features ( h i , h j The impact on the score is calculated using the following formula. e ij , e ij = a T ⋅ LeakyReLU W [ h i || h j ]), Then through w ij =sigmoid(LayerNorm( e ij The causal weights are normalized to the interval [0,1], where a For attention parameter vector, W Let T be the linear projection matrix, and T be the transpose of the matrix.

4. The pedestrian intent recognition method based on hierarchical spatiotemporal causal graph and state diffusion according to claim 3, characterized in that, In step S3, the processing procedure of the graph evolution encoder includes: Step S3.1: At each time step, a causal modulated graph attention network (GAT) is used, based on the causal weights of the inter-layer edges. w ij Modulate the attention coefficient, aggregate neighborhood information for the current hierarchical spatiotemporal causal graph, and update the node representation; The calculation of attention coefficients in the Causal Modulation Graph Attention Network (GAT) includes: Raw attention score: ; Modulation coefficients: , N i For nodes i The set of neighboring nodes; Node update: ; Add a dropout layer with a probability of 0.1-0.2 after each GAT layer; Step S3.2: Extract the temporal representation sequence of the target pedestrian entity node, input it into the Transformer encoder to capture long-term dependencies, and obtain the context condition vector through the label vector or average pooling; The Transformer encoder includes: Multi-head self-attention sublayer: 4-8 heads, each head has a dimension of... D model / Number of heads, using scaled dot product attention and future time step mask; Feedforward Neural Network (FFN) Sublayer: Hidden layer dimension is 4× D model The activation function is GELU; Residual connections and layer normalization: Output = Layer Norm(Input + Sublayer(Input)), the input sequence needs to be added with sinusoidal position encoding; where Input is the original input feature sequence / feature vector of the multi-head self-attention sub-layer of the Transformer encoder and the feedforward neural network FFN sub-layer.

5. The pedestrian intent recognition method based on hierarchical spatiotemporal causal graph and state diffusion according to claim 4, characterized in that, Step S4 specifically includes: Step S4.1: Define the latent variables of the original future state z 0, from the future T pred The pedestrian two-dimensional relative coordinate sequence at each time step and N class The vector is formed by concatenating one-hot vectors, z0∈R (Tpred×2)+Nclass) ; T pred The time step for future prediction is set to 20-50 frames. Each frame contains two dimensions: the pedestrian's horizontal position coordinate X and vertical position coordinate Y relative to the reference point. N class This represents the number of pedestrian intent categories, ranging from 3 to 5 categories, encoded as a one-hot vector. Step S4.2: Design the forward diffusion process, in T Dispatching via monotonically increasing noise within each time step β t Gaussian noise is gradually injected into z0 to generate arbitrary time intervals. t Noisy latent variables z t , ； in, Let I be the standard Gaussian noise vector, and let α be the identity matrix. t =1-β t , α t To retain a certain percentage of true information for each step. The percentage of true information retained from the previous t steps; Set the total number of diffusion steps T = 800-1200, and define the noise variance sequence. β t t∈[1,T], noise variance β at any time step t The noise variance β1 increases linearly from the initial noise variance β1 = 0.0001 - 0.001 to the final noise variance β2. T =0.01-0.02; Step S4.3: Construct a U-Net-based denoising network Input noisy latent variables z t Time step coding emb ( t and context condition vector c Predict the noise added during the forward process. The time-step encoding is implemented using sinusoidal positional encoding, and the context condition vector is injected into each residual block of the U-Net through a cross-attention mechanism; the denoising network architecture design includes: Step S4.3.1: The U-Net basic skeleton of downsampling-intermediate feature processing-upsampling is adopted, with 4-6 layers in each of the downsampling and upsampling stages. Downsampling uses convolution to extract global spatiotemporal features, and upsampling uses transposed convolutional layers to restore local trajectory details. Step S4.3.2: In the intermediate feature processing stage of U-Net, 1-3 layers of self-attention modules are introduced. The number of attention heads in the self-attention modules is 4-8, which are used to capture the long-term dependencies of pedestrian future trajectory sequences. Step S4.3.3: Encoding the time step emb ( t Generated through sinusoidal positional encoding, with dimensions 256-512, and related to the context condition vector. c After splicing, each residual block of U-Net is injected through a cross-attention module. The number of attention heads in the cross-attention module is the same as that in the self-attention module, which is used to guide the network to focus on key causal interaction information in the scene.

6. The pedestrian intent recognition method based on hierarchical spatiotemporal causal graph and state diffusion according to claim 5, characterized in that, The iterative calculation of the reverse denoising process includes: Initial noise vector is sampled from a standard normal distribution. z T ~N(0,I), iterate in reverse from t=T to t=1, and perform the denoising step: Each iteration calculates the noisy latent variable of the previous time step using the following formula. z t-1 : ； in, w t For random noise, when t>1, w t ~N(0,I); at t=1, w t =0; , σ t This is a parameter for controlling the randomness of sampling; The reverse denoising process is executed in parallel N times, and cluster analysis is performed on the generated N future state latent variables to obtain multimodal prediction results and corresponding probabilities.

7. The pedestrian intent recognition method based on hierarchical spatiotemporal causal graph and state diffusion according to claim 6, characterized in that, Step S5 specifically includes: Step S5.1: Construct a structure including diffusion loss L diff Intent Classification Loss L intent With trajectory regression loss L traj Joint total loss function L total ; L total =λ diff L diff +λ intent L intent +λ traj L traj ， Where, λ diff , λ intent , λ traj These are the loss weighting coefficients; Diffusion loss L diff To calculate the mean square error between the predicted noise and the actual noise, i.e. E represents the... z 0, c, Calculation of the expected value of t, ||·|| 2 It is an L2 norm; Intended classification loss L intent = CrossEntropy ( softmax ( Int pred ), Int true ),in Int pred To predict z The intent logic value separated in 0, Int true Labeling as true intent softmax The function is used to transform logical values ​​into a probability distribution. CrossEntropy This is the function for calculating cross-entropy; Trajectory Regression Loss L traj = SmoothL 1( Tr pred , Tr true ),in Tr pred To predict z The future trajectory sequence separated from 0, Tr true The data represents the true trajectory sequence, and SmoothL1 is the function for calculating the smoothing L1 loss. Step S5.2: Based on the joint total loss function, update the graph evolution encoder and PFSDM parameters through backpropagation using the AdamW optimizer for end-to-end training; the training batch size is set to 16-64, and the number of training rounds is set to 300-600 rounds. Mixed precision training is used to accelerate convergence. The early stopping strategy is set as follows: if the pedestrian trajectory endpoint error on the validation set does not decrease for 15-25 consecutive rounds, then training is stopped and the current optimal model weights are saved.

8. The pedestrian intent recognition method based on hierarchical spatiotemporal causal graph and state diffusion according to claim 7, characterized in that, In step S6, the clustering analysis uses the DBSCAN algorithm: eps=0.4-0.6, min_samples=3-5. Clustering is performed based on trajectory similarity and intent consistency. The probability of each cluster is: number of samples in the cluster / total number of samples. The top-3 clusters with the highest probabilities are output as the multimodal results.

9. A pedestrian intent recognition system based on hierarchical spatiotemporal causal graphs and state diffusion, characterized in that, To implement the method of any one of claims 1-8, comprising: Data acquisition and preprocessing module: Includes a high-resolution camera and feature processing unit, performs target detection, pose estimation and scene segmentation, and outputs structured initial features; Graph Construction and Encoding Module: Includes HSTCG construction unit and graph evolution encoding unit. The construction unit dynamically generates a three-layer causal graph sequence, and the encoding unit outputs a context condition vector through causal modulation GAT and Transformer. c ; Probability prediction module: Loaded with the PFSDM, including forward diffusion unit, reverse denoising unit and clustering unit, performs multiple rounds of sampling and clustering, and outputs multimodal future state prediction results; Results output module: Includes formatting unit and communication interface, which converts intent category, trajectory sequence, probability distribution and uncertainty evaluation results into standardized format and transmits them to downstream decision planning module via CAN / Ethernet; Computing platform: It adopts a dedicated computing platform for autonomous driving, which supports mixed-precision training and real-time inference.

10. A computer-readable storage medium storing a computer program, characterized in that, When the program is executed by the processor, it implements the pedestrian intent recognition method based on hierarchical spatiotemporal causal graph and state diffusion as described in any one of claims 1-8.

Images