Intelligent traffic management and control method based on mobile object position prediction

Abstract

The application provides an intelligent traffic control method based on mobile object position prediction, which comprises the following steps: adopting a YOLOv8 target detection framework and a DeepSORT cross-frame tracking algorithm to extract a structured trajectory dataset of mobile objects mainly including pedestrians and vehicles in a video; performing space-time coding on historical trajectories to construct a space-time sequence input available for a model; capturing motion trend features through a time attention branch and quantifying the spatial interaction relationship between mobile objects through a spatial attention branch; constructing a multi-factor interaction force function model; fusing the space-time attention features and social force constraints to initialize an LSTM decoder; injecting Gaussian noise in the decoding process to sample and generate multiple candidate trajectories; converting the predicted trajectories into a traffic heat map at a road intersection, and automatically generating a congestion warning and a signal light control instruction when the traffic exceeds a threshold value. The application has high prediction accuracy and strong robustness, and can be effectively applied to intelligent control and congestion warning at a traffic intersection.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent transportation systems, specifically to an intelligent traffic control method based on the prediction of moving object location. This method is used to automatically predict the future movement trajectory of pedestrians and quantify the interactive influence of multiple factors, so as to improve the efficiency of intelligent traffic control, reduce the risk of traffic accidents, and provide decision support for autonomous driving systems. Background Technology

[0002] With the acceleration of urbanization and the rapid development of intelligent transportation systems, pedestrian trajectory prediction technology is playing an increasingly important role in fields such as autonomous driving, intelligent monitoring, and traffic flow management. Accurately predicting the future movement trajectory of pedestrians is crucial for ensuring traffic safety, optimizing traffic signal control, and enhancing the decision-making capabilities of autonomous driving systems. However, pedestrian movement behavior is influenced by multiple factors, including their own intentions, social interactions with surrounding pedestrians, vehicle avoidance behavior, and environmental obstacles, making trajectory prediction a highly challenging task.

[0003] Traditional pedestrian trajectory prediction methods are mainly based on physical models or simple statistical learning methods. These methods describe pedestrian behavior by manually designing rules. Although they have a certain degree of interpretability, they perform poorly in complex and ever-changing real-world scenarios and are difficult to adapt to complex social interactions in densely populated environments. With the rapid development of deep learning technology, methods based on recurrent neural networks and long short-term memory networks have made some progress in trajectory prediction, but the following shortcomings still exist: 1) Existing deep learning-based methods do not adequately model spatiotemporal dependencies, usually only performing sequence modeling in the time dimension, failing to simultaneously and effectively capture the motion trend in the time dimension and the social interaction between pedestrians in the spatial dimension, resulting in limited prediction accuracy; 2) Existing methods lack effective quantification mechanisms for multi-factor interactions between pedestrians, vehicles, and the environment, ignoring the impact of vehicle avoidance behavior and environmental obstacles on pedestrian trajectories, resulting in low accuracy of prediction results in mixed traffic scenarios; 3) Most methods output a single deterministic trajectory prediction result, failing to reflect the multimodal characteristics and uncertainties of pedestrian behavior, and making it difficult to cover the various movement modes that pedestrians may adopt; 4) Existing methods usually only focus on trajectory prediction itself, lacking an effective mechanism to transform the prediction results into practical applications, limiting the practical value of the technology. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide an intelligent traffic management method based on the prediction of the location of moving objects. By constructing a spatiotemporal multi-head attention mechanism and a social force function model, high-precision multimodal trajectory prediction is achieved, and the prediction results are transformed into traffic flow heat maps for intelligent traffic management.

[0005] The present invention provides an intelligent traffic control method based on moving object location prediction, comprising the following steps: Step S1: Using the YOLOv8 object detection framework and DeepSORT cross-frame tracking algorithm, extract the structured trajectory dataset of moving objects in the video. The moving objects include people and / or vehicles. Each trajectory includes: frame number, target ID, pixel coordinates of the bounding box center point, and size. Step S2: Spatiotemporally encode the historical trajectory to generate a structured representation containing motion features and environmental interaction features, and construct a spatiotemporal sequence input usable by the model; Step S3: Capture motion trend features through the temporal attention branch, and quantify the spatial interaction relationships between moving objects through the spatial attention branch; Step S4: Construct a multi-factor interactive force function model to quantify the repulsive force from surrounding pedestrians, the avoidance force from vehicles, and the environmental obstacle force experienced by the moving object; Step S5: Fuse spatiotemporal attention features and social force constraints to initialize the LSTM decoder; inject Gaussian noise during the decoding process and sample to generate multiple candidate trajectories; Step S6: Convert the predicted trajectory into an intersection traffic flow heatmap. When the traffic flow exceeds the threshold, automatically generate congestion warnings and traffic light control instructions, and upload them to the traffic management cloud platform in real time.

[0006] Furthermore, step S1 specifically includes: Step S101: Perform object detection based on the YOLOv8 framework; Step S102: Perform multi-target tracking using the DeepSORT algorithm; Furthermore, step S101 specifically includes: First, images are extracted frame by frame from the original surveillance video stream to obtain a continuous image sequence I1, I2, …, I T , where T is the total number of video frames.

[0007] Then, each frame image I t (Where t=1, 2, …, T) Input to a pre-trained YOLOv8 network, and after network detection, output a set β of candidate detection boxes for all pedestrians and vehicles in the current frame. t ; Where the bounding box set β t It can be represented as: β t ={b1, b2, …, b Mbox} Where Mbox represents the number of boxes detected in the current frame; Each bounding box b i It can be represented as: bi =(x, y, w, h, conf, cls) Where x is the x-coordinate of the center point, y is the y-coordinate of the center point, w is the width of the bounding box, h is the height of the bounding box, conf is the confidence score of the current bounding box, and cls is the target category identifier. This method detects and tracks two types of targets: pedestrians and vehicles.

[0008] Finally, the nonmaximum suppression algorithm is applied to the detection box set β. t The detection boxes are filtered, and the non-overlapping detection boxes with the highest confidence are retained to generate the final optimized detection set. .

[0009] Furthermore, step S102 specifically includes: First, for the detection results of each frame ={d1, d2, …, d Nbox The state prediction of all established tracking trajectories (including pedestrian and vehicle trajectories) is performed using a Kalman filter to obtain their predicted state vector X and prior covariance matrix P in the current frame. The state vector X is defined as an 8-dimensional vector: X=[x, y, w, h, , , , ] T Where x is the x-coordinate of the center point, y is the y-coordinate of the center point, w is the width of the bounding box, and h is the height of the bounding box. For lateral velocity, For longitudinal velocity, The width change rate, The superscript indicates the first derivative with respect to time; State prediction calculations are based on the assumption of uniform motion, where the rates of change of position and size remain constant, and are performed using the state transition matrix F: Among them, X t|t-1 X is the predicted state vector at time t based on data up to time t-1. t-1|t-1 F is the optimal state vector after fusing the observation data at time t-1, and F is an 8×8 state transition matrix; The formula for predicting the prior covariance matrix is: Among them, P t|t-1 Let P be the prior covariance matrix. t-1|t-1 Let F be the posterior covariance matrix. TLet Q be the transpose of the state transition matrix, and let Q be the process noise covariance matrix.

[0010] Then, construct the association cost matrix between the detection box and the predicted tracking target; Finally, the Hungarian algorithm is used for optimal matching and to manage the tracking trajectory; The final output is a set of trajectory segments for all pedestrians and vehicles. Each trajectory segment T id ={(x1, y1, w1, h1), (x2, y2, w2, h2), …} contains the timestamps and bounding box sequences of pedestrians and vehicles with ID id in consecutive frames.

[0011] Furthermore, step S2 specifically includes: Step S201: Encode the trajectory segment using spatiotemporal features to generate a structured feature representation that includes a time decay factor and spatial correlation weights; Step S202: In structured features Based on this, motion pattern compensation and anomaly correction are performed; Furthermore, step S201 specifically includes: First, calculate the velocity vector and spatial correlation degree to generate an m-dimensional spatial encoding vector. ; Space Vector Encoding It consists of the spatial correlation of all time steps. Then, the time decay factor is calculated to generate an m-dimensional time encoding vector. ; Time-coded vector It consists of the time decay factor of all time steps; Finally, structured trajectory features The generation operation is represented as: Among them, Γ k For the trajectory segment at time k, LSTM(•) is a Long Short-Term Memory network encoder that takes the trajectory coordinate sequence as input and outputs the hidden state features; ⊙ is an element-wise multiplication operation.

[0012] Furthermore, step S202 specifically includes: First, calculate the attention weight vector A based on motion features. v , used for feature compensation; Then, perform the feature compensation correction operation: in, For the compensated feature vector, W c For the feature compensation weight matrix, For spatial encoding vectors, This is a tensor outer product operation; Finally, output the compensated structured pedestrian trajectory sequence T: T= Where N is the length of the trajectory sequence.

[0013] Furthermore, step S3 specifically includes: Step S301: Construct a temporal attention branch to capture motion trend features; Step S302: Construct spatial attention branches to quantify spatial interaction relationships between pedestrians; Step S303: Fuse spatiotemporal attention features. Output O from the temporal branch. t With spatial branch output O s The features are fused to generate the final spatiotemporal context feature F. st : F st = LayerNorm(O t + O s ) LayerNorm(•) is the layer normalization operation.

[0014] Furthermore, step S301 specifically includes: First, the structured pedestrian trajectory sequence T output in step S2 is... As input, each of Let N represent the trajectory feature vector at time t, and N be the number of observation time steps. A multi-head self-attention mechanism is applied in the time dimension to calculate the dynamic dependency weights on the historical sequence. For the h-th attention head, the query vector is calculated. Key vector Sum value vector ; Then, the attention weight matrix in the time dimension is calculated. ; Finally, the temporal attention output of the h-th head is calculated. And then splice and project the outputs of all H heads: in, To output the projection matrix, Concat(•) is the concatenation operation; Furthermore, step S302 specifically includes: First, suppose there are Nped pedestrians in the scene, and their features at time t are used to define { , , …, As input, calculate the spatial query vector for each pedestrian at that time step. Spatial key vector and spatial value vector : Then, the spatial interaction attention weight e between pedestrian i and pedestrian j is calculated. ij The spatial attention weights a are then normalized to obtain the standardized spatial attention weights a. ij : Finally, based on the spatial attention weighted aggregate value vector, the updated features of pedestrian i at time step t are obtained. The spatial features of each time step are concatenated in chronological order to obtain the output feature O of the spatial attention branch. s .

[0015] Furthermore, step S4 specifically includes: Step S401: Construct a pedestrian-vehicle interaction force model. Based on the structured trajectory features T and vehicle historical trajectories output in step S2, calculate the avoidance force exerted by vehicle v on pedestrian i. .

[0016] Step S402: Construct a pedestrian-environment interaction force model. Using the gridded representation φ of the environmental scene, calculate the repulsive force exerted on pedestrian i by the static environmental obstacle o. .

[0017] Step S403: Construct a target attraction model. Pedestrians typically move towards their target location. Calculate the attraction force exerted by the pedestrians in the target direction. .

[0018] Step S404: Construct a pedestrian-pedestrian interaction force model. Pedestrian i experiences social repulsion from other pedestrians j. This repulsive force is used to maintain social distancing and avoid collisions: Among them, A pp B is the strength coefficient of the repulsive force between pedestrians. pp d is the distance attenuation coefficient between pedestrians. i,j Let be the Euclidean distance between the center points of pedestrian i and pedestrian j. Let be the unit direction vector pointing from pedestrian j to pedestrian i.

[0019] Step S405: Calculation of resultant force and quantification of impact. Total social force experienced by pedestrian i. The vector sum of target attraction, pedestrian repulsion, vehicle avoidance force, and environmental obstacle force: Map the magnitude of the social force vector to the scalar interaction strength I. i ∈[0,1], used for subsequent trajectory decoding: Where σ(•) is the Sigmoid function, w I and b I The weights and bias parameters of the linear mapping are learned end-to-end for trajectory prediction tasks.

[0020] Furthermore, step S5 specifically includes: Step S501: Apply the spatiotemporal context features generated in step S3 The interaction impact level I quantized in step S4 is fused together to initialize an independent LSTM decoder for each pedestrian, generating their respective initial hidden state h0 and cell state c0. Step S502: Use an LSTM decoder to perform autoregressive trajectory prediction and generate multimodal trajectories by injecting random noise; Furthermore, step S501 specifically includes: First, the feature vector The fused feature vector is obtained by concatenating it with the scalar influence level I. : Then, the initial hidden state h0 of the decoder is calculated through a fully connected layer and a ReLU activation function: Among them, W h Let b be the weight matrix of the hidden state. h This is the corresponding bias vector; Finally, the initial cell state c0 of the decoder is calculated: Among them, W c Let b be the weight matrix of the cell states. c This is the corresponding bias vector.

[0021] Furthermore, step S502 specifically includes: First, set the time window for trajectory prediction to T. pred Step. In each step t (t=1, 2, ...,T) of the decoding process. pred ), and the coordinates predicted in the previous step (Using the last observed coordinates at t=1) and the sampled Gaussian noise ɛt Perform concatenation, and use it as input for the current step; Then, by independently sampling K groups of noise sequences from a Gaussian distribution { }, { }, …, { (Each group contains Tpred time steps of noise), which can generate K different future trajectories. }, { }, …, { Each trajectory corresponds to a future motion pattern, and the rationality score of each trajectory can be calculated based on scene constraints or historical patterns.

[0022] Furthermore, step S6 specifically includes: Step S601: Divide the prediction region into a uniform grid and calculate the future time window. The density of trajectory points in each grid cell is used to generate a traffic flow heatmap H. flow .

[0023] Step S602: When the flow rate value of a certain area in the heat map exceeds the preset threshold τ con At that time, the system automatically generates a congestion warning message and sends it to the management and control platform, with a threshold τ. con The system dynamically adjusts traffic flow based on historical traffic statistics. Simultaneously, it generates traffic light timing optimization suggestion instructions (C) based on predicted traffic distribution. sign .

[0024] Step S603: H flow Warning signals and C sign The data is packaged into data packets according to traffic data exchange standards and uploaded to the traffic management cloud platform in real time via a wireless communication module. The update frequency can be set to once per second for macro-level traffic monitoring and real-time decision support.

[0025] The beneficial effects of this invention include: (1) By constructing a complete technical framework of "trajectory extraction-spatiotemporal coding-attention fusion-social force modeling-multimodal prediction", this invention can accurately predict the future trajectory of pedestrians at complex traffic intersections, effectively overcoming the bottleneck problem that existing methods are difficult to effectively capture long temporal dependencies and complex spatial interactions, and providing reliable technical support for behavior understanding and decision planning in intelligent transportation systems; (2) The spatiotemporal multi-head attention mechanism proposed in this invention captures the dynamic trend of pedestrian movement and the micro-interaction relationship between pedestrians through independent temporal attention branches and spatial attention branches, respectively, realizing the decoupling and refined modeling of spatiotemporal dimensions, enhancing the feature extraction capability of key time segments and important interactive objects, and significantly improving the accuracy of trajectory prediction. (3) This invention innovatively introduces a social force function model, which quantifies the interactions between pedestrians, pedestrians, vehicles, and the environment into computable repulsive forces, avoidance forces, and attractive forces, transforming the implicit social influences that are difficult to model into explicit physical constraints, thereby enhancing the interpretability and robustness of the model in complex and dense scenarios. (4) In the decoding and prediction stage, the present invention performs multimodal sampling by injecting Gaussian noise, which can generate multiple reasonable and diverse future trajectories, effectively reflecting the uncertainty of pedestrian future behavior, overcoming the shortcomings of deterministic prediction models that output a single value and cannot cope with multiple possibilities, and providing more comprehensive situational awareness information for downstream decision-making systems. (5) This invention transforms the predicted trajectory into an intuitive intersection traffic heat map and sets dynamic thresholds to generate congestion warnings and traffic light control instructions, realizing a seamless connection from micro-trajectory prediction to macro-traffic control, enabling the research results to be directly applied to the intelligent traffic management platform, and improving the practical value and implementation efficiency of the technology. (6) The trajectory extraction scheme combining YOLOv8 and DeepSORT adopted in this invention takes into account both the accuracy of detection and the stability of tracking, ensuring the reliability of the model input data. The entire system is based on an end-to-end deep learning architecture, has powerful representation learning capabilities, and is easy to fuse with other sensor data (such as lidar and millimeter-wave radar), and has good scalability and adaptability. Attached Figure Description

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the process of an intelligent traffic control method based on the location prediction of a moving object according to the present invention; Figure 2 This is a schematic diagram illustrating the data flow process of target detection and tracking based on YOLOv8 and DeepSORT in this invention; Figure 3 This is a schematic diagram illustrating the spatiotemporal dual-stream attention network architecture and its internal computational details in this invention; Figure 4 This is a schematic diagram illustrating the principle of the social force interaction calculation model and the multi-factor fusion mapping of the present invention; Figure 5 This is a schematic diagram illustrating the process of generating intersection traffic heatmaps based on predicted trajectories according to the present invention. Figure 6 This is a schematic diagram of the visual terminal interface of the intelligent traffic monitoring system in an embodiment of the present invention. Detailed Implementation

[0027] The preferred embodiments of the present invention will now be described in detail. It should be understood that the preferred embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0028] To address the problems of insufficient spatiotemporal modeling, lack of physical constraints, and limited prediction results in traditional pedestrian trajectory prediction methods, this invention proposes a deep learning prediction method that integrates spatiotemporal attention mechanisms and social force functions. By utilizing a multi-head attention mechanism to construct a perception-encoding-prediction closed loop, it dynamically predicts the future movement trajectory of pedestrians and applies it to intelligent traffic flow management, significantly improving prediction accuracy and practical value.

[0029] This invention extracts structured trajectory data using YOLOv8 and DeepSORT algorithms, and completes the feature representation of historical trajectories based on a spatiotemporal coding network; it constructs a spatiotemporal multi-head attention mechanism to capture motion trends in the time dimension and quantify social interactions in the spatial dimension; it introduces a social force function to explicitly model the third-order interaction influence of pedestrians, vehicles, and the environment; and it uses an LSTM decoder combined with a Gaussian noise injection mechanism to generate multiple candidate trajectories to achieve multimodal prediction.

[0030] In the prediction process, a sliding mechanism between historical trajectory windows and future prediction windows is introduced to enhance the model's temporal generalization ability. The social force function design considers multiple factors such as target attraction, pedestrian repulsion, vehicle avoidance, and environmental obstacle forces. A comprehensive influence vector is constructed through weighted summation to ensure that the prediction results conform to both data-driven statistical laws and the real constraints of the physical scenario.

[0031] like Figure 1 As shown, the present invention provides an intelligent traffic control method based on moving object location prediction, comprising the following steps: Step S1: Using the YOLOv8 object detection framework and DeepSORT cross-frame tracking algorithm, extract the structured trajectory dataset of moving objects in the video. The moving objects include people and / or vehicles. Each trajectory includes: frame number, target ID, pixel coordinates of the bounding box center point, and size. Step S2: Spatiotemporally encode the historical trajectory to generate a structured representation containing motion features and environmental interaction features, and construct a spatiotemporal sequence input usable by the model; Step S3: Capture motion trend features through the temporal attention branch, and quantify the spatial interaction relationships between moving objects through the spatial attention branch; Step S4: Construct a multi-factor interactive force function model to quantify the repulsive force from surrounding pedestrians, the avoidance force from vehicles, and the environmental obstacle force experienced by the moving object; Step S5: Fuse spatiotemporal attention features and social force constraints to initialize the LSTM decoder; inject Gaussian noise during the decoding process and sample to generate multiple candidate trajectories; Step S6: Convert the predicted trajectory into an intersection traffic flow heatmap. When the traffic flow exceeds the threshold, automatically generate congestion warnings and traffic light control instructions, and upload them to the traffic management cloud platform in real time.

[0032] It should be noted that any process or method description in the flowcharts of this invention or otherwise described herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which the embodiments of the invention pertain.

[0033] like Figures 2 to 6 As shown, the specific steps of the above method will be further explained below through a specific embodiment.

[0034] Step S1 includes the following sub-steps: In this embodiment, the target detection and tracking data transfer process based on YOLOv8 and DeepSORT is as follows: Figure 2 As shown; Step S101: Perform object detection based on the YOLOv8 framework; In step S101, the object detection steps based on the YOLOv8 framework are as follows: First, images are extracted frame by frame from the original surveillance video stream to obtain a continuous image sequence I1, I2, …, I T , where T is the total number of video frames.

[0035] In this embodiment, traffic intersection surveillance video was selected as test data. The surveillance camera was installed at a height of 6 meters, with a downward angle of 45 degrees. The video resolution was 1920×1080 pixels, the frame rate was 25fps, and the total video duration was 300 seconds, totaling T=7500 frames. The monitoring area covered the pedestrian crossing and surrounding lanes, and the scene contained an average of 15-30 pedestrian targets and 5-10 vehicle targets.

[0036] Then, each frame image I t (Where t=1, 2, …, T) Input to a pre-trained YOLOv8 network, and after network detection, output a set β of candidate detection boxes for all pedestrians and vehicles in the current frame. t ; In this embodiment, the YOLOv8-m model is used, with an input size of 640×640 pixels. It is pre-trained on the COCO dataset and fine-tuned on the pedestrian-vehicle dataset. Taking frame 100 as an example, the number of detected candidate bounding boxes Mbox=18, including 14 pedestrian categories and 4 vehicle categories. For example, the first detection box in frame 100 is: (x, y, w, h, conf,cls)=(320, 450, 60, 180, 0.92, person); Finally, the nonmaximum suppression algorithm is applied to the detection box set β. t The detection boxes are filtered, and the non-overlapping detection boxes with the highest confidence are retained to generate the final optimized detection set. ; In this embodiment, the cross-union ratio threshold for non-maximum suppression is set to T. nms =0.45. Taking frame 100 as an example, the initial candidate detection box set β 100 ={b1, b2, …, b 18 The output set is} The specific process of step-by-step screening is as follows: (1) For set β 100 The 18 candidate detection boxes in the dataset have confidence scores of conf(b1)=0.92, conf(b2)=0.88, conf(b3)=0.95, conf(b4)=0.65, conf(b5)=0.71, ..., conf(b 18 =0.58; (2) The first iteration starts from β 100 Select the detection box b with the highest confidence level. max =b3, move b3 from β 100 Remove and add to In the middle, at this time ={b3}, calculate the intersection-union ratio (IoU) of b3 with the remaining detection boxes. The IoU ratios are as follows: IoU(b3, b1) = 0.00 < 0.45, keep b1; IoU(b3, b2) = 0.00 < 0.45, keep b2; the remaining detection boxes do not overlap with b3, so all are kept. (3) The second round of iterations starts from the remaining β 100 Select the detection box b with the highest confidence level. max =b1, move b1 from β 100 Remove and add to In the middle, at this time ={b3, b1}, calculate the intersection-union ratio (IoU) of b1 with the remaining detection boxes: IoU(b1, b2)=0.00<0.45, keep b2; (4) When the fifth iteration processes the overlapping detection boxes b4 and b5, when it is b5's turn, calculate IoU(b5, b4)=0.738>0.45, delete b4, and keep b5; (5) Repeat the above steps until β 100 Empty.

[0037] After nonmaximum suppression processing, the final output set of detection boxes ={d1, d2, …, d 15 This includes 12 pedestrian detection boxes and 3 vehicle detection boxes.

[0038] Step S102: Perform multi-target tracking using the DeepSORT algorithm; In step S102, the steps for multi-target tracking using the DeepSORT algorithm are as follows: First, for the detection results of each frame ={d1, d2, …, d Nbox The state prediction of all established tracking trajectories (including pedestrian and vehicle trajectories) is performed using a Kalman filter to obtain their predicted state vector X and prior covariance matrix P in the current frame. The state vector X is defined as an 8-dimensional vector: X=[x, y, w, h, , , , ] T Where x is the x-coordinate of the center point, y is the y-coordinate of the center point, w is the width of the bounding box, and h is the height of the bounding box. For lateral velocity, For longitudinal velocity, The width change rate, The superscript indicates the first derivative with respect to time; State prediction calculations are based on the assumption of uniform motion, where the rates of change of position and size remain constant, and are performed using the state transition matrix F: Among them, X t|t-1 X is the predicted state vector at time t based on data up to time t-1. t-1|t-1 To represent the optimal state vector after fusing the observation data at time t-1, in this embodiment, F is an 8×8 state transition matrix, specifically in the form of: F= In this context, the 1 on the main diagonal indicates that the state component retains its original value, the Δt in the first 4 rows and columns 5-8 indicates that the position state is updated by the corresponding velocity state at time intervals Δt, and the 0 in the remaining positions indicates that the state component is not affected by other components.

[0039] In this embodiment, taking the 100th frame as an example, the set of detection boxes after non-maximum suppression is... ={d1, d2,…, d 15 There are a total of 15 detection boxes. In frame 99, 13 tracking trajectories have been established, with IDs of {001, 002, 003, ..., 013}. Taking target ID=001 as an example, its posterior state vector and posterior covariance matrix in frame 99 are as follows: X 99|99 =[315, 445, 58, 178, 5.2, 5.8, 0.1, 0.2] T P 99|99 = Prediction is performed using the state transition matrix F to obtain the predicted state vector X. 100|99 , where Δt=1: X 100|99 =[320.2, 450.8, 58.1, 178.2, 5.2, 5.8, 0.1, 0.2] T Prior covariance matrix P 100|99 The calculation formula is: The process noise covariance matrix Q is: Q=diag([0.5, 0.5, 0.2, 0.2, 1.0, 1.0, 0.1, 0.1]) The calculation yields: P 100|99 = Then, the association cost matrix between the detection box and the predicted tracking target is constructed. The specific steps are as follows: (1) Calculate the Mahalanobis distance of motion similarity. : Among them, Z jLet R be the observation vector of the j-th detection box, R be the observation noise covariance matrix, and H be a 4×8 observation matrix used to extract observable 4-dimensional position information (x, y, w, h) from the 8-dimensional state vector. Its specific form is as follows: H= In each row, only the corresponding state component position is 1, and the rest are 0. Rows 1-4 extract x, y, w, and h respectively, while the velocity component ( , , , Since it cannot be directly observed, all corresponding columns are 0.

[0040] In this embodiment, the observation vector Z1 of the detection box d1 is [320, 450, 60, 180]. T Observed and predicted value HX 100|99 = [320.2, 450.8, 58.1, 178.2] T Mahalanobis distance ≈ 1.16; (2) Calculate appearance similarity (using cosine similarity) : Among them, f i and f j Let represent the appearance feature vector of the i-th tracked target and the appearance feature vector extracted by the ReID network for the j-th detection box, respectively. In this embodiment, a 128-dimensional feature vector is extracted using a ReID network. The feature vector f for tracking target ID=001 is... 001 = [0.12, 0.34, -0.15, ..., 0.28], and the feature vector f1=[0.15, 0.32, -0.18, ..., 0.26] of the detection box d1, both have been normalized. The appearance similarity is calculated. = 0.11; (3) Construct the comprehensive cost matrix C i,j : Where 𝕀(•) is the indicator function, which is used when the square of the Mahalanobis distance is less than the threshold of the chi-square distribution at a given confidence level. The value is 1 if the condition is met, and 0 otherwise. λ d and λ a Let λ be the weighting coefficient, satisfying λ d +λ a =1, and in this embodiment, the value is λ. d =0.7, λ a =0.3; In this embodiment, the weighting coefficient λ d =0.7, λ a =0.3, chi-square distribution threshold =9.488.

[0041] Finally, the Hungarian algorithm is used for optimal matching and to manage the tracking trajectory. The specific steps are as follows: (1) Use the Hungarian algorithm to perform optimal matching on the cost matrix C to obtain the set of matching pairs M={(i,j)} In this embodiment, the number of tracking trajectories established in the 100th frame is 13, and the number of detection boxes is 15, resulting in a matching pair set M = {(001, d1), (002, d2), ..., (012, d...}. 12 A total of 12 pairs were successfully matched. (2) For a successfully matched tracking target-detection pair (i,j), use the detection box Z j The state estimate X is corrected by updating the equation using Kalman filtering. t|t-1 and P t|t-1 The optimal estimate X is obtained. t|t and P t|t ; (3) For unmatched detection boxes, initialize a new tracker and assign a unique identifier ID; In this embodiment, the detection box d 13 Corresponding new pedestrian ID=014; (4) For a mismatched tracker, if the number of consecutive lost frames exceeds a threshold, it shall be removed from the tracking list; In this embodiment, the threshold for consecutive frame loss is set to 30 frames. After 300 seconds of tracking processing, a total of 135 valid pedestrian trajectory segments and 48 vehicle trajectory segments are finally output.

[0042] The final output is a set of trajectory segments for all pedestrians and vehicles. Each trajectory segment T id ={(x1, y1, w1, h1), (x2, y2, w2, h2), …} contains the timestamps and bounding box sequences of pedestrians and vehicles with ID id in consecutive frames. For example, the trajectory segment data of trajectory ID=001 is: {(315, 445, 58, 178), (320, 450, 60,180), (325, 455, 59, 179), ..., (785, 680, 61, 182)}.

[0043] Step S2 includes the following sub-steps: Step S201: Encode the trajectory segment with spatiotemporal features to generate a structured feature representation that includes a time decay factor and spatial correlation weights; In step S201, the specific steps for spatiotemporal calibration and feature encoding of the trajectory segment are as follows: First, calculate the velocity vector and spatial correlation degree to generate an m-dimensional spatial encoding vector. ; Wherein, velocity vector The calculation formula is as follows, calculated by the position difference between adjacent time points: = Where, p k Let p be the position coordinates of the object at time k. k-1 Let be the position coordinates of the object at time k-1. The time interval between two consecutive frames; Spatial correlation of the k-th segment of the trajectory This indicates the intensity of interaction between the trajectory segment and its surrounding environment. Defined as: Where σ(•) is the Sigmoid activation function, W s Here, φ(•) is the spatial weight matrix, and φ(•) is the environmental feature extraction function, which maps the scene grid map into feature vectors; b s is the spatial bias vector, and ⊕ is the vector concatenation operation. Let be the Euclidean norm of the vector; Space Vector Encoding Composed of the spatial correlation of all time steps, it is expressed as: In this embodiment, trajectory segment ID=005 is selected as an example, and the observation time window length is set to m=5, with a sampling interval of =0.2 seconds. Taking k=3 as an example, the object's position coordinates p3=(12.0, 5.0), and the position coordinates p2=(11.8, 4.9) at k=2. The calculated velocity vector V3=(1.0, 0.5) has a magnitude of... ≈1.12. The feature vector extracted by the environmental feature extraction function φ(grid) is [0.8, 0.1, 0.5, 0.9]. The spatial correlation at this moment is calculated. =0.75; the final generated 5D spatial encoding vector is E s =[0.65, 0.68, 0.75, 0.72, 0.70] T .

[0044] Then, the time decay factor is calculated to generate an m-dimensional time-coded vector. ; Time decay factor of the Kth segment of the trajectory It can be represented as: Wherein, λ is the time decay coefficient, and its value ranges from [0.01, 0.1]. The time interval between two consecutive frames, conf K ε is the average confidence score of all detection boxes within the Kth segment of the trajectory, where ε is a very small positive number used to prevent numerical errors in log(0); Time-coded vector Composed of the time decay factors for all time steps, it is expressed as: In this embodiment, the time decay coefficient λ is set to 0.1, and the minimum positive number ε is set to 0.001. Taking time k=3 as an example, the average confidence of the detection box at this time is conf3=0.92. Substituting these values ​​into the formula, the time decay factor is calculated. ≈0.98×(1 / 0.92)≈1.06; the final generated 5-dimensional time-coded vector is E. t =[1.02, 1.05, 1.06, 1.08, 1.10] T .

[0045] Finally, structured trajectory features The generation operation is represented as: Among them, Γ k For the trajectory segment at time k, LSTM(•) is a Long Short-Term Memory network encoder that takes the trajectory coordinate sequence as input and outputs the hidden state features; ⊙ is an element-wise multiplication operation.

[0046] In this embodiment, the hidden layer dimension of the LSTM encoder is 64. For time k=3, the comprehensive weights are calculated. =0.75×1.06≈0.795. This weight is applied to the feature vector output by the LSTM, and the features from m=5 time steps are accumulated, ultimately outputting a 64-dimensional structured trajectory feature. This is used for subsequent attention mechanism calculations.

[0047] Step S202, in structured features Based on this, motion pattern compensation and anomaly correction are performed; In step S202, the specific steps for performing motion pattern compensation and anomaly correction are as follows: First, calculate the attention weight vector A based on motion features. v This is used for feature compensation. Weight vector A v The calculation formula is as follows: Where d represents the structured feature. The dimension, W v W is the trajectory feature transformation weight matrix. a Transform the weight matrix for acceleration features. Let k be the acceleration vector at time k, and softmax(•) be the normalized exponential function; Wherein, acceleration vector The calculation formula is as follows: = Among them, v k Let v be the magnitude of the object's velocity at time k. k-1 Let $\mathbf{k}$ be the magnitude of the object's velocity at time $k-1$. The time interval between two consecutive frames; In this embodiment, the feature dimension d = 64. Taking time k = 3 as an example, the velocity at this time v3 = 1.12 m / s, the velocity at the previous time k = 2 v2 = 1.08 m / s, and the time interval Δt = 0.2 s. The calculated magnitude of the acceleration is |a3| = 0.2 m / s². 2 Substituting the acceleration vector into the attention formula and performing Softmax normalization, we obtain the attention weight vector A at that moment. v Its maximum component value is 0.35, which corresponds to the feature dimension of a pedestrian's sudden acceleration.

[0048] Then, perform the feature compensation correction operation: in, For the compensated feature vector, W c For the feature compensation weight matrix, For spatial encoding vectors, This is a tensor outer product operation; In this embodiment, the calculated weight A is used v With spatial encoding vector An outer product operation is performed to capture the correlation between sudden motion changes and the environment. The result is then processed by a 64×32 weight matrix W. c After transformation, the original feature vectors are superimposed. .

[0049] Finally, output the compensated structured pedestrian trajectory sequence T: T= Where N is the length of the trajectory sequence.

[0050] Step S3 includes the following sub-steps: In this embodiment, the specific architecture and computational details of the spatiotemporal dual-stream attention network are as follows: Figure 3 As shown; Step S301: Construct a time attention branch to capture motion trend features; In step S301, the specific steps for spatiotemporal calibration and feature encoding of the trajectory segment are as follows: First, the structured pedestrian trajectory sequence T output in step S2 is... As input, each of Let N represent the trajectory feature vector at time t, and N be the number of observation time steps. A multi-head self-attention mechanism is applied in the time dimension to calculate the dynamic dependency weights on the historical sequence. For the h-th attention head, the query vector is calculated. Key vector Sum value vector : in, , , This is the learnable weight matrix for the time attention mechanism.

[0051] In this embodiment, the observation time steps are set to N=8, and the feature dimension d=64. The input sequence T is an 8×64 matrix. The number of attention heads is set to H=4. For the first attention head (h=1), the weight matrix... The dimension is 64×16. Multiplying the input T by the weight matrix yields the query vector sequence of the first head. Its dimensions are 8×16. Similarly, the key vector sequence is calculated. Sum value vector sequence .

[0052] Then, the attention weight matrix in the time dimension is calculated. : =Softmax( ) Where Softmax(•) is the normalization exponential function, d is a scaling factor used to stabilize the gradient; k Let d be the dimension of the key vector. k The calculation formula is: d k = Where d is the total feature dimension and H is the number of attention heads; In this embodiment, the total feature dimension d=64, the number of heads H=4, and d is calculated. k =16, scaling factor =4. Taking the calculation of the dependency between frame 8 (current moment) and frame 4 (historical moment) as an example: the query vector of frame 8 Key vector of frame 4 Performing a dot product operation yields an original score of 12.4. Dividing this by a scaling factor of 4 gives 3.1. After Softmax normalization, the attention weights are obtained. =0.35. This relatively high weight value indicates that the motion state change that occurred in the 4th frame has significant guiding significance for the prediction of the current 8th frame.

[0053] Finally, the temporal attention output of the h-th head is calculated. And then splice and project the outputs of all H heads: in, To output the projection matrix, Concat(•) is the concatenation operation; In this embodiment, the outputs of the four attention heads are calculated separately, with each head having an output dimension of 8×16. After performing the concatenation operation Concat(•), an 8×64 matrix is ​​obtained. This matrix is ​​then projected onto the output matrix, which has a dimension of 64×64. Multiplying these together yields the final output of the time-attention branch. (8×64 dimensions). This output vector aggregates key action features from historical time steps for subsequent spatiotemporal feature fusion.

[0054] Step S302: Construct a temporal attention branch to quantify the spatial interaction relationships between pedestrians; In step S302, the specific steps for constructing the spatial attention branch are as follows: First, suppose there are Nped pedestrians in the scene, and their features at time t are used to define { , , …, As input, calculate the spatial query vector for each pedestrian at that time step. Spatial key vector and spatial value vector : = = = Among them, the set { , , …, This represents the feature set of all pedestrians at time step t. , , These are the learnable weight matrices corresponding to the query, key, and value in the spatial attention mechanism.

[0055] In this embodiment, there are Nped=10 pedestrians in the scene. The input feature vector is defined. The feature dimension d=64. To maintain consistent feature dimensions, a learnable weight matrix is ​​set. , , The dimensions are all d×d (i.e., 64×64). Taking pedestrian ID=001 as an example, its 64-dimensional feature vector... Input, with a 64×64 weight matrix Multiply to obtain its spatial query vector. The dimension remains 64. Similarly, we obtain... and .

[0056] Then, the spatial interaction attention weight e between pedestrian i and pedestrian j is calculated. i,j The spatial attention weights a are then normalized to obtain the standardized spatial attention weights a. i,j : ) in, and Let represent the query vector of pedestrian i and the key vector of pedestrian j at time step t, respectively. The attention weights are calculated as a matrix, with parameters shared among all pedestrians; LeakyReLU(•) is the activation function, and exp(•) is the exponential function.

[0057] In this embodiment, the attention weight calculation matrix is ​​also set. The dimension is 64×64. Using the target pedestrian ID=001 as the query party, the interaction weights between it and all other pedestrians are calculated. When pedestrian ID=003 is within 1.5 meters of ID=001, the calculated raw score is higher due to the high correlation of spatial features. The spatial interaction attention weights are relatively large, and after Softmax normalization, they are... =0.45; while when pedestrian ID=010 is 8 meters away, its interaction weight is... =0.01. This indicates that the mechanism can accurately distinguish and quantify the strong influence of nearby neighbors on the target trajectory.

[0058] Finally, based on the spatial attention weighted aggregate value vector, the updated features of pedestrian i at time step t are obtained. The spatial features of each time step are concatenated in chronological order to obtain the output feature O of the spatial attention branch. s : in, Let j be the value vector of pedestrian j at time step t. is the transformation matrix of the value vector, Concat(•) is the concatenation operation, LayerNorm(•) is the layer normalization operation, and Tobs is the total number of time steps in the observation sequence.

[0059] In this embodiment, the total number of time steps T in the observation sequence is set. obs =8. For pedestrian ID=001 at time t, aggregate the weighted vectors of all pedestrians j that it receives to generate a 64-dimensional update feature. Finally, the update features for the eight time points from t=1 to t=8 will be used. The data is concatenated and then processed through layer normalization to obtain the final output of the spatial attention branch. (8×64 dimensions) This output accurately encodes the dynamic spatial interaction relationships between pedestrians.

[0060] Step S303: Fuse spatiotemporal attention features. Output O from the temporal branch. t With spatial branch output O s The features are fused to generate the final spatiotemporal context feature F. st ; In step S303, the specific steps of spatiotemporal feature fusion are as follows: Output O to the time branch t With spatial branch output O s The features are fused to generate the final spatiotemporal context feature F. st : F st =LayerNorm(O t +O s ) LayerNorm(•) is the layer normalization operation.

[0061] In this embodiment, the time branch output is O t The dimension is 8×64 (8 time steps, 64-dimensional features), and the spatial branch output is O. s The dimension is also 8×64. First, for O... t and O s Element-wise addition is performed, followed by layer normalization to stabilize the feature distribution, ultimately yielding the 64-dimensional spatiotemporal context feature F for the next step. st This feature integrates pedestrian movement trends and complex social interaction information.

[0062] Step S4 includes the following sub-steps: In this embodiment, the parallel computation and fusion mapping process of the social force interaction model is as follows: Figure 4 As shown; Step S401: Construct a pedestrian-vehicle interaction force model. Based on the structured trajectory features T output in step S2 and the vehicle's historical trajectory, calculate the avoidance force exerted by vehicle v on pedestrian i. ; In step S401, the specific steps for constructing the pedestrian-vehicle interaction force model are as follows: Based on the structured trajectory feature T output in step S2 and the vehicle's historical trajectory, calculate the avoidance force exerted by vehicle v on pedestrian i. : Among them, A pv B is the strength coefficient of the force. pv The attenuation coefficient for effective distance has a range of values: A pv ∈[1,10], B pv ∈[0.5,2.0], can be learned through empirical settings or data-driven learning; d i,v The Euclidean distance between the pedestrian center point and the vehicle center point is calculated after converting the pixel coordinates to world coordinates through camera calibration. Let be the unit direction vector pointing from vehicle v to pedestrian i.

[0063] In this embodiment, the strength coefficient A is set. pv =5.0, B pv =1.0. Taking pedestrian ID=005 as an example, the detected distance d between the pedestrian and vehicle ID=002 is 1.0. i,v =3.0 meters. The unit direction vector from the vehicle to the pedestrian is =(-1.0, 0.0). Substituting the above values ​​into the formula, the vehicle avoidance force vector is calculated. ≈(-0.25, 0.0)N.

[0064] Step S402: Construct a pedestrian-environment interaction force model. Using the gridded representation φ of the environmental scene, calculate the repulsive force exerted on pedestrian i by the static environmental obstacle o. : In step S402, the specific steps for constructing the pedestrian-environment interaction force model are as follows: Using the gridded representation φ of the environmental scene, the repulsive force exerted on pedestrian i by the static environmental obstacle o is calculated. : Among them, A pe and B pe For environmental force parameters, d i,o The distance between the pedestrian and the obstacle. Let be the unit direction vector from the obstacle to the pedestrian.

[0065] In this embodiment, A is set. pe =3.0, B pe =0.6. The closest distance d between pedestrian ID=005 and the roadside guardrail obstacle was detected. i,o =0.7 meters. The calculated unit direction vector from the obstacle to the pedestrian is... =(0.0, 1.0). Substituting the above values ​​into the formula, the environmental repulsive force vector is calculated. ≈(0.0, 0.93)N.

[0066] Step S403: Construct a target attraction model. Pedestrians typically move towards their target location; calculate the attraction force exerted on them in the target direction. : In step S403, the specific steps for constructing the target attraction model are as follows: Pedestrians typically move towards their target location; the attractive force they experience in the direction of the target is calculated. : in, Let be the expected speed of pedestrian i. Let be the unit vector of the expected direction of movement of pedestrian i, pointing to the target position; τ is the current actual velocity vector. i is the reaction time constant.

[0067] In this embodiment, a desired speed is set. =1.5m / s, reaction time τ i =0.5s. The system detected the pedestrian's expected direction of motion unit vector as... =(0.0, 1.0), the current actual velocity vector is =(0.1, 0.8) m / s. Substituting these values ​​into the formula, we obtain the target attraction vector. ≈(-0.2, 1.4)N.

[0068] Step S404: Construct a pedestrian-pedestrian interaction force model. Pedestrian i experiences social repulsion from other pedestrians j. This repulsive force is used to maintain social distancing and avoid collisions: In step S404, the specific steps for constructing the pedestrian-pedestrian interaction force model are as follows: Calculate the social exclusion force exerted on pedestrian i by other pedestrian j. This repulsive force is used to maintain social distancing and avoid collisions: Among them, A pp B is the strength coefficient of the repulsive force between pedestrians. pp d is the distance attenuation coefficient between pedestrians. i,j Let be the Euclidean distance between the center points of pedestrian i and pedestrian j. Let be the unit direction vector pointing from pedestrian j to pedestrian i.

[0069] In this embodiment, A is set. pp =2.1, B pp =0.3. The distance d between pedestrian ID=005 and neighboring pedestrian ID=008 was detected. i,j =0.6 meters. The calculated unit direction vector from a nearby pedestrian to the target pedestrian is... =(1.0, 0.0). Substituting the above values ​​into the formula, the social repulsion vector is calculated. ≈(0.28, 0.0)N.

[0070] Step S405: Calculation of Resultant Force and Quantification of Impact. The total social force experienced by pedestrian i. The vector sum of target attraction, pedestrian repulsion, vehicle avoidance force, and environmental obstacle force: In step S405, the specific steps for calculating the resultant force and quantifying the impact are as follows: Total social force on pedestrian i The vector sum of target attraction, pedestrian repulsion, vehicle avoidance force, and environmental obstacle force: Map the magnitude of the social force vector to the scalar interaction strength I. i ∈[0,1], used for subsequent trajectory decoding: Where σ(•) is the Sigmoid function, w I and b I The weights and bias parameters of the linear mapping are learned end-to-end for trajectory prediction tasks.

[0071] In this embodiment, the above-mentioned component force vectors are superimposed to obtain the total social force vector. ≈(-0.17, 2.33)N, its modulus is ≈2.336N. Set the mapping parameter w. I =1.0, b I =0, the scalar interaction strength I is calculated. 005 ≈0.91.

[0072] Step S5 includes the following sub-steps: Step S501: The spatiotemporal context features generated in step S3 are... The interaction impact level I quantized in step S4 is fused to initialize an independent LSTM decoder for each pedestrian, generating their respective initial hidden state h0 and cell state c0: In step S501, the specific steps for initializing the LSTM decoder are as follows: First, the feature vector The fused feature vector is obtained by concatenating it with the scalar influence level I. : Then, the initial hidden state h0 of the decoder is calculated through a fully connected layer and a ReLU activation function: Among them, W h Let b be the weight matrix of the hidden state. h This is the corresponding bias vector; Finally, the initial cell state c0 of the decoder is calculated: Among them, W c Let b be the weight matrix of the cell states. cThis is the corresponding bias vector.

[0073] In this embodiment, the spatiotemporal context feature F output in step S3 st The dimension is 64, and the interaction influence level I quantized in step S4 is a 1-dimensional scalar. The resulting fused feature vector is... The dimension is 65. Set the hidden layer dimension d of the LSTM decoder. h =64. Correspondingly, the weight matrix W h and W c The dimensions of each element are 64×65. Taking pedestrian ID=005 as an example, its fused features are input into the fully connected layer. The calculated initial hidden state h0 and initial cell state c0 are both 64-dimensional vectors, which serve as the starting state of the decoder.

[0074] Step S502: Use an LSTM decoder to perform autoregressive trajectory prediction, and generate multimodal trajectories by injecting random noise. In step S502, the specific steps for performing autoregressive trajectory prediction using the LSTM decoder are as follows: First, set the time window for trajectory prediction to T. pred Step. In each step t (t=1, 2, ...,T) of the decoding process. pred ), and the coordinates predicted in the previous step (Using the last observed coordinates at t=1) and the sampled Gaussian noise ɛ t Perform concatenation, and use it as the input for the current step: (h t , c t )=LSTM([ ], h t-1 , c t-1 ) Among them, ɛ t ~N(0, Σ) represents a noise vector sampled from a multivariate Gaussian distribution, where Σ is the noise covariance matrix. This method uses a diagonal covariance matrix. Among them, the adjustable The noise standard deviation parameter; To output the weight matrix, For bias, dh is the dimension of the LSTM hidden layer, and the output is... The predicted two-dimensional coordinates; Then, by independently sampling K groups of noise sequences from a Gaussian distribution { }, { }, …, { (Each group contains Tpred time steps of noise), which can generate K different future trajectories.}, { }, …, { Each trajectory corresponds to a future motion pattern, and the rationality score of each trajectory can be calculated based on scene constraints or historical patterns.

[0075] In this embodiment, a prediction time window T is set. pred =12, setting the noise standard deviation parameter. Output weight matrix W out The dimension is 2×64. Taking pedestrian ID=005 as an example, the number of sampling groups K=20. The system independently samples 20 noise sequences and decodes them to generate 20 candidate trajectories. Calculations show that 13 of these trajectories indicate that the pedestrian will maintain the current speed when crossing the intersection, while 7 trajectories show that the pedestrian decelerates at the end of the trajectory to avoid vehicles. This set of multimodal trajectories covers the main possible movement patterns of the pedestrian under the current high interaction intensity.

[0076] Step S6 includes the following sub-steps: In this embodiment, the principle of generating traffic heatmaps based on predicted trajectories and its visualization application in intelligent traffic monitoring systems are described as follows: Figure 5 and Figure 6 As shown; Step S601: Divide the prediction area into a uniform grid and calculate the future time window. The density of trajectory points in each grid cell is used to generate a traffic flow heatmap H. flow : In step S601, the specific steps for generating the traffic flow heatmap are as follows: The prediction region is divided into a uniform grid, and future time windows are statistically analyzed. The density of trajectory points in each grid cell is used to generate a traffic flow heatmap H. flow : H flow (x,y)= in, The Gaussian kernel function; w K N is the normalized weight for the Kth trajectory, calculated based on the consistency between this trajectory and historical movement patterns; ped Let N be the number of pedestrians in the scene. traj The number of candidate trajectories generated for each pedestrian. Let be the predicted position of the Kth candidate trajectory of pedestrian i at time t.

[0077] In this embodiment, the intersection prediction area is divided into a uniform 50×50 grid, with each grid cell measuring 0.5m×0.5m. The number of pedestrians N in the scene is... ped =15, N are generated for each pedestrian.traj =20 candidate trajectories. Future time window =12 frames. All trajectory points were mapped to the grid. The cumulative density value of the central area of ​​the zebra crossing (corresponding to the pixel coordinate range [200:300,150:450]) reached 0.92. The heat map showed a dark red color in this area, which intuitively reflected the risk of high-density crowd gathering in this area in the future.

[0078] Step S602: When the flow rate value of a certain area in the heat map exceeds the preset threshold τ con At that time, the system automatically generates a congestion warning message and sends it to the management and control platform, with a threshold τ. con The system dynamically adjusts traffic flow based on historical traffic statistics. Simultaneously, it generates traffic light timing optimization suggestion instructions (C) based on predicted traffic distribution. sign : In step S602, the specific operational steps for congestion warning and traffic light optimization are as follows: When the flow rate value of a certain area in the heat map exceeds the preset threshold τ con At that time, the system automatically generates a congestion warning message and sends it to the management and control platform, with a threshold τ. con The system dynamically adjusts traffic flow based on historical traffic statistics. Simultaneously, it generates traffic light timing optimization suggestion instructions (C) based on predicted traffic distribution. sign : C sign = Among them, H flow (Region j ) represents the flow density of the region corresponding to the j-th direction, ∑H flow τ represents the total flow density across all regions of the entire heatmap. priority This is a traffic share threshold; when the traffic share in a certain direction exceeds this value, the green light duration for that direction is extended to improve traffic efficiency; it is used to determine when to initiate priority passage logic, L j For the lane marking in the j-th direction, To send instructions to the traffic signal controller to extend lane L j The green light duration Δt seconds is implemented through the standard traffic control protocol (GB / T 20999); Default_Cycle maintains the default signal timing scheme.

[0079] In this embodiment, a congestion warning threshold τ is set. con =0.8. The system detected a current traffic density of 0.92 in the zebra crossing area, exceeding the preset threshold, and automatically generated a warning message. Simultaneously, a traffic percentage threshold τ was set. priority=0.60. The system calculates that the current pedestrian crossing area's traffic flow accounts for 0.68% of the total traffic flow, meeting the priority release condition. The system generates traffic light optimization instruction C. sign The instruction is: Increase_Green_Time(Lane_Pedestrian,5), which suggests extending the green light duration for pedestrian crossings by 5 seconds. The instruction is encapsulated and sent to the traffic signal controller in the GB / T 20999 protocol format.

[0080] Step S603, H flow Warning signals and C sign The data is encapsulated into data packets according to the DATEX II traffic data exchange standard and uploaded to the traffic management cloud platform in real time via a wireless communication module (such as a 4G / 5G communication module). The update frequency is once per second, which is used for macro-level traffic monitoring and real-time decision support.

[0081] In step S603, the specific steps for uploading data to the cloud platform are as follows: H flow Warning signals and C sign The data is encapsulated into data packets according to the DATEX II traffic data exchange standard and uploaded to the traffic management cloud platform in real time via a 4G / 5G wireless communication module. In this embodiment, the update frequency is once per second, which is used for macro-level traffic monitoring and real-time decision support.

[0082] In this embodiment, the system encapsulates the 50×50 heat map matrix data generated in step S601, the early warning message generated in step S602, and the control commands into a data frame conforming to the DATEX II standard. Through the 5G communication module integrated in the roadside unit (RSU), the data is uploaded to the city-level intelligent transportation cloud control platform with low latency at a 1-second interval for real-time dispatching and historical data analysis by the command center.

[0083] In this embodiment, to address the issues of significant multimodal characteristics of pedestrian movement trajectories and difficulty in quantifying social interaction relationships at complex traffic intersections, an intelligent traffic control method based on moving object location prediction is proposed. Figure 2 The data flow process for target detection and tracking in this embodiment of the invention is demonstrated, including key steps such as YOLOv8 detection, DeepSORT tracking, and Kalman filter prediction. Figure 3 It is a spatiotemporal dual-stream attention network architecture that effectively decouples and captures pedestrian movement trends and social interaction features through independent temporal and spatial branches; Figure 4 Based on the principle of social force interaction calculation model, it realizes explicit modeling and fusion of physical constraints such as target attraction, pedestrian repulsion force, vehicle avoidance force and environmental obstacle force; Figure 5The process of generating traffic flow heatmaps at intersections is visually demonstrated, showing the entire process from predicting trajectory point distribution to calculating Gaussian kernel density and generating the traffic matrix. Figure 6 The diagram shows the interface of a traffic intelligent monitoring system, demonstrating the practical application effects of congestion warning and traffic light optimization commands. This invention significantly improves the accuracy of trajectory prediction and adaptability to emergencies through the deep integration of spatiotemporal attention mechanisms and social force functions, providing a highly interpretable technical path for intelligent management and congestion warning at urban traffic intersections.

[0084] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of these embodiments are merely for the purpose of helping to understand the method and core ideas of this invention. Furthermore, those skilled in the art will recognize that, based on the ideas of this invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this invention. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the essence of this invention, and these modifications and combinations are still within the protection scope of this invention.

Claims

1. An intelligent traffic control method based on moving object location prediction, characterized in that: Includes the following steps: Step S1: Using the YOLOv8 object detection framework and DeepSORT cross-frame tracking algorithm, extract the structured trajectory dataset of moving objects in the video. The moving objects include people and / or vehicles. Each trajectory includes: frame number, target ID, pixel coordinates of the bounding box center point, and size. Step S2: Spatiotemporally encode the historical trajectory to generate a structured representation containing motion features and environmental interaction features, and construct a spatiotemporal sequence input usable by the model; Step S3: Capture motion trend features through the temporal attention branch, and quantify the spatial interaction relationships between moving objects through the spatial attention branch; Step S4: Construct a multi-factor interactive force function model to quantify the repulsive force from surrounding pedestrians, the avoidance force from vehicles, and the environmental obstacle force experienced by the moving object; Step S5: Fuse spatiotemporal attention features and social force constraints to initialize the LSTM decoder; inject Gaussian noise during the decoding process and sample to generate multiple candidate trajectories; Step S6: Convert the predicted trajectory into an intersection traffic flow heatmap. When the traffic flow exceeds the threshold, automatically generate congestion warnings and traffic light control instructions, and upload them to the traffic management cloud platform in real time.

2. The intelligent traffic control method based on moving object location prediction according to claim 1, characterized in that: Step S1 specifically includes: Step S101: Perform object detection based on the YOLOv8 framework. The specific steps are as follows: First, images are extracted frame by frame from the original surveillance video stream to obtain a continuous image sequence I1, I2, …, I T , where T is the total number of video frames; Then, each frame image I t Input a pre-trained YOLOv8 network, where t=1, 2, …, T. After network detection, output a set β of candidate detection boxes for all pedestrians and vehicles in the current frame. t ; Where the bounding box set β t Represented as: β t ={b1, b2, …, b Mbox } Where Mbox represents the number of boxes detected in the current frame; Each bounding box b i Represented as: b i =(x, y, w, h, conf, cls) Where x is the x-coordinate of the center point, y is the y-coordinate of the center point, w is the width of the bounding box, h is the height of the bounding box, conf is the confidence score of the current bounding box, and cls is the target category identifier. Finally, the nonmaximum suppression algorithm is applied to the detection box set β. t The detection boxes are filtered, and the non-overlapping detection boxes with the highest confidence are retained to generate the final optimized detection set. ; Step S102: Perform multi-target tracking using the DeepSORT algorithm. The specific steps are as follows: First, for the detection results of each frame ={d1, d2, …, d Nbox }, using a Kalman filter to predict the state of all established tracking trajectories, obtaining their predicted state vector X and prior covariance matrix P in the current frame; The state vector X is defined as an 8-dimensional vector: X=[x, y, w, h, , , , ] T Where x is the x-coordinate of the center point, y is the y-coordinate of the center point, w is the width of the bounding box, and h is the height of the bounding box. For lateral velocity, For longitudinal velocity, The width change rate, The superscript indicates the first derivative with respect to time; State prediction calculations are based on the assumption of uniform motion, where the rates of change of position and size remain constant, and are performed using the state transition matrix F: Among them, X t|t-1 X is the predicted state vector at time t based on data up to time t-1. t-1|t-1 Let F be the optimal state vector after fusing the observation data at time t-1, and let F be the state transition matrix. The formula for predicting the prior covariance matrix is: Among them, P t|t-1 Let P be the prior covariance matrix. t-1|t-1 Let F be the posterior covariance matrix. T Let Q be the transpose of the state transition matrix, and let Q be the process noise covariance matrix. Then, construct the association cost matrix between the detection box and the predicted tracking target; Finally, the Hungarian algorithm is used for optimal matching and to manage the tracking trajectory.

3. The intelligent traffic control method based on moving object location prediction according to claim 2, characterized in that: In step S101, the specific steps of the nonmaximum suppression algorithm are as follows: (1) Set the cross-union ratio threshold for nonmaximum suppression to T. nms ; (2) If set β t If the bounding box is not empty, select the detection box b with the highest confidence level (conf). max ; Otherwise, terminate the iteration; (3) b max From β t Remove from the set and add to the final output set. middle; (4) Calculate b max With set β t Each remaining detection box b j The Intersection over Union (IoU) is calculated using the following formula: Where Area(•) is the area calculation function, ∩ is the intersection operation, and ∪ is the union operation; (5) From set β t Delete all expressions that satisfy IoU>T nms Detection box b j ; (6) Repeat steps (2) to (5) until set β is reached. t If empty, you get the set of duplicate detection boxes. Represented as: ={d1, d2, …, d Nbox } Where Nbox represents the number of bounding boxes remaining after filtering.

4. The intelligent traffic control method based on moving object location prediction according to claim 2, characterized in that: In step S102, the specific steps for constructing the association cost matrix between the detection box and the predicted tracking target are as follows: (1) Calculate the Mahalanobis distance of motion similarity. : Among them, Z j Let R be the observation vector of the j-th detection box, R be the observation noise covariance matrix, and H be a 4×8 observation matrix used to extract observable 4-dimensional position information (x, y, w, h) from the 8-dimensional state vector. Its specific form is as follows: H= In each row, only the corresponding state component position is 1, and the rest are 0; rows 1-4 extract x, y, w, and h respectively, while the velocity component ( , , , Since it cannot be directly observed, all corresponding columns are 0; (2) Calculate appearance similarity : Among them, f i and f j Let represent the appearance feature vector of the i-th tracked target and the appearance feature vector extracted by the ReID network for the j-th detection box, respectively. (3) Construct the comprehensive cost matrix C i,j : in, When the square of the Mahalanobis distance is less than the threshold of the chi-square distribution at a given confidence level The value is 1 when the time is right, and 0 otherwise; λ d and λ a Let λ be the weighting coefficient, satisfying λ d +λ a =1.

5. The intelligent traffic control method based on moving object location prediction according to claim 2, characterized in that: In step S102, the specific steps for using the Hungarian algorithm to perform optimal matching and manage the tracking trajectory are as follows: (1) Use the Hungarian algorithm to perform optimal matching on the cost matrix C to obtain the matching pair set M={(i,j)}; (2) For a successfully matched tracking target-detection pair (i,j), use the detection box Z j The state estimate X is corrected by updating the equation using Kalman filtering. t|t-1 and P t|t-1 The optimal estimate X is obtained. t|t and P t|t ; (3) For unmatched detection boxes, initialize a new tracker and assign a unique identifier ID; (4) For a mismatched tracker, if the number of consecutive lost frames exceeds a threshold, it shall be removed from the tracking list; The final output is a set of trajectory segments for all pedestrians and vehicles. Each trajectory segment T id ={(x1,y1,w1,h1), (x2,y2,w2,h2), …} contains the timestamps and bounding box sequences of pedestrians and vehicles with ID id in consecutive frames.

6. The intelligent traffic control method based on moving object location prediction according to claim 1, characterized in that: Step S2 specifically includes: Step S201: Perform spatiotemporal feature encoding on the trajectory segment to generate a structured feature representation containing a time decay factor and spatial correlation weights. The specific steps are as follows: First, calculate the velocity vector and spatial correlation degree to generate an m-dimensional spatial encoding vector. ; Wherein, velocity vector The calculation formula is as follows, calculated by the position difference between adjacent time points: = Where, p k Let p be the position coordinates of the object at time k. k-1 Let be the position coordinates of the object at time k-1. The time interval between two consecutive frames; Spatial correlation of the k-th segment of the trajectory This indicates the intensity of interaction between the trajectory segment and its surrounding environment. Defined as: Where σ(•) is the Sigmoid activation function, W s Here, φ(•) is the spatial weight matrix, and φ(•) is the environmental feature extraction function, which maps the scene grid map into feature vectors; b s is the spatial bias vector, and ⊕ is the vector concatenation operation. Let be the Euclidean norm of the vector; Space Vector Encoding Composed of the spatial correlation of all time steps, it is expressed as: Then, the time decay factor is calculated to generate an m-dimensional time-coded vector. ; Time decay factor of the Kth segment of the trajectory Represented as: Wherein, λ is the time decay coefficient, and its value ranges from [0.01, 0.1]. The time interval between two consecutive frames, conf K ε is the average confidence score of all detection boxes within the Kth segment of the trajectory, where ε is a very small positive number used to prevent numerical errors in log(0); Time-coded vector Composed of the time decay factors for all time steps, it is expressed as: Finally, structured trajectory features The generation operation is represented as: Among them, Γ k For the trajectory segment at time k, LSTM(•) is a Long Short-Term Memory network encoder that takes the trajectory coordinate sequence as input and outputs the hidden state features; ⊙ represents the element-wise multiplication operation. Step S202: In structured features Based on this, motion pattern compensation and anomaly correction are performed. The specific steps are as follows: First, calculate the attention weight vector A based on motion features. v Used for feature compensation, weight vector A v The calculation formula is as follows: Where d represents the structured feature. The dimension, W v W is the trajectory feature transformation weight matrix. a Transform the weight matrix for acceleration features. Let k be the acceleration vector at time k, and softmax(•) be the normalized exponential function; Wherein, acceleration vector The calculation formula is as follows: = Among them, v k Let v be the magnitude of the object's velocity at time k. k-1 Let $\mathbf{k}$ be the magnitude of the object's velocity at time $k-1$. The time interval between two consecutive frames; Then, perform the feature compensation correction operation: in, For the compensated feature vector, W c For the feature compensation weight matrix, For spatial encoding vectors, This is a tensor outer product operation; Finally, output the compensated structured pedestrian trajectory sequence T: T= Where N is the length of the trajectory sequence.

7. The intelligent traffic control method based on moving object location prediction according to claim 6, characterized in that: Step S3 specifically includes: Step S301: Construct a temporal attention branch to capture motion trend features. The specific steps are as follows: First, the structured pedestrian trajectory sequence T output in step S2 is... As input, each of The trajectory feature vector represents time t, and N is the number of observation time steps. A multi-head self-attention mechanism is applied in the time dimension to calculate the dynamic dependency weights on the historical sequence. For the h-th attention head, the query vector is calculated. Key vector Sum value vector : in, , , This represents the learnable weight matrix for the temporal attention mechanism. Then, the attention weight matrix in the time dimension is calculated. : =Softmax( ) Where Softmax(•) is the normalization exponential function, d is a scaling factor used to stabilize the gradient; k Let d be the dimension of the key vector. k The calculation formula is: d k = Where d is the total feature dimension and H is the number of attention heads; Finally, the temporal attention output of the h-th head is calculated. And then splice and project the outputs of all H heads: in, To output the projection matrix, Concat(•) is the concatenation operation; Step S302: Construct a spatial attention branch to quantify the spatial interaction relationships between pedestrians. The specific steps are as follows: First, suppose there are Nped pedestrians in the scene, and their features at time t are used to define { , , …, As input, calculate the spatial query vector for each pedestrian at that time step. Spatial key vector and spatial value vector : = = = Among them, the set { , , …, This represents the feature set of all pedestrians at time step t. , , These are the learnable weight matrices corresponding to the query, key, and value in the spatial attention mechanism; Then, the spatial interaction attention weight e between pedestrian i and pedestrian j is calculated. ij The spatial attention weights a are then normalized to obtain the standardized spatial attention weights a. ij : ) in, and Let represent the query vector of pedestrian i and the key vector of pedestrian j at time step t, respectively. The attention weight matrix is ​​calculated, and the parameters are shared among all pedestrians; LeakyReLU(•) is the activation function, and exp(•) is the exponential function; Finally, based on the spatial attention weighted aggregate value vector, the updated features of pedestrian i at time step t are obtained. The spatial features of each time step are concatenated in chronological order to obtain the output feature O of the spatial attention branch. s : in, Let j be the value vector of pedestrian j at time step t. The transformation matrix is ​​the value vector, Concat(•) is the concatenation operation, LayerNorm(•) is the layer normalization operation, and Tobs is the total number of time steps in the observation sequence; Step S303: Fuse spatiotemporal attention features: output O from the time branch t With spatial branch output O s The features are fused to generate the final spatiotemporal context feature F. st : F st = LayerNorm(O t + O s ) LayerNorm(•) is the layer normalization operation.

8. The intelligent traffic control method based on moving object location prediction according to claim 7, characterized in that: Step S4 specifically includes: Step S401: Construct a pedestrian-vehicle interaction force model: Based on the structured trajectory features T and vehicle historical trajectories output in step S2, calculate the avoidance force exerted by vehicle v on pedestrian i. : Among them, A pv B is the strength coefficient of the force. pv The attenuation coefficient for effective distance has a range of values: A pv ∈[1,10], B pv ∈[0.5,2.0], learned through empirical settings or data-driven learning; d i,v The Euclidean distance between the pedestrian center point and the vehicle center point is calculated after converting the pixel coordinates to world coordinates through camera calibration. Let v be the unit direction vector pointing from vehicle v to pedestrian i; Step S402: Construct a pedestrian-environment interaction force model: Utilize the gridded representation φ of the environmental scene to calculate the repulsive force exerted on pedestrian i by the static environmental obstacle o. : Among them, A pe and B pe For environmental force parameters, d i,o The distance between the pedestrian and the obstacle. This is the unit direction vector from the obstacle to the pedestrian; Step S403: Construct a target attraction model: Pedestrians typically move towards their target location; calculate the attraction force exerted on them in the target direction. : in, Let be the expected speed of pedestrian i. Let be the unit vector of the expected direction of movement of pedestrian i, pointing to the target position; τ is the current actual velocity vector. i The reaction time constant; Step S404: Construct a pedestrian-pedestrian interaction force model: Pedestrian i is subjected to social repulsion forces from other pedestrians j. This repulsive force is used to maintain social distancing and avoid collisions: Among them, A pp B is the strength coefficient of the repulsive force between pedestrians. pp d is the distance attenuation coefficient between pedestrians. i,j Let be the Euclidean distance between the center points of pedestrian i and pedestrian j. Let be the unit direction vector pointing from pedestrian j to pedestrian i; Step S405: Calculation of Resultant Force and Quantification of Impact: Total Social Force Experiencing Pedestrian i The vector sum of target attraction, pedestrian repulsion, vehicle avoidance force, and environmental obstacle force: Map the magnitude of the social force vector to the scalar interaction strength I. i ∈[0,1], used for subsequent trajectory decoding: Where σ(•) is the Sigmoid function, w I and b I The weights and bias parameters of the linear mapping are learned end-to-end for trajectory prediction tasks.

9. The intelligent traffic control method based on moving object location prediction according to claim 8, characterized in that: Step S5 specifically includes: Step S501: Apply the spatiotemporal context features generated in step S3 The interaction impact level I quantized in step S4 is fused to initialize an independent LSTM decoder for each pedestrian, generating their respective initial hidden state h0 and cell state c0. The specific operation steps are as follows: First, the feature vector The fused feature vector is obtained by concatenating it with the scalar influence level I. : Then, the initial hidden state h0 of the decoder is calculated through a fully connected layer and a ReLU activation function: Among them, W h Let b be the weight matrix of the hidden state. h This is the corresponding bias vector; Finally, the initial cell state c0 of the decoder is calculated: Among them, W c Let b be the weight matrix of the cell states. c This is the corresponding bias vector; Step S502: Use an LSTM decoder for autoregressive trajectory prediction, and generate multimodal trajectories by injecting random noise. The specific steps are as follows: First, set the time window for trajectory prediction to T. pred Step: In each step t of the decoding process, t=1, 2, ..., T pred The coordinates predicted in the previous step , and the Gaussian noise ɛ obtained from sampling t Perform concatenation, and use it as the input for the current step: (h t , c t )=LSTM([ ], h t-1 , c t-1 ) Among them, ɛ t ~N(0, Σ) represents a noise vector sampled from a multivariate Gaussian distribution, where Σ is the noise covariance matrix. This method uses a diagonal covariance matrix. ,in This is an adjustable noise standard deviation parameter; To output the weight matrix, For bias, dh is the dimension of the LSTM hidden layer, and the output is... The predicted two-dimensional coordinates; Then, by independently sampling K groups of noise sequences from a Gaussian distribution { }, { }, …, { Each group contains Tpred time steps of noise, generating K distinct future trajectories. }, { }, …, { Each trajectory corresponds to a future movement pattern.

10. The intelligent traffic control method based on moving object location prediction according to claim 1, characterized in that: Step S6 specifically includes: Step S601: Divide the prediction region into a uniform grid and calculate the future time window. The density of trajectory points in each grid cell is used to generate a traffic flow heatmap H. flow : H flow (x,y)= in, The Gaussian kernel function; w K The normalized weight for the Kth trajectory is calculated based on the consistency between this trajectory and historical motion patterns; Nped represents the number of pedestrians in the scene, and Ntraj represents the number of candidate trajectories generated for each pedestrian. Let K be the predicted position of the Kth candidate trajectory of pedestrian i at time t; Step S602: When the flow rate value of a certain area in the heat map exceeds the preset threshold τ con At that time, the system automatically generates a congestion warning message and sends it to the management and control platform, with a threshold τ. con Based on historical traffic statistics, the system dynamically adjusts traffic flow. Simultaneously, based on predicted traffic distribution, it generates a traffic light timing optimization suggestion instruction C. sign : C sign = Among them, H flow (Region j ) represents the flow density of the region corresponding to the j-th direction, ∑H flow τ represents the total flow density across all regions of the entire heatmap. priority This is a traffic share threshold; when the traffic share in a certain direction exceeds this value, the green light duration for that direction is extended to improve traffic efficiency; it is used to determine when to initiate priority passage logic, L j For the lane marking in the j-th direction, To send instructions to the traffic signal controller to extend lane L j The green light duration is Δt seconds; Default_Cycle maintains the default traffic light timing scheme. Step S603: H flow Warning signals and C sign The data is packaged into data packets according to traffic data exchange standards and uploaded to the traffic management cloud platform in real time via a wireless communication module.

Images