A roadside monitoring traffic accident prediction method based on scale perception time domain aggregation

Abstract

This invention discloses a roadside monitoring traffic accident prediction method based on scale-aware temporal aggregation. It acquires traffic video data from a roadside monitoring perspective and utilizes an accident prediction model to achieve early accident warnings. The accident prediction model includes a feature extraction module, a spatial feature aggregation module, a risk inference module, and a prediction output module. The feature extraction module is responsible for target detection and feature extraction from the video data. The spatial feature aggregation module uses masking and spatial self-attention mechanisms to fuse and stitch together global and local features. The risk inference module constructs a scale-aware temporal aggregation strategy through multi-head self-attention and multilayer perceptron mechanisms, and outputs an accident risk representation by combining a temporal inference gated recurrent unit. The prediction output module calculates the accident probability through a fully connected prediction head. This invention solves the problem of instability in roadside monitoring traffic accident prediction caused by severe oscillations in target scale and frequent occlusion, thus improving the accuracy of accident prediction.

Description

Technical Field

[0001] This invention relates to the technical field of road traffic safety early warning, and in particular to a method for predicting traffic accidents based on scale-aware temporal aggregation of roadside monitoring. Background Technology

[0002] Fixed roadside surveillance, as the front-end sensing hub of a smart city's transportation system, has become a core infrastructure for proactive traffic safety management and emergency command due to its wide field of view, broad coverage, and continuous operation around the clock. By conducting in-depth real-time analysis of surveillance video streams and predicting potential collision risks, the logic of traffic accident handling can be shifted from a delayed "post-incident passive evidence collection" to real-time "pre-incident proactive prediction," thereby significantly improving road traffic safety and reducing casualties and property damage.

[0003] However, early-stage traffic accident prediction methods from the perspective of roadside monitoring face complex geometric and topological challenges. For example, the images of moving targets at a fixed viewpoint exhibit severe nonlinear scaling oscillations with changes in physical distance, and frequent interaction and occlusion between targets lead to highly sparse and unstable visual semantic features. Furthermore, existing traffic accident prediction methods are primarily designed for the driver's perspective in-vehicle environment, making it difficult to effectively adapt to the extreme geometric changes under the roadside monitoring perspective. This results in weak model capture of key behavioral features during accident evolution, limiting prediction accuracy.

[0004] Therefore, proposing an accident prediction method that can adapt to the observation quality of roadside monitoring videos, and realizing accurate prediction and early warning of traffic accidents from fixed roadside viewpoints, is an urgent need in the field of smart city traffic management. Summary of the Invention

[0005] The purpose of this invention is to solve the problem of unstable frame interference caused by drastic fluctuations in the scale of moving targets and frequent occlusion in road monitoring traffic accident prediction. It proposes a roadside monitoring traffic accident prediction method based on scale-aware temporal aggregation. This method can effectively suppress noise frame interference and effectively improve the accuracy and reliability of traditional accident prediction models. It shows high performance in roadside monitoring traffic accident prediction and is more efficient than previous traffic accident prediction methods.

[0006] To achieve the above objectives, the technical solution provided by this invention is as follows: a roadside monitoring traffic accident prediction method based on scale-aware temporal aggregation. This method utilizes an accident prediction model to achieve early warning of road traffic accidents. The accident prediction model includes a feature extraction module, a spatial feature aggregation module, a risk reasoning module, and a prediction output module. The feature extraction module is responsible for target detection and feature extraction on standardized traffic processing unit videos. It extracts target local features and frame global features from video frames by combining detection boxes and feature maps, and constructs a target local feature set by combining a target masking mechanism. The spatial feature aggregation module is responsible for fusing and stitching the frame global features and the target local feature set, and enhancing the information interaction of target local features between target objects through a spatial self-attention mechanism. A spatial frame-level fusion feature is constructed to achieve spatial feature fusion before risk inference. The risk inference module is responsible for outputting the accident risk representation of the current frame. Relying on the temporal inference gated recurrent unit and the filtering enhancement strategy, the continuous spatial frame-level fusion features are input into the filtering enhancement GRU temporal inference gated recurrent unit to achieve filtering enhancement and iterative inference of the accident risk representation. The filtering enhancement strategy achieves fine-grained scale perception and temporal aggregation of the accident risk representation by fusing a multi-head self-attention mechanism and a multilayer perceptron mechanism, which is called the scale perception temporal aggregation strategy. The prediction output module is responsible for calculating the accident probability score. It performs inference through a classification prediction head with a two-layer fully connected architecture, outputs the accident probability score of the current prediction frame, and compares it with a predetermined accident threshold to complete the triggering of accident prediction.

[0007] The specific implementation of the roadside monitoring traffic accident prediction method includes the following steps:

[0008] 1) Acquire traffic video data from the perspective of road monitoring and preprocess it to obtain standardized traffic processing unit videos with accident information marked and of equal length;

[0009] 2) The standardized traffic processing unit video obtained in step 1) is input into the accident prediction model for training. First, the target local features and frame global features are obtained through the feature extraction module, and then fused and spliced ​​into spatial frame-level fusion features through the spatial feature aggregation module. The spatial frame-level fusion features are iteratively input into the risk inference module to output the accident risk representation of the prediction frame. The accident risk representation of the prediction frame is input into the prediction output module to finally output the accident prediction state of the prediction frame. The accident prediction model is trained by comparing the accident prediction state of the prediction frame output by the model inference with the true value of the labeled accident information of the prediction frame. In the training process, a weighted cross-entropy loss function with a time penalty factor is introduced. The time penalty factor changes exponentially with time, which effectively guides the prediction accuracy of the model during the training process.

[0010] 3) Input the unlabeled video data to be tested into the trained accident prediction model, and the accident probability score of the corresponding frame will be output. The score will be compared with the predetermined accident threshold to complete the accident prediction.

[0011] Further, in step 1), the preprocessing includes: performing fixed-length slice sampling and resolution unification on the video at a set frame rate, marking the start point of the video accident and the accident status of each video frame, performing clone padding repair on samples with insufficient frame count after sampling, and forming standardized traffic processing unit videos with accident information marked and equal time length, as detailed below:

[0012] First, the motion interpolation operator is invoked to uniformly process the traffic video data from the original road monitoring perspective at a target frame rate of 30fps for a length of [length missing]. Fixed-length time-series slicing of frames eliminates feature nonlinearity deviations caused by sampling frequency drift from different monitoring devices. Simultaneously, the video resolution is uniformly mapped to a 1280×720 standard pixel matrix to eliminate the impact of spatial scale differences on subsequent feature extraction modules. Then, the video accident start point and accident status information for each video frame are manually labeled. Finally, to address the issue of insufficient video frames after slicing... The video samples of each frame are used to perform tail frame cloning and filling repair by calling the FFmpeg boundary fill filter tpad, forming a standardized traffic processing unit video with accident information labeled and time of equal length.

[0013] Furthermore, in step 2), the accident start frame with accident information already labeled in the standardized traffic processing unit video is used. and video sampling frame rate Based on this, a time penalty factor that grows exponentially over time is constructed. :

[0014] ;

[0015] In the formula: A sequence of frames after video sampling for a standardized traffic processing unit. The time index of each frame in the middle, where Represents a frame sequence The first in frame, ;

[0016] Using the time penalty factor The final prediction loss of each frame sequence after inference from the accident prediction model is weighted to construct a weighted cross-entropy loss function with a time penalty factor. :

[0017] ;

[0018] In the formula: This is the batch size for model training. The video index number for the standardized traffic processing unit video. For the first The first standardized traffic processing unit video Accident information in frames, The first output of the accident prediction model Frame accident prediction probability;

[0019] Using the above weighted cross-entropy loss function The accident prediction model is trained to achieve strong suppression of delayed warning behavior and significant forward shift of the risk perception threshold. During the training process, the accident prediction model is forced to capture weak early warning features before the accident occurs, thereby effectively guiding the model's prediction accuracy during the training process.

[0020] Furthermore, the feature extraction module performs the following operations:

[0021] First, for each frame A high-order feature map is generated by capturing global features of frames through a backbone network based on the VGG-16 architecture. ;

[0022] Then, the target detection and local feature extraction process for each target object in the video frame is executed: the trained Cascade R-CNN cascaded regression detection model is used to detect targets in the video frame and generate a set of bounding boxes for the target objects. , ,in For the first The first video frame The bounding box of a target object. For the first The total number of target objects corresponding to the nth video frame; for the nth video frame For each target object in each video frame, in the high-order feature map Perform RoI alignment and pooling to obtain standard target local features. ,in For the first The first video frame Local features of a target object.

[0023] To avoid inconsistencies in the target local feature set tensor caused by fluctuations in the number of target objects, the total number of target objects... This will perform truncation or padding, set... The maximum number of target object slots is And construct an effective target mask set. , To indicate the real target object and the filled noise, where , For slot pointers, Indicates the first Within the first video frame The target mask of a target object;

[0024] Ultimately, the local features of each target The features are mapped to a high-dimensional continuous feature space through a shared linear projection layer, resulting in the... The first video frame Target local embedding features of a target object and construct the first Set of target local embedding features of video frames Simultaneously extract the high-order feature map of the frame. Corresponding global embedding features .

[0025] Furthermore, the spatial feature aggregation module performs the following operations:

[0026] First, a spatial self-attention mechanism is used to interactively model and fuse the global features and target local feature sets of the corresponding video frames, while quantifying the semantic association weights between target local features; the target local embedding feature set... As input, a spatial self-attention mechanism is used to enhance feature interactions, and an effective target mask set is also incorporated. Attention masking is implemented for invalid slots, and a weighted set of target local enhancement features is obtained. ,in For the first The first video frame Local enhancement features of a target object;

[0027] Subsequently, the global mean pooling operator was combined with the pair Perform compression mapping and output the first... Target local aggregation features of video frames and will With the Global embedding features of each video frame Perform fusion and stitching along the channel dimension to form the first Spatial frame-level fusion features of video frames .

[0028] Furthermore, the specific operation steps of the spatial feature aggregation module are as follows:

[0029] Embedding feature set of target Using the effective target mask set Pre-filtering is performed, followed by a linear mapping using a spatial self-attention matrix to generate the first... Spatial self-attention triplet tensor of a video frame :

[0030] ;

[0031] In the formula, Representing spatial self-attention In the feature space of the query ,key ,value Learnable projective weight matrix in dimensionality The query matrix that carries the original feature information. The key-value matrix that carries the original feature information. The value matrix that carries the original feature information. It represents the Hadamardi (or Hadama) stack;

[0032] Subsequently, the effective target mask set Transform into spatial bias matrix By performing a single dot product attention calculation, weight normalization, noise masking, and feature aggregation are completed, resulting in a set of target local enhancement features. ,in For the first The first video frame Local enhancement features of a target object The calculation process is as follows:

[0033] ;

[0034] In the formula, This is the transpose of the key-value matrix that carries the original feature information; This represents the original spatial correlation score matrix, which measures the strength of the correlation between slots. This is a scaling factor used to stabilize the gradient in the dot product operation; For the first The spatial attention mask matrix for each video frame is constructed according to the following criteria:

[0035] ;

[0036] In the formula, , , For the first Within the first video frame and the Joint target mask for multiple target objects. For the first Within the first video frame The target mask of a target object. For the first Within the first video frame The target mask of a target object;

[0037] Enhance the local feature set of the target using the global mean pooling operator. Perform compression mapping to obtain the target local aggregated features. ;

[0038] Finally, to achieve efficient spatial feature extraction and semantic encapsulation in a single frame, With global embedded features Perform fusion and stitching processing along the channel dimension to form the first Spatial frame-level fusion features of video frames .

[0039] Furthermore, the risk reasoning module performs the following operations:

[0040] First, the continuous spatial frame-level fused feature sequence input from the spatial feature aggregation module is received. It is used as the GRU temporal inference gated loop unit for iterative input filtering of temporal information, and combined with the historical state and hidden state generated inside the GRU temporal inference gated loop unit, the corrected hidden state of each frame is generated iteratively.

[0041] The filtering enhancement strategy, which integrates multi-head self-attention mechanism and multilayer perceptron mechanism, achieves fine-grained scale perception and temporal aggregation of accident risk representation. This strategy is called scale perception temporal aggregation strategy. It captures the hidden state sequence generated inside the GRU temporal reasoning gated loop unit through iterative local time window, and combines scale filtering and dynamic temporal aggregation to output the historical state of the next prediction frame in the GRU temporal reasoning gated loop unit, thereby achieving filtering enhancement and iterative reasoning of accident risk representation.

[0042] The risk reasoning module consists of a GRU temporal reasoning gated loop unit and a scale-aware temporal aggregation strategy. The scale-aware temporal aggregation strategy constructs a multi-head self-attention-based temporal enhancement subunit, a multilayer perceptron-based scale consistency evaluation subunit, a scale-weighted noise suppression subunit, and a temporal attention aggregation subunit within an iterable local time window. This strategy is used to filter and correct the historical state output within the GRU temporal reasoning gated loop unit.

[0043] Furthermore, the process of the scale-aware temporal aggregation strategy is as follows:

[0044] First, continuous spatial frame-level fusion feature sequences The hidden state sequences at the current and historical moments are recursively output through the GRU temporal inference gated loop unit, and then a preset iterable local time window step size is used. Slice the hidden state sequence to construct an iterable local time window set of states. ,in For the first Spatial frame-level fusion features of video frames The expression after an iterable local time window is input, i.e., the first expression in the GRU temporal inference gated recurrent unit. A set of hidden states, which can be used as a local benchmark for temporal quality assessment, will be used to solve the problem of non-stationarity of temporal characteristics caused by abrupt changes or jitter under fixed roadside viewpoints;

[0045] Then, the temporal enhancement subunit mines iterable local time windows through a multi-head self-attention mechanism. Long-range dependencies between frames, utilizing Learnable time query weight matrix for each parallel attention head Time key weight matrix and time value weight matrix , The set of iterable local time window states Projected onto the temporal augmentation space, and then calculated and aggregated. frame, Temporal attention between head features, performing iterable local temporal window state set. Self-correlation modeling in the time domain outputs a set of temporal context state augmented representations that fuse long-range dependencies. ,in For the first Temporal context state augmentation representation of each video frame The calculation process is as follows:

[0046] ;

[0047] ;

[0048] In the formula: For the number of attention heads, For the first The temporal context state enhancement representation of each attention head For the first Feature dimension scaling factor for each attention head For multi-head attention fusion projection matrix, Corresponding to the first Feature query for each attention head ,key ,value The learnable projective weight matrix, For the first A set of iterable local temporal window states for each video frame. It is the transpose symbol;

[0049] Next, the time context state enhancement representation set The input is fed into the scale consistency evaluation subunit, and then processed through a scale evaluation network consisting of multiple fully connected layers. A nonlinear mapping is performed, and the scale stability features of the target in spatiotemporal evolution are extracted using the first and second linear layers. These features are then compressed into a scale consistency score sequence using the third linear layer. This enables precise quantification of temporal and spatial fluctuations and frequent occlusion, among which... For the iterable local time window, the first The frame scale consistency score is calculated using the following logic:

[0050] ;

[0051] ;

[0052] ;

[0053] In the formula: express The Middle Transpose of the temporal context state augmentation representation of each video frame. The first Layer, First Layer and first The network weight matrix of the layer, For the bias terms corresponding to each fully connected layer, respectively handle the first Outputs of the first and second linear layers per video frame; scale consistency score. ,in The aim is to quantitatively characterize the first The reliability of frame observation under roadside monitoring perspective is affected by drastic scale changes or occlusion;

[0054] Next, the scale-weighted noise suppression subunit utilizes the obtained scale consistency score sequence Enhanced representation set of temporal context states Element-level weighted masking is performed to actively suppress the enhanced representation components of low-confidence frames, thereby increasing the proportion of enhanced representations of high-quality precursor information. This results in a denoised state representation sequence with a more robust temporal feature distribution. ,in For the first The denoising state representation of each video frame is calculated based on the following criteria:

[0055] ;

[0056] In the formula: express The Middle Temporal context state augmentation representation of each video frame For Hadamah accumulation;

[0057] Finally, the denoised state representation sequence after weighted filtering Historical aggregation will be achieved through a time-attention aggregation subunit, which introduces an independent original temporal importance evaluation network. By calculating the iterable local time window The original frame pairs The correction contribution of the final historical state of the frame is used to obtain the correction contribution weight set. ,in For the first The video frame is the first The contribution of the final historical state of the frame to the correction, and the denoised state representation sequence based on this weight set. Perform dynamic weighted aggregation and output the first... Frame scale-aware aggregated state representation ;

[0058] The original time-series importance evaluation network Consists of a fully connected layer It consists of a Tanh activation function layer and first utilizes the weight matrix of the temporally fully connected layer. and corresponding bias weights Within the iterable local time window Iterable local time window state set of a frame The elements of the set in the code are mapped to one-dimensional scalars, and the numerical values ​​are mapped to scalars using the Tanh function. Finally, the softmax layer is used to normalize each element along the time dimension, outputting a set of corrected contribution weights. The calculation process is as follows:

[0059] ;

[0060] Subsequently, the modified contribution weight set was used. Denoising state representation sequence Perform a weighted summation and output the first... Frame scale-aware aggregated state representation :

[0061] ;

[0062] The native spatial frame-level fusion features achieve confidence filtering of historically damaged information by explicitly modeling the temporal scale consistency between window frames, and correct and aggregate the scale-aware aggregated state representation. Will be the first to be predicted The historical state of the frame is input into the GRU sequential inference gated loop unit.

[0063] Furthermore, the flow of the GRU sequential inference gated loop unit is as follows:

[0064] First, the spatial feature aggregation module outputs the first prediction value to be predicted. Spatial frame-level fusion features The scale-aware aggregation state representation of the previous frame output by the current scale-aware temporal aggregation strategy. By utilizing the update and reset gates of the GRU sequential reasoning gated loop unit, information filtering and risk evolution reasoning are performed. The internal recursive state update process is expressed as follows:

[0065] ;

[0066] ;

[0067] ;

[0068] In the formula: For the first The update gate vector of a frame in the GRU temporal inference gated recurrent unit. For the first The reset gate vector of a frame in the GRU sequential inference gated loop unit. For the first Candidate hidden states of frames in GRU temporal inference gated recurrent units Indicates the first The joint input vector is formed by concatenating the spatial frame-level fusion features of the frame with the scale-aware aggregated state representation of the previous frame along the channel dimension. This indicates that for the input number... The weight matrix part of the spatial frame-level fusion features of the frame. This represents the aggregated state representation for historical scale perception. The weight matrix part, Indicates the first The weight matrix for calculating the candidate hidden states of a frame. This represents the bias vector between the update gate and the reset gate in the GRU sequential inference gated loop unit;

[0069] The output of the GRU sequential inference gated loop unit is the first... Frame hidden state Synchronous output and recursive input of the first... Iterable local time window state set of a frame In the middle, as the sliding window benchmark for the next prediction period, and at the same time, the first Hidden state of a frame Will be the first The accident risk characterization of the frame will be synchronously input into the downstream prediction output module.

[0070] Furthermore, the prediction output module performs the following operations:

[0071] First, the risk reasoning module outputs the first... Frame hidden state Will be the first The accident risk characterization of a frame is obtained through a classification prediction head with a two-layer fully connected architecture, which ultimately outputs the accident prediction status of the current frame.

[0072] The first fully connected layer performs dimensionality reduction and abstraction of high-dimensional features and utilizes the ReLU activation function to enhance the model's nonlinear representation capability. The second fully connected layer maps the features to a binary classification space, and the output reflects the... Binary logits sequence of frame incident risk :

[0073] ;

[0074] In the formula: These are the weight matrices for the two fully connected layers. Its corresponding bias term;

[0075] The results reflect the first Binary logits sequence of frame incident risk The probability will be normalized through a Softmax layer to calculate the... Accident probability score of a frame :

[0076] ;

[0077] In the formula, Corresponding to the The original score of the non-accident category in the binary classification results of the frame. Corresponding to the The original score of the accident category in the binary classification results of the frame;

[0078] Finally, using predetermined accident thresholds With the Accident probability score of a frame Perform real-time comparison; when the accident probability score exceeds a threshold, the accident prediction is triggered.

[0079] ;

[0080] In the formula, For the first The frame's accident trigger state, when When this happens, a safety warning is triggered.

[0081] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0082] 1. This invention proposes a scale-aware temporal aggregation strategy. This strategy mines long-range temporal dependencies by constructing a multi-head self-attention-based temporal enhancement subunit, and combines it with a multilayer perceptron-based scale consistency evaluation subunit, scale-weighted denoising subunit, and temporal attention aggregation subunit to perform dynamic weighting and denoising processing on the video frames under test. This strategy can significantly suppress observation noise introduced by severe nonlinear oscillations of target scale and frequent occlusion under fixed roadside viewpoints, effectively enhancing the model's ability to capture high-quality accident precursor information, thereby greatly improving the robustness and accuracy of traffic accident prediction in complex traffic scenarios.

[0083] 2. This invention constructs a spatial feature aggregation module, which uses a spatial self-attention mechanism to interactively model the global features and target local feature sets of video frames, and introduces an effective target mask to perform attention masking on invalid slots. This structure can effectively alleviate the interference of fluctuations in the number of multiple targets and background noise on feature extraction, significantly enhance the purity of the model's representation of the semantics of potential conflicts between traffic entities, and provide highly discriminative feature inputs for subsequent temporal inference.

[0084] 3. This invention introduces a weighted cross-entropy loss function that incorporates a time penalty factor. This function uses a penalty factor that increases exponentially with the approaching collision time to weight the classification loss. This mechanism can force the model to discover subtle evolutionary precursors before accidents occur during training, effectively suppressing the common early warning lag in traditional methods. While maintaining high confidence, it significantly advances the risk perception threshold, improving the model's prediction accuracy for traffic accidents. Attached Figure Description

[0085] Figure 1This is an architecture diagram of the method of the present invention. In the diagram, Input Video (N Frames) represents the input standardized traffic processing unit video frames, Frame Feature Extractor represents the backbone network based on the VGG-16 architecture in the feature extraction module, Frame Feature represents the global embedded features in the feature extraction module, Object Detector represents the trained CascadeR-CNN cascaded regression detection model in the feature extraction module, Traffic Agent Regions represents the set of bounding boxes of the target objects, ROI Feature Extractor represents the target local feature extractor that performs RoI alignment and pooling in the feature extraction module, ROI Features represents the target local features, Spatial Attention represents the spatial attention module, Enhanced ROI Features represents the target local enhanced features after passing through the spatial attention module, and FeatureConcatenation represents the target local enhanced features. The symbols represent the spatial feature fusion and splicing operation, Feature Encoder represents the fusion feature encoder that specifically performs the fusion and splicing operation, Frame-level Embeddings are spatial frame-level fusion features, GRU represents the temporal reasoning gated recurrent unit in the risk reasoning module, Hidden State represents the hidden state in the reasoning module, Multi-head Self-Attention represents the multi-head self-attention layer, Multi-scale temporal representation is the temporal context state enhancement representation, Scale Consistency Evaluation represents the scale consistency evaluation unit, Scale Consistency Score is the scale consistency score, Scale-Aware Hidden State is the denoised state representation, Temporal Attention Aggregation represents the temporal attention aggregation unit, scale-weighted temporal features is the scale-aware aggregated state representation, Prediction Head represents the prediction output module, used to trigger the accident state, Accident means the current frame triggers the accident warning, and No Accident means the current frame does not trigger the accident warning.

[0086] Figure 2This is an architecture diagram of a scale-aware temporal aggregation strategy. In the diagram, Multi-Head Self-Attention represents a multi-head self-attention mechanism for capturing context dependencies, Head 1, ..., Head 4 represent attention heads, State t is the current iterable local time window, Multi-head Output is the temporal context state enhancement representation of the multiple attention heads, Multi-scale temporal representation is the set of temporal context state enhancement representations, ScaleConsistency Assessment represents the scale consistency assessment unit, Scale Consistency Score represents the scale consistency score, Scale Score represents the scale consistency score sequence, and Scale-Aware Feature Weighting represents a more robust denoised state representation of temporal feature distribution. The symbol represents the Hadamard product, TemporalAttention Aggregation represents the temporal attention aggregation sub-unit, and Scale-weighted TemporalFeatures represents the scale-aware aggregation state representation of the final output. Detailed Implementation

[0087] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0088] like Figure 1 and Figure 2As shown, this embodiment discloses a roadside monitoring traffic accident prediction method based on scale-aware temporal aggregation. This method utilizes an accident prediction model to achieve early warning of road traffic accidents. The accident prediction model includes a feature extraction module, a spatial feature aggregation module, a risk reasoning module, and a prediction output module. The feature extraction module is responsible for target detection and feature extraction from standardized traffic processing unit videos. It combines detection boxes and feature maps to extract target local features and frame global features from video frames, and constructs a target local feature set using a target masking mechanism. The spatial feature aggregation module is responsible for fusing and stitching the frame global features and target local feature set, enhancing the information interaction of target local features between target objects through a spatial self-attention mechanism to construct a spatial frame. The system integrates spatial features at the spatial frame level to achieve spatial feature fusion before risk inference. The risk inference module is responsible for outputting the accident risk representation of the current frame. Relying on the temporal inference gated loop unit and the filtering enhancement strategy, it inputs the continuous spatial frame-level fused features into the filtering enhancement GRU temporal inference gated loop unit to achieve filtering enhancement and iterative inference of the accident risk representation. The filtering enhancement strategy achieves fine-grained scale perception and temporal aggregation of the accident risk representation by integrating a multi-head self-attention mechanism and a multi-layer perceptron mechanism, which is called the scale perception temporal aggregation strategy. The prediction output module is responsible for calculating the accident probability score. It performs inference through a classification prediction head with a two-layer fully connected architecture, outputs the accident probability score of the current prediction frame, and compares it with a predetermined accident threshold to complete the triggering of accident prediction.

[0089] The specific implementation of the roadside monitoring traffic accident prediction method includes the following steps:

[0090] 1) Acquire traffic video data from a road monitoring perspective and preprocess it to obtain standardized traffic processing unit videos with labeled accident information and equal time length. The preprocessing includes: sampling the video in fixed-length slices at a set frame rate and unifying the resolution; labeling the start point of the video accident and the accident status of each video frame; and performing clone padding repair on samples with insufficient frame counts after sampling to form standardized traffic processing unit videos with labeled accident information and equal time length, as detailed below:

[0091] First, the motion interpolation operator is invoked to uniformly process the traffic video data from the original road monitoring perspective at a target frame rate of 30fps for a length of [length missing]. Fixed-length time-series slicing of frames eliminates feature nonlinearity deviations caused by sampling frequency drift from different monitoring devices. Simultaneously, the video resolution is uniformly mapped to a 1280×720 standard pixel matrix to eliminate the impact of spatial scale differences on subsequent feature extraction modules. Then, the video accident start point and accident status information for each video frame are manually labeled. Finally, to address the issue of insufficient video frames after slicing... The video samples of each frame are used to perform tail frame cloning and filling repair by calling the FFmpeg boundary fill filter tpad, forming a standardized traffic processing unit video with accident information labeled and time of equal length.

[0092] 2) The standardized traffic processing unit video obtained in step 1) is input into the accident prediction model for training. First, the target local features and frame global features are obtained through the feature extraction module, and then fused and spliced ​​into spatial frame-level fusion features through the spatial feature aggregation module. The spatial frame-level fusion features are iteratively input into the risk inference module to output the accident risk representation of the prediction frame. The accident risk representation of the prediction frame is input into the prediction output module to finally output the accident prediction state of the prediction frame. The accident prediction model is trained by comparing the accident prediction state of the prediction frame output by the model inference with the true value of the labeled accident information of the prediction frame. In the training process, a weighted cross-entropy loss function with a time penalty factor is introduced. The time penalty factor changes exponentially with time, which effectively guides the prediction accuracy of the model during the training process.

[0093] Specifically, the accident start frame, which already contains labeled accident information, is used in the video of the standardized traffic processing unit. and video sampling frame rate Based on this, a time penalty factor that grows exponentially over time is constructed. :

[0094] ;

[0095] In the formula: A sequence of frames after video sampling for a standardized traffic processing unit. The time index of each frame in the middle, where Represents a frame sequence The first in frame, ;

[0096] Using the time penalty factor The final prediction loss of each frame sequence after inference from the accident prediction model is weighted to construct a weighted cross-entropy loss function with a time penalty factor. :

[0097] ;

[0098] In the formula: This is the batch size for model training. The video index number for the standardized traffic processing unit video. For the first The first standardized traffic processing unit video Accident information in frames, The first output of the accident prediction model Frame accident prediction probability;

[0099] Using the above weighted cross-entropy loss function The accident prediction model is trained to achieve strong suppression of delayed warning behavior and significant forward shift of the risk perception threshold. During the training process, the accident prediction model is forced to capture weak early warning features before the accident occurs, thereby effectively guiding the model's prediction accuracy during the training process.

[0100] 3) Input the unlabeled video data to be tested into the trained accident prediction model, and the accident probability score of the corresponding frame will be output. The score will be compared with the predetermined accident threshold to complete the accident prediction.

[0101] Specifically, the feature extraction module performs the following operations:

[0102] First, for each frame A high-order feature map is generated by capturing global features of frames through a backbone network based on the VGG-16 architecture. ;

[0103] Then, the target detection and local feature extraction process for each target object in the video frame is executed: the trained Cascade R-CNN cascaded regression detection model is used to detect targets in the video frame and generate a set of bounding boxes for the target objects. , ,in For the first The first video frame The bounding box of a target object. For the first The total number of target objects corresponding to the nth video frame; for the nth video frame For each target object in each video frame, in the high-order feature map Perform RoI alignment and pooling to obtain standard target local features. ,in For the first The first video frame Local features of a target object.

[0104] To avoid inconsistencies in the target local feature set tensor caused by fluctuations in the number of target objects, the total number of target objects... This will perform truncation or padding, set... The maximum number of target object slots is And construct an effective target mask set. , To indicate the real target object and the filled noise, where , For slot pointers, Indicates the first Within the first video frame The target mask of a target object;

[0105] Ultimately, the local features of each target The features are mapped to a high-dimensional continuous feature space through a shared linear projection layer, resulting in the... The first video frame Target local embedding features of a target object and construct the first Set of target local embedding features of video frames Simultaneously extract the high-order feature map of the frame. Corresponding global embedding features .

[0106] Specifically, the spatial feature aggregation module performs the following operations:

[0107] First, a spatial self-attention mechanism is used to interactively model and fuse the global features and target local feature sets of the corresponding video frames, while quantifying the semantic association weights between target local features; the target local embedding feature set... As input, a spatial self-attention mechanism is used to enhance feature interactions, and an effective target mask set is also incorporated. Attention masking is implemented for invalid slots, and a weighted set of target local enhancement features is obtained. ,in For the first The first video frame Local enhancement features of a target object;

[0108] Subsequently, the global mean pooling operator was combined with the pair Perform compression mapping and output the first... Target local aggregation features of video frames and will With the Global embedding features of each video frame Perform fusion and stitching along the channel dimension to form the first Spatial frame-level fusion features of video frames .

[0109] Specifically, the specific steps for spatial feature aggregation are as follows:

[0110] Embedding feature set of target Using the effective target mask set Pre-filtering is performed, followed by a linear mapping using a spatial self-attention matrix to generate the first... Spatial self-attention triplet tensor of a video frame :

[0111] ;

[0112] In the formula, Representing spatial self-attention In the feature space of the query ,key ,value Learnable projective weight matrix in dimensionality The query matrix that carries the original feature information. The key-value matrix that carries the original feature information. The value matrix that carries the original feature information. It represents the Hadamardi (or Hadama) stack;

[0113] Subsequently, the effective target mask set Transform into spatial bias matrix By performing a single dot product attention calculation, weight normalization, noise masking, and feature aggregation are completed, resulting in a set of target local enhancement features. ,in For the first The first video frame Local enhancement features of a target object The calculation process is as follows:

[0114] ;

[0115] In the formula, This is the transpose of the key-value matrix that carries the original feature information; This represents the original spatial correlation score matrix, which measures the strength of the correlation between slots. This is a scaling factor used to stabilize the gradient in the dot product operation; For the first The spatial attention mask matrix for each video frame is constructed according to the following criteria:

[0116] ;

[0117] In the formula, , , For the first Within the first video frame and the Joint target mask for multiple target objects. For the first Within the first video frame The target mask of a target object. For the first Within the first video frame The target mask of a target object;

[0118] Enhance the local feature set of the target using the global mean pooling operator. Perform compression mapping to obtain the target local aggregated features. ;

[0119] Finally, to achieve efficient spatial feature extraction and semantic encapsulation in a single frame, With global embedded features Perform fusion and stitching processing along the channel dimension to form the first Spatial frame-level fusion features of video frames .

[0120] Specifically, the risk reasoning module performs the following operations:

[0121] First, the continuous spatial frame-level fused feature sequence input from the spatial feature aggregation module is received. It is used as the GRU temporal inference gated loop unit for iterative input filtering of temporal information, and combined with the historical state and hidden state generated inside the GRU temporal inference gated loop unit, the corrected hidden state of each frame is generated iteratively.

[0122] The filtering enhancement strategy, which integrates multi-head self-attention mechanism and multilayer perceptron mechanism, achieves fine-grained scale perception and temporal aggregation of accident risk representation. This strategy is called scale perception temporal aggregation strategy. It captures the hidden state sequence generated inside the GRU temporal reasoning gated loop unit through iterative local time window, and combines scale filtering and dynamic temporal aggregation to output the historical state of the next prediction frame in the GRU temporal reasoning gated loop unit, thereby achieving filtering enhancement and iterative reasoning of accident risk representation.

[0123] The risk reasoning module consists of a GRU temporal reasoning gated loop unit and a scale-aware temporal aggregation strategy. The scale-aware temporal aggregation strategy constructs a multi-head self-attention-based temporal enhancement subunit, a multilayer perceptron-based scale consistency evaluation subunit, a scale-weighted noise suppression subunit, and a temporal attention aggregation subunit within an iterable local time window. This strategy is used to filter and correct the historical state output within the GRU temporal reasoning gated loop unit.

[0124] The process of the scale-aware temporal aggregation strategy is as follows:

[0125] First, continuous spatial frame-level fusion feature sequences The hidden state sequences at the current and historical moments are recursively output through the GRU temporal inference gated loop unit, and then a preset iterable local time window step size is used. Slice the hidden state sequence to construct an iterable local time window set of states. ,in For the first Spatial frame-level fusion features of video frames The expression after an iterable local time window is input, i.e., the first expression in the GRU temporal inference gated recurrent unit. A set of hidden states, which can be used as a local benchmark for temporal quality assessment, will be used to solve the problem of non-stationarity of temporal characteristics caused by abrupt changes or jitter under fixed roadside viewpoints;

[0126] Then, the temporal enhancement subunit mines the iterable local time window through a multi-head self-attention mechanism. Long-range dependencies between frames, utilizing Learnable time query weight matrix for each parallel attention head Time key weight matrix and time value weight matrix , The set of iterable local time window states Projected onto the temporal augmentation space, and then calculated and aggregated. frame, Temporal attention between head features, performing iterable local temporal window state set. Self-correlation modeling in the time domain outputs a set of temporal context state augmented representations that fuse long-range dependencies. ,in For the first Temporal context state augmentation representation of each video frame The calculation process is as follows:

[0127] ;

[0128] ;

[0129] In the formula: For the number of attention heads, For the first The temporal context state enhancement representation of each attention head For the first Feature dimension scaling factor for each attention head For multi-head attention fusion projection matrix, Corresponding to the first Feature query for each attention head ,key ,value The learnable projective weight matrix, For the first A set of iterable local temporal window states for each video frame. It is the transpose symbol;

[0130] Next, the time context state enhancement representation set The input is fed into the scale consistency evaluation subunit, and then processed through a scale evaluation network consisting of multiple fully connected layers. A nonlinear mapping is performed, and the scale stability features of the target in spatiotemporal evolution are extracted using the first and second linear layers. These features are then compressed into a scale consistency score sequence using the third linear layer. This enables precise quantification of temporal and spatial fluctuations and frequent occlusion, among which... For the iterable local time window, the first The frame scale consistency score is calculated using the following logic:

[0131] ;

[0132] ;

[0133] ;

[0134] In the formula: express The Middle Transpose of the temporal context state augmentation representation of each video frame. The first Layer, First Layer and first The network weight matrix of the layer, For the bias terms corresponding to each fully connected layer, respectively handle the first Outputs of the first and second linear layers per video frame; scale consistency score. ,in The aim is to quantitatively characterize the first The reliability of frame observation under roadside monitoring perspective is affected by drastic scale changes or occlusion;

[0135] Next, the scale-weighted noise suppression subunit utilizes the obtained scale consistency score sequence Enhanced representation set of temporal context states Element-level weighted masking is performed to actively suppress the enhanced representation components of low-confidence frames, thereby increasing the proportion of enhanced representations of high-quality precursor information. This results in a denoised state representation sequence with a more robust temporal feature distribution. ,in For the first The denoising state representation of each video frame is calculated based on the following criteria:

[0136] ;

[0137] In the formula: express The Middle Temporal context state augmentation representation of each video frame For Hadamah accumulation;

[0138] Finally, the denoised state representation sequence after weighted filtering Historical aggregation will be achieved through a time-attention aggregation subunit, which introduces an independent original temporal importance evaluation network. By calculating the iterable local time window The original frame pairs The correction contribution of the final historical state of the frame is used to obtain the correction contribution weight set. ,in For the first The video frame is the first The contribution of the final historical state of the frame to the correction, and the denoised state representation sequence based on this weight set. Perform dynamic weighted aggregation and output the first... Frame scale-aware aggregated state representation ;

[0139] The original time-series importance evaluation network Consists of a fully connected layer It consists of a Tanh activation function layer and first utilizes the weight matrix of the temporally fully connected layer. and corresponding bias weights Within the iterable local time window Iterable local time window state set of a frame The elements of the set in the code are mapped to one-dimensional scalars, and the numerical values ​​are mapped to scalars using the Tanh function. Finally, the softmax layer is used to normalize each element along the time dimension, outputting a set of corrected contribution weights. The calculation process is as follows:

[0140] ;

[0141] Subsequently, the modified contribution weight set was used. Denoising state representation sequence Perform a weighted summation and output the first... Frame scale-aware aggregated state representation :

[0142] ;

[0143] The native spatial frame-level fusion features achieve confidence filtering of historically damaged information by explicitly modeling the temporal scale consistency between window frames, and correct and aggregate the scale-aware aggregated state representation. Will be the first to be predicted The historical state of the frame is input into the GRU sequential inference gated loop unit.

[0144] The flow of the GRU sequential inference gated loop unit is as follows:

[0145] First, the spatial feature aggregation module outputs the first prediction value to be predicted. Spatial frame-level fusion features The scale-aware aggregation state representation of the previous frame output by the current scale-aware temporal aggregation strategy. By utilizing the update and reset gates of the GRU sequential reasoning gated loop unit, information filtering and risk evolution reasoning are performed. The internal recursive state update process is expressed as follows:

[0146] ;

[0147] ;

[0148] ;

[0149] In the formula: For the first The update gate vector of a frame in the GRU temporal inference gated recurrent unit. For the first The reset gate vector of a frame in the GRU sequential inference gated loop unit. For the first Candidate hidden states of frames in GRU temporal inference gated recurrent units Indicates the first The joint input vector is formed by concatenating the spatial frame-level fusion features of the frame with the scale-aware aggregated state representation of the previous frame along the channel dimension. This indicates that for the input number... The weight matrix part of the spatial frame-level fusion features of the frame. This represents the aggregated state representation for historical scale perception. The weight matrix part, Indicates the first The weight matrix for calculating the candidate hidden states of a frame. This represents the bias vector between the update gate and the reset gate in the GRU sequential inference gated loop unit;

[0150] The output of the GRU sequential inference gated loop unit is the first... Frame hidden state Synchronous output and recursive input of the first... Iterable local time window state set of a frame In the middle, as the sliding window benchmark for the next prediction period, and at the same time, the first Hidden state of a frame Will be the first The accident risk characterization of the frame will be synchronously input into the downstream prediction output module.

[0151] Specifically, the prediction output module performs the following operations:

[0152] First, the risk reasoning module outputs the first... Frame hidden state Will be the first The accident risk characterization of a frame is obtained through a classification prediction head with a two-layer fully connected architecture, which ultimately outputs the accident prediction status of the current frame.

[0153] The first fully connected layer performs dimensionality reduction and abstraction of high-dimensional features and utilizes the ReLU activation function to enhance the model's nonlinear representation capability. The second fully connected layer maps the features to a binary classification space, and the output reflects the... Binary logits sequence of frame incident risk :

[0154] ;

[0155] In the formula: These are the weight matrices for the two fully connected layers. Its corresponding bias term;

[0156] The results reflect the first Binary logits sequence of frame incident risk The probability will be normalized through a Softmax layer to calculate the... Accident probability score of a frame :

[0157] ;

[0158] In the formula, Corresponding to the The original score of the non-accident category in the binary classification results of the frame. Corresponding to the The original score of the accident category in the binary classification results of the frame;

[0159] Finally, using predetermined accident thresholds With the Accident probability score of a frame Perform real-time comparison; when the accident probability score exceeds a threshold, the accident prediction is triggered.

[0160] ;

[0161] In the formula, For the first The frame's accident trigger state, when When this happens, a safety warning is triggered.

[0162] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for predicting traffic accidents using roadside monitoring based on scale-aware temporal aggregation, characterized in that, This method utilizes an accident prediction model to achieve early warning of road traffic accidents. The model includes a feature extraction module, a spatial feature aggregation module, a risk reasoning module, and a prediction output module. The feature extraction module is responsible for target detection and feature extraction from standardized traffic processing unit videos. It extracts target local features and frame-wide features from video frames by combining detection boxes and feature maps, and constructs a target local feature set using a target masking mechanism. The spatial feature aggregation module is responsible for fusing and stitching the frame-wide features and target local feature sets. It enhances the information interaction of target local features between target objects through a spatial self-attention mechanism, constructing spatial frame-level fusion features to realize spatial features before risk reasoning. The risk inference module is responsible for outputting the accident risk representation of the current frame. It relies on the temporal inference gated recurrent unit and the filtering enhancement strategy. By inputting the continuous spatial frame-level fused features into the filtering enhancement GRU temporal inference gated recurrent unit, the accident risk representation is filtered and enhanced and iteratively inferred. The filtering enhancement strategy achieves fine-grained scale perception and temporal aggregation of the accident risk representation by fusing a multi-head self-attention mechanism and a multilayer perceptron mechanism. This is called the scale-aware temporal aggregation strategy. The prediction output module is responsible for calculating the accident probability score. It performs inference through a classification prediction head with a two-layer fully connected architecture, outputs the accident probability score of the current prediction frame, and compares it with a predetermined accident threshold to complete the triggering of accident prediction. The specific implementation of the roadside monitoring traffic accident prediction method includes the following steps: 1) Acquire traffic video data from the perspective of road monitoring and preprocess it to obtain standardized traffic processing unit videos with accident information marked and of equal length; 2) The standardized traffic processing unit video obtained in step 1) is input into the accident prediction model for training. First, the target local features and frame global features are obtained through the feature extraction module, and then fused and spliced ​​into spatial frame-level fusion features through the spatial feature aggregation module. The spatial frame-level fusion features are iteratively input into the risk inference module to output the accident risk representation of the prediction frame. The accident risk representation of the prediction frame is input into the prediction output module to finally output the accident prediction state of the prediction frame. The accident prediction model is trained by comparing the accident prediction state of the prediction frame output by the model inference with the true value of the labeled accident information of the prediction frame. In the training process, a weighted cross-entropy loss function with a time penalty factor is introduced. The time penalty factor changes exponentially with time, which effectively guides the prediction accuracy of the model during the training process. 3) Input the unlabeled video data to be tested into the trained accident prediction model, and the accident probability score of the corresponding frame will be output. The score will be compared with the predetermined accident threshold to complete the accident prediction.

2. The method for predicting traffic accidents based on scale-aware temporal aggregation for roadside monitoring according to claim 1, characterized in that, In step 1), the preprocessing includes: performing fixed-length slice sampling and resolution unification on the video at a set frame rate, marking the start point of the video accident and the accident status of each video frame, and performing clone padding repair on samples with insufficient frame counts after sampling, forming standardized traffic processing unit videos with accident information marked and equal time length, as detailed below: First, the motion interpolation operator is invoked to uniformly process the traffic video data from the original road monitoring perspective at a target frame rate of 30fps for a length of [length missing]. Fixed-length time-series slicing of frames eliminates feature nonlinearity deviations caused by sampling frequency drift from different monitoring devices. Simultaneously, the video resolution is uniformly mapped to a 1280×720 standard pixel matrix to eliminate the impact of spatial scale differences on subsequent feature extraction modules. Then, the video accident start point and accident status information for each video frame are manually labeled. Finally, to address the issue of insufficient video frames after slicing... The video samples of each frame are used to perform tail frame cloning and filling repair by calling the FFmpeg boundary fill filter tpad, forming a standardized traffic processing unit video with accident information labeled and time of equal length.

3. The method for predicting roadside traffic accidents based on scale-aware temporal aggregation according to claim 1, characterized in that, In step 2), the accident start frame with accident information already labeled in the standardized traffic processing unit video is used. and video sampling frame rate Based on this, a time penalty factor that grows exponentially over time is constructed. : ； In the formula: A sequence of frames after video sampling for a standardized traffic processing unit. The time index of each frame in the middle, where Represents a frame sequence The first in frame, ; Using the time penalty factor The final prediction loss of each frame sequence after inference from the accident prediction model is weighted to construct a weighted cross-entropy loss function with a time penalty factor. : ； In the formula: This is the batch size for model training. The video index number for the standardized traffic processing unit video. For the first The first standardized traffic processing unit video Accident information in frames, The first output of the accident prediction model Frame accident prediction probability; Using the above weighted cross-entropy loss function The accident prediction model is trained to achieve strong suppression of delayed warning behavior and significant forward shift of the risk perception threshold. During the training process, the accident prediction model is forced to capture weak early warning features before the accident occurs, thereby effectively guiding the model's prediction accuracy during the training process.

4. The method for predicting roadside traffic accidents based on scale-aware temporal aggregation according to claim 3, characterized in that, The feature extraction module performs the following operations: First, for each frame A high-order feature map is generated by capturing global features of frames through a backbone network based on the VGG-16 architecture. ; Then, the target detection and local feature extraction process for each target object in the video frame is executed: the trained Cascade R-CNN cascaded regression detection model is used to detect targets in the video frame and generate a set of bounding boxes for the target objects. , ,in For the first The first video frame The bounding box of a target object. For the first The total number of target objects corresponding to the nth video frame; for the nth video frame For each target object in each video frame, in the high-order feature map Perform RoI alignment and pooling to obtain standard target local features. ,in For the first The first video frame Local features of a target object; To avoid inconsistencies in the target local feature set tensor caused by fluctuations in the number of target objects, the total number of target objects... This will perform truncation or padding, set... The maximum number of target object slots is And construct an effective target mask set. , To indicate the real target object and the filled noise, where , For slot pointers, Indicates the first Within the first video frame The target mask of a target object; Ultimately, the local features of each target The features are mapped to a high-dimensional continuous feature space through a shared linear projection layer, resulting in the... The first video frame Target local embedding features of a target object and construct the first Set of target local embedding features of video frames Simultaneously extract the high-order feature map of the frame. Corresponding global embedding features .

5. The method for predicting roadside traffic accidents based on scale-aware temporal aggregation according to claim 4, characterized in that, The spatial feature aggregation module performs the following operations: First, a spatial self-attention mechanism is used to interactively model and fuse the global features and target local feature sets of the corresponding video frames, while quantifying the semantic association weights between target local features; the target local embedding feature set... As input, a spatial self-attention mechanism is used to enhance feature interactions, and an effective target mask set is also incorporated. Attention masking is implemented for invalid slots, and a weighted set of target local enhancement features is obtained. ,in For the first The first video frame Local enhancement features of a target object; Subsequently, the global mean pooling operator was combined with the pair Perform compression mapping and output the first... Target local aggregation features of video frames and will With the Global embedding features of each video frame Perform fusion and stitching along the channel dimension to form the first Spatial frame-level fusion features of video frames .

6. The method for predicting roadside traffic accidents based on scale-aware temporal aggregation according to claim 5, characterized in that, The specific operation steps of the spatial feature aggregation module are as follows: Embedding feature set of target Using the effective target mask set Pre-filtering is performed, followed by a linear mapping using a spatial self-attention matrix to generate the first... Spatial self-attention triplet tensor of a video frame : ； In the formula, Representing spatial self-attention In the feature space of the query ,key ,value Learnable projective weight matrix in dimensionality The query matrix that carries the original feature information. The key-value matrix that carries the original feature information. The value matrix that carries the original feature information. It represents the Hadamardi (or Hadama) stack; Subsequently, the effective target mask set Transform into spatial bias matrix By performing a single dot product attention calculation, weight normalization, noise masking, and feature aggregation are completed, resulting in a set of target local enhancement features. ,in For the first The first video frame Local enhancement features of a target object The calculation process is as follows: ； In the formula, This is the transpose of the key-value matrix that carries the original feature information; This represents the original spatial correlation score matrix, which measures the strength of the correlation between slots. This is a scaling factor used to stabilize the gradient in the dot product operation; For the first The spatial attention mask matrix for each video frame is constructed according to the following criteria: ； In the formula, , , For the first Within the first video frame and the Joint target mask for multiple target objects. For the first Within the first video frame The target mask of a target object. For the first Within the first video frame The target mask of a target object; Enhance the local feature set of the target using the global mean pooling operator. Perform compression mapping to obtain the target local aggregated features. ; Finally, to achieve efficient spatial feature extraction and semantic encapsulation in a single frame, With global embedded features Perform fusion and stitching processing along the channel dimension to form the first Spatial frame-level fusion features of video frames .

7. The method for predicting roadside traffic accidents based on scale-aware temporal aggregation according to claim 6, characterized in that, The risk reasoning module performs the following operations: First, the continuous spatial frame-level fused feature sequence input from the spatial feature aggregation module is received. It is used as the GRU temporal inference gated loop unit for iterative input filtering of temporal information, and combined with the historical state and hidden state generated inside the GRU temporal inference gated loop unit, the corrected hidden state of each frame is generated iteratively. The filtering enhancement strategy, which integrates multi-head self-attention mechanism and multilayer perceptron mechanism, achieves fine-grained scale perception and temporal aggregation of accident risk representation. This strategy is called scale perception temporal aggregation strategy. It captures the hidden state sequence generated inside the GRU temporal reasoning gated loop unit through iterative local time window, and combines scale filtering and dynamic temporal aggregation to output the historical state of the next prediction frame in the GRU temporal reasoning gated loop unit, thereby achieving filtering enhancement and iterative reasoning of accident risk representation. The risk reasoning module consists of a GRU temporal reasoning gated loop unit and a scale-aware temporal aggregation strategy. The scale-aware temporal aggregation strategy constructs a multi-head self-attention-based temporal enhancement subunit, a multilayer perceptron-based scale consistency evaluation subunit, a scale-weighted noise suppression subunit, and a temporal attention aggregation subunit within an iterable local time window. This strategy is used to filter and correct the historical state output within the GRU temporal reasoning gated loop unit.

8. The method for predicting roadside traffic accidents based on scale-aware temporal aggregation according to claim 7, characterized in that, The process of the scale-aware temporal aggregation strategy is as follows: First, continuous spatial frame-level fusion feature sequences The hidden state sequences at the current and historical moments are recursively output through the GRU temporal inference gated loop unit, and then a preset iterable local time window step size is used. Slice the hidden state sequence to construct an iterable local time window set of states. ,in For the first Spatial frame-level fusion features of video frames The expression after an iterable local time window is input, i.e., the first expression in the GRU temporal inference gated recurrent unit. A set of hidden states, which can be used as a local benchmark for temporal quality assessment, will be used to solve the problem of non-stationarity of temporal characteristics caused by abrupt changes or jitter under fixed roadside viewpoints; Then, the temporal enhancement subunit mines iterable local time windows through a multi-head self-attention mechanism. Long-range dependencies between frames, utilizing Learnable time query weight matrix for each parallel attention head Time key weight matrix and time value weight matrix , The set of iterable local time window states Projected onto the temporal augmentation space, and then calculated and aggregated. frame, Temporal attention between head features, performing iterable local temporal window state set. Self-correlation modeling in the time domain outputs a set of temporal context state augmented representations that fuse long-range dependencies. ,in For the first Temporal context state augmentation representation of each video frame The calculation process is as follows: ； ； In the formula: For the number of attention heads, For the first The temporal context state enhancement representation of each attention head For the first Feature dimension scaling factor for each attention head For multi-head attention fusion projection matrix, Corresponding to the first Feature query for each attention head ,key ,value The learnable projective weight matrix, For the first A set of iterable local temporal window states for each video frame. It is the transpose symbol; Next, the time context state enhancement representation set The input is fed into the scale consistency evaluation subunit, and then processed through a scale evaluation network consisting of multiple fully connected layers. A nonlinear mapping is performed, and the scale stability features of the target in spatiotemporal evolution are extracted using the first and second linear layers. These features are then compressed into a scale consistency score sequence using the third linear layer. This enables precise quantification of temporal and spatial fluctuations and frequent occlusion, among which... For the iterable local time window, the first The frame scale consistency score is calculated using the following logic: ； ； ； In the formula: express The Middle Transpose of the temporal context state augmentation representation of each video frame. The first Layer, First Layer and first The network weight matrix of the layer, For the bias terms corresponding to each fully connected layer, respectively handle the first Outputs of the first and second linear layers per video frame; scale consistency score. ,in The aim is to quantitatively characterize the first The reliability of frame observation under roadside monitoring perspective is affected by drastic scale changes or occlusion; Next, the scale-weighted noise suppression subunit utilizes the obtained scale consistency score sequence Enhanced representation set of temporal context states Element-level weighted masking is performed to actively suppress the enhanced representation components of low-confidence frames, thereby increasing the proportion of enhanced representations of high-quality precursor information. This results in a denoised state representation sequence with a more robust temporal feature distribution. ,in For the first The denoising state representation of each video frame is calculated based on the following criteria: ； In the formula: express The Middle Temporal context state augmentation representation of each video frame For Hadamah accumulation; Finally, the denoised state representation sequence after weighted filtering Historical aggregation will be achieved through a time-attention aggregation subunit, which introduces an independent original temporal importance evaluation network. By calculating the iterable local time window The original frame pairs The correction contribution of the final historical state of the frame is used to obtain the correction contribution weight set. ,in For the first The video frame is the first The contribution of the final historical state of the frame to the correction, and the denoised state representation sequence based on this weight set. Perform dynamic weighted aggregation and output the first... Frame scale-aware aggregated state representation ; The original time-series importance evaluation network Consists of a fully connected layer It consists of a Tanh activation function layer and first utilizes the weight matrix of the temporally fully connected layer. and corresponding bias weights Within the iterable local time window Iterable local time window state set of a frame The elements of the set in the code are mapped to one-dimensional scalars, and the numerical values ​​are mapped to scalars using the Tanh function. Finally, the softmax layer is used to normalize each element along the time dimension, outputting a set of corrected contribution weights. The calculation process is as follows: ； Subsequently Using the modified contribution weight set Denoising state representation sequence Perform a weighted summation and output the first... Frame scale-aware aggregated state representation : ； The native spatial frame-level fusion features achieve confidence filtering of historically damaged information by explicitly modeling the temporal scale consistency between window frames, and correct and aggregate the scale-aware aggregated state representation. Will be the first to be predicted The historical state of the frame is input into the GRU sequential inference gated loop unit.

9. A method for predicting traffic accidents based on scale-aware temporal aggregation for roadside monitoring, as described in claim 8, is characterized in that... The flow of the GRU sequential inference gated loop unit is as follows: First, the spatial feature aggregation module outputs the first prediction value to be predicted. Spatial frame-level fusion features The scale-aware aggregation state representation of the previous frame output by the current scale-aware temporal aggregation strategy. By utilizing the update and reset gates of the GRU sequential reasoning gated loop unit, information filtering and risk evolution reasoning are performed. The internal recursive state update process is expressed as follows: ； ； ； In the formula: For the first The update gate vector of a frame in the GRU temporal inference gated recurrent unit. For the first The reset gate vector of a frame in the GRU sequential inference gated loop unit. For the first Candidate hidden states of frames in GRU temporal inference gated recurrent units Indicates the first The joint input vector is formed by concatenating the spatial frame-level fusion features of the frame with the scale-aware aggregated state representation of the previous frame along the channel dimension. This indicates that for the input number... The weight matrix part of the spatial frame-level fusion features of the frame. This represents the aggregated state representation for historical scale perception. The weight matrix part, Indicates the first The weight matrix for calculating the candidate hidden states of a frame. This represents the bias vector between the update gate and the reset gate in the GRU sequential inference gated loop unit; The output of the GRU sequential inference gated loop unit is the first... Frame hidden state Synchronous output and recursive input of the first... Iterable local time window state set of a frame In the middle, as the sliding window benchmark for the next prediction period, and at the same time, the first Hidden state of a frame Will be the first The accident risk characterization of the frame will be synchronously input into the downstream prediction output module.

10. A method for predicting roadside traffic accidents based on scale-aware temporal aggregation according to claim 9, characterized in that, The prediction output module performs the following operations: First, the risk reasoning module outputs the first... Frame hidden state Will be the first The accident risk characterization of a frame is obtained through a classification prediction head with a two-layer fully connected architecture, which ultimately outputs the accident prediction status of the current frame. The first fully connected layer performs dimensionality reduction and abstraction of high-dimensional features and utilizes the ReLU activation function to enhance the model's nonlinear representation capability. The second fully connected layer maps the features to a binary classification space, and the output reflects the... Binary logits sequence of frame incident risk : ； In the formula: These are the weight matrices for the two fully connected layers. Its corresponding bias term; The results reflect the first Binary logits sequence of frame incident risk The probability will be normalized through a Softmax layer to calculate the... Accident probability score of a frame : ； In the formula, Corresponding to the The original score of the non-accident category in the binary classification results of the frame. Corresponding to the The original score of the accident category in the binary classification results of the frame; Finally, using predetermined accident thresholds With the Accident probability score of a frame Perform real-time comparison; when the accident probability score exceeds a threshold, the accident prediction is triggered. ； In the formula, For the first The frame's accident trigger state, when When this happens, a safety warning is triggered.

Images