Video abnormal behavior detection method and system based on space-time pyramid

Abstract

The application provides a spatio-temporal pyramid-based video abnormal behavior detection method and system, and relates to the technical field of video detection.The method comprises the following steps: acquiring a monitoring video sequence and performing pretreatment; constructing a spatio-temporal pyramid structure based on preset spatio-temporal scale parameters, dividing multi-scale spatio-temporal blocks through time hierarchical sampling and spatial down-sampling; extracting features of each layer spatio-temporal block by using an improved three-dimensional convolutional neural network, and combining channel and spatial double-path attention mechanisms to perform feature enhancement; fusing features of each layer from top to bottom, constructing a global fusion spatio-temporal feature map, and inputting the global fusion spatio-temporal feature map into a preset abnormality detection model to calculate an abnormality probability; performing time consistency verification and scene prior verification on the preliminary result, and outputting a structured detection result; and the system comprises a video sequence pretreatment and detection result post-processing module and the like.The application can effectively capture macro patterns and micro details of abnormal behaviors, improve detection accuracy and robustness, and is suitable for complex scenes such as subway stations.

Description

Technical Field

[0001] This invention relates to the field of video detection technology, and in particular to a method and system for detecting abnormal video behavior based on a spatiotemporal pyramid. Background Technology

[0002] Video anomaly detection is crucial for public safety monitoring. For example, in subway security checkpoints, it is necessary to detect abnormal behaviors such as sudden running, lingering, or walking against traffic in real time. Most existing methods rely on feature analysis at a single spatiotemporal scale, such as traditional optical flow methods or simple convolutional networks, which struggle to simultaneously capture macroscopic behavioral patterns over long periods and subtle changes in movement over short periods. Specifically, in real-world subway scenarios, existing methods are prone to false detections due to differences in camera perspectives, lighting variations, and background interference. For instance, brief changes in light and shadow might be misinterpreted as abnormal running, or normal local behavior might be falsely reported due to a lack of global context. Furthermore, current technologies do not adequately consider the spatiotemporal continuity of anomalies and cannot effectively integrate multi-granularity information, resulting in insensitivity to periodic or slow anomalies (such as prolonged lingering). These issues limit the practicality and reliability of existing technologies in complex real-world scenarios, thus necessitating a novel video anomaly detection method and system based on a spatiotemporal pyramid. Summary of the Invention

[0003] The purpose of this invention is to provide a video abnormal behavior detection method and system based on spatiotemporal pyramids, in order to solve the problems in existing technologies, such as the inability to effectively quantify the stability and concentration of user preferences, the neglect of the consensus on the attractiveness of topics in related user groups, and the difficulty in comprehensively considering users' long-term static preferences, short-term dynamic behaviors, and topic group characteristics for integrated decision-making. These problems lead to the recommendation topics being easily affected by random behavior and resulting in insufficient recommendation accuracy. The specific technical solution is as follows: This invention provides a video anomaly behavior detection method based on a spatiotemporal pyramid, comprising: Step S101: Obtain a continuous monitoring video sequence of the scene to be detected, perform frame synchronization preprocessing on the video sequence, remove redundant noise frames and unify the frame resolution; Step S102: Based on preset spatiotemporal scale parameters, a spatiotemporal pyramid structure is constructed on the preprocessed video sequence. The spatiotemporal pyramid includes multiple sampling levels in the time dimension and multiple scale levels in the spatial dimension. Normalized spatiotemporal blocks are obtained by hierarchical sampling and spatial downsampling, thus completing the bottom-up multi-scale spatiotemporal block division. Step S103: Feature extraction is performed on the spatiotemporal blocks of each layer of the spatiotemporal pyramid. An improved three-dimensional convolutional neural network is used to extract multi-scale spatiotemporal features of each layer's spatiotemporal subsequence. An attention mechanism is used to assign weights and enhance the features at each level to obtain enhanced spatiotemporal features at each level. Step S104: Perform cross-level fusion of enhanced spatiotemporal features of each layer of the pyramid from top to bottom to construct a global fused spatiotemporal feature map. Input the map into the preset anomaly detection model for feature matching and anomaly probability calculation to obtain preliminary anomaly detection results. Step S105: Post-process the preliminary anomaly detection results, remove time-isolated false detection results based on the feature similarity of adjacent frames, and verify the spatiotemporal rationality of the abnormal behavior by combining the prior information of the scene to be detected. Finally, output the structured detection results containing the abnormal behavior area, type, time interval and precise spatiotemporal positioning information.

[0004] Furthermore, the frame synchronization preprocessing of the video sequence in step S101 includes: By deploying a network time protocol server to provide a unified time reference for all surveillance cameras, each video frame is stamped with a high-precision timestamp based on this unified time reference during video acquisition. Based on the timestamp, the video frames acquired by different cameras are aligned and recombined to obtain a set of multiple video frames that are strictly synchronized on the timeline. The synchronized continuous monitoring video sequence is subjected to redundant noise frame removal operation. The method based on inter-frame difference measurement is used to calculate the gray value difference of corresponding pixels between consecutive video frames, and to count the proportion of pixels with differences exceeding a preset threshold to the total number of pixels. When the proportion is lower than the preset sensitivity threshold, the next frame is determined to be a redundant noise frame and is removed. The frame resolution of the deredundant video sequence is unified by using a bilinear interpolation algorithm to scale all video frames to a uniform standard resolution.

[0005] Furthermore, the construction of the spatiotemporal pyramid structure in step S102 includes: The spacetime pyramid is defined as containing L in the time dimension. t Each sampling level contains L in the spatial dimension. s Each scale level, where L t =3,L s =3; For the lth time dimension t Layer sampling level, where l t =0,1,2, defining the time sampling step S. t (l t Following the exponential increasing rule, the calculation formula is as follows: S0 is the base time step, set to 2 frames, starting from the first frame of the video sequence and incrementing by S. t (l t The video frames are extracted at uniform intervals to form a sub-video sequence at that time level; For the lth spatial dimension s Layer scale level, where l s=0,1,2, define the spatial downsampling factor D. s (l s Following the exponentially decreasing rule, each frame of the temporal subsequence is scaled to the target spatial resolution H′×W′ using a bilinear interpolation algorithm. The video subsequence under each spatiotemporal combination is divided according to rules. The size of the spatiotemporal block is set as h×w×d, where h=16, w=16, and d=8. The video subsequence is cut into multiple spatiotemporal blocks of the same size using a sliding window method.

[0006] Furthermore, the feature extraction using an improved three-dimensional convolutional neural network in step S103 includes: The spatiotemporal block is input into a pre-trained 3D convolutional neural network model, which is an improvement on the C3D network architecture and includes multiple 3D convolutional layers, 3D pooling layers and fully connected layers. A dual-path attention module is integrated on the feature map output by the last convolutional layer to compute channel attention weights and spatial attention weights in parallel. The channel attention path generates channel descriptors through global average pooling and global max pooling. After being input into a shared multilayer perceptron, it is activated by Sigmoid to generate channel attention weight vectors, which are then multiplied element-wise with the original feature map in the channel dimension. The spatial attention path obtains spatial feature maps through global average pooling and global max pooling along the channel dimension. After concatenation, it generates a spatial attention weight matrix through 7×7 convolution and Sigmoid activation. The matrix is ​​then multiplied element-wise with the channel attention-weighted feature map in the spatial dimension. The calibrated and enhanced features are aggregated into a fixed-dimensional enhanced spatiotemporal feature vector through global average pooling.

[0007] Furthermore, the top-down cross-level fusion in step S104 includes: The enhanced spatiotemporal feature map F2 of the top layer L2 of the spatiotemporal pyramid is spatially upsampled and fused with the feature map F1 of the next layer L1. Before fusion, the number of channels of F1 is adjusted by 1×1 convolution. The fusion operation includes element-wise addition and channel concatenation to generate an intermediate fused feature map F. mid1 ; F mid1 Spatial upsampling is performed and fused with the feature map F0 of the underlying L0. Similarly, the number of channels is adjusted through 1×1 convolution. After addition and concatenation, a global fused spatiotemporal feature map F is generated through a convolutional block. global ; F global Input a preset anomaly detection model, which is a classification network pre-trained on normal and abnormal behavior videos, and output the abnormal behavior category and probability as the preliminary detection result.

[0008] Further, the post-processing in step S105 includes: Based on time consistency verification, the cosine similarity between the global fusion spatiotemporal feature maps of adjacent frames is calculated. For abnormal frames with a probability Pt greater than the primary probability threshold, the number of consecutive frames greater than the probability threshold η within the time window W before and after the frame is checked. If the number of consecutive frames is less than the minimum number of continuous frames K, the abnormality is removed. Based on scenario prior verification, a scenario prior rule base is constructed. The spatial region, start timestamp, and behavior type of abnormal events are matched with the rule base to eliminate detection results that do not conform to logic. The system outputs structured results for real-world abnormal events, including the type of abnormal behavior, the precise time interval, the spatial location bounding box, and the confidence score.

[0009] Furthermore, in the redundant noise frame removal operation, the preset sensitivity threshold is set to 0.001; in the frame resolution unification, the standard resolution is set to 640x480 pixels.

[0010] This invention also provides a video anomaly behavior detection system based on a spatiotemporal pyramid, for implementing the method described above, the system comprising: The video sequence preprocessing module is used to acquire continuous monitoring video sequences of the scene to be detected, perform frame synchronization preprocessing on the video sequences, remove redundant noise frames, and unify frame resolution. The spatiotemporal pyramid construction module is used to construct a spatiotemporal pyramid structure on the preprocessed video sequence based on preset spatiotemporal scale parameters, and complete the bottom-up multi-scale spatiotemporal block division. The multi-scale feature extraction and enhancement module is used to extract features from each spatiotemporal block of the spatiotemporal pyramid. It uses an improved three-dimensional convolutional neural network to extract multi-scale spatiotemporal features and uses an attention mechanism to assign weights and enhance features at each level. The cross-level feature fusion and anomaly detection module is used to perform cross-level fusion of enhanced spatiotemporal features of each layer of the pyramid from top to bottom, construct a global fused spatiotemporal feature map, and input the preset anomaly detection model for feature matching and anomaly probability calculation. The post-processing module for detection results is used to post-process the preliminary anomaly detection results, remove time-isolated false detection results based on the feature similarity of adjacent frames, and verify the spatiotemporal rationality of abnormal behavior by combining the prior information of the scene to be detected.

[0011] Furthermore, the video sequence preprocessing module includes: The frame synchronization processing unit is used to provide a unified time reference for all surveillance cameras by deploying a network time protocol server, and to align and reassemble video frames collected by different cameras according to timestamps to obtain a set of multiple video frames that are strictly synchronized on the timeline. The redundant frame removal unit is used to calculate the gray value difference of corresponding pixels between consecutive video frames using a method based on inter-frame difference measurement, count the proportion of pixels whose differences exceed a preset threshold, and when the proportion is lower than a preset sensitivity threshold, it is determined to be a redundant noise frame and removed. The resolution unification unit is used to scale all video frames to a uniform standard resolution using a bilinear interpolation algorithm; The spatiotemporal pyramid construction module includes: The spatiotemporal scale parameter setting unit is used to set the sampling level Lt in the time dimension and the scale level Ls in the spatial dimension of the spatiotemporal pyramid, and to define the time sampling step size and spatial downsampling factor of each layer. The time-level sampling unit is used to set the time sampling step size of each time sampling level according to the exponential increment rule. Starting from the starting frame of the video sequence, video frames are extracted evenly with the corresponding step size to form sub-video sequences of different time levels. The spatial hierarchical downsampling unit is used to set the spatial downsampling factor for each spatial scale according to the exponential decreasing rule, and to scale each frame of the temporal hierarchical subsequence to the target spatial resolution using the bilinear interpolation algorithm. The spatiotemporal block partitioning unit is used to divide the video subsequence at each spatiotemporal level into multiple sets of spatiotemporal blocks of the same size using a sliding window method.

[0012] Furthermore, the multi-scale feature extraction and enhancement module includes: The 3DCNN feature extraction unit is used to perform three-dimensional convolution and pooling operations on the input spatiotemporal block using an improved three-dimensional convolutional neural network as the basic feature extractor, and output a high-dimensional feature vector. The dual-path attention unit is used to compute channel attention weights and spatial attention weights in parallel, recalibrate the feature map, and enhance the feature response of important channels and key spatial regions. The feature enhancement output unit is used to aggregate the features that have been calibrated and enhanced by the dual-path attention mechanism into a fixed-dimensional enhanced spatiotemporal feature vector through a global average pooling layer. The cross-level feature fusion and anomaly detection module includes: The hierarchical feature fusion unit is used to gradually fuse high-level semantic information with low-level detailed information from top to bottom. It uses bilinear interpolation for spatial upsampling and achieves cross-level feature fusion by combining element-wise addition and channel concatenation. The global feature construction unit is used to process the fused features through convolutional blocks to generate a global fused spatiotemporal feature map containing multi-scale contextual information; The anomaly detection model unit is used to input the global fused feature map into a pre-trained classification network, calculate the probability of belonging to each category through feature matching, and output preliminary detection results containing behavior category and anomaly probability. The post-processing module for the detection results includes: The temporal consistency verification unit is used to calculate the cosine similarity between features of adjacent frames, check the number of consecutive frames greater than the probability threshold within the time window before and after the abnormal frame, and remove false detection results that are temporally isolated. The scenario prior verification unit is used to build a scenario prior rule base, match and verify the spatial location, start timestamp, and behavior type of abnormal events with the rule base, and remove detection results that do not conform to the scenario logic; An electronic device for outputting structured detection results containing abnormal behavior type, precise time interval, spatial location information and confidence score for confirmed abnormal events, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described above.

[0013] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method.

[0014] The beneficial effects of this invention are as follows: By constructing a spatiotemporal pyramid structure, this invention achieves collaborative analysis of multi-granular spatiotemporal information of video data. Combined with feature enhancement through attention mechanisms and top-down cross-level feature fusion, it can effectively capture the representation of abnormal behavior in both macroscopic patterns and microscopic details, significantly improving the detection accuracy of abnormal behaviors such as sudden running in complex scenarios (such as subway security checkpoints). At the same time, through the post-processing mechanism of time consistency verification and scene prior rules, false alarms caused by changes in light and shadow and brief interference are effectively filtered out, enhancing the robustness and practicality of the system. Finally, it outputs structured results with accurate spatiotemporal positioning information, providing efficient and reliable technical support for security monitoring.

[0015] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram illustrating the steps of the video abnormal behavior detection method based on spatiotemporal pyramids of the present invention; Figure 2 This is a schematic diagram of the video abnormal behavior detection system based on the spatiotemporal pyramid of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0018] In embodiments of the present invention, a video abnormal behavior detection method based on a spatiotemporal pyramid is provided. Please refer to [link to relevant documentation]. Figure 1 The method includes the following steps: Step S101: Obtain a continuous monitoring video sequence of the scene to be detected, perform frame synchronization preprocessing on the video sequence, remove redundant noise frames and unify the frame resolution.

[0019] This invention uses a subway station security check area as the target detection scenario for illustration. Abnormal behaviors in the subway station security check area may include people suddenly running, lingering for a long time, or crossing against the flow of traffic. In order to ensure the quality and consistency of the surveillance video data and lay the foundation for subsequent multi-scale spatiotemporal analysis, the original video stream must be standardized and preprocessed.

[0020] First, multiple network surveillance cameras are deployed at key locations such as the security checkpoints, waiting areas, and passageways of the target subway station to ensure that the monitoring range covers the entire target area and key details. Each camera captures a video stream at a frame rate of 25 frames per second. This frame rate is not limited and can be adjusted according to actual network bandwidth and storage conditions. Since there may be slight differences in the clocks of different cameras, the directly acquired raw video sequences are not strictly synchronized in time. Therefore, frame synchronization preprocessing is required. Specifically, a network time protocol server is deployed to provide a unified time reference for all surveillance cameras. During video acquisition, each video frame is stamped with a high-precision timestamp based on this unified time reference. When the backend server receives the video stream, it aligns and reassembles the video frames captured by different cameras based on this timestamp, thereby obtaining a set of multiple video frames that are strictly synchronized on the timeline and correspond to the same physical moment, forming the synchronized surveillance video sequence to be processed.

[0021] Next, redundant noise frames are removed from the synchronized continuous monitoring video sequence. In monitoring scenarios, when the content of the screen remains unchanged for a long time, a large number of redundant approximate frames are generated. These frames not only increase the processing burden but may also dilute the effective motion information. To efficiently remove these redundant frames, this embodiment uses a method based on inter-frame difference measurement to calculate the difference in grayscale values ​​of corresponding pixels between consecutive video frames and to count the proportion of pixels with differences exceeding a preset threshold to the total number of pixels. When this proportion is lower than a preset sensitivity threshold, such as one in a thousand, the next frame is determined to be a redundant noise frame and is removed. This process can be described as follows: Let the pixel difference rate between frame t and frame t-1 be D. t When D t When the threshold τ is less than τ, the t-th frame is discarded. The threshold τ is adjustable based on the level of scene activity, with a default value of 0.001. This step significantly compresses the amount of video data to be processed while preserving all frames showing key action changes.

[0022] Finally, the frame resolution of the deredundant video sequence is unified. Since the deployed surveillance cameras may differ in model or configuration, the acquired video frames may have various resolutions, such as 1920x1080, 1280x720, etc. To facilitate subsequent feature extraction network processing, all video frames need to be scaled to a unified standard resolution. In this embodiment, the standard resolution is set to 640x480 pixels. The scaling process uses a bilinear interpolation algorithm, which strikes a good balance between computational efficiency and image quality, reducing image detail distortion caused by scaling. After resolution unification, all input video data has a regular format in the spatial dimension, ensuring that models such as convolutional neural networks can receive and process it correctly in subsequent steps.

[0023] Thus, a standardized continuous surveillance video sequence with time synchronization, removal of static redundant frames, and uniform spatial resolution was obtained, providing high-quality and standardized input data for subsequent construction of the spatiotemporal pyramid and multi-scale feature analysis.

[0024] Step S102: Based on preset spatiotemporal scale parameters, a spatiotemporal pyramid structure is constructed on the preprocessed video sequence. The spatiotemporal pyramid includes multiple sampling levels in the time dimension and multiple scale levels in the spatial dimension. Normalized spatiotemporal blocks are obtained through hierarchical sampling and spatial downsampling, thus completing the bottom-up multi-scale spatiotemporal block division.

[0025] After obtaining the preprocessed, standardized video sequence, in order to simultaneously capture the macroscopic patterns and local details of anomalous behavior over time, as well as its overall situation and fine movements in space, it is necessary to construct a structured representation capable of characterizing the multi-granular spatiotemporal information of the video data. This step achieves this goal by constructing a spatiotemporal pyramid structure. This structure decomposes the original video sequence into a series of sub-sequence blocks with different spatiotemporal granularities through hierarchical sampling and scaling transformation in both time and space dimensions, thus laying the foundation for subsequent multi-scale feature extraction and fusion.

[0026] Specifically, the construction parameters of the spatiotemporal pyramid are first set. The spatiotemporal pyramid contains L in the time dimension. t Each sampling level contains L in the spatial dimension. s Each scale level. As a preferred implementation, this embodiment sets L... t =3,L s =3. That is, there are three sampling levels in the time dimension and three scale levels in the spatial dimension. Each level is uniquely determined by a specific time sampling step size and a spatial downsampling factor.

[0027] Layered sampling in the time dimension is performed as follows. Let the total number of frames in the preprocessed video sequence be T, and the original frame rate be F0. For the l-th... t Layer time sampling level, where l t =0,1,2,...L t-1 Its time sampling step size is defined as S. t (l t The step size is set according to an exponential increasing rule to capture behavioral context over longer time spans at higher levels and preserve finer temporal dynamics at lower levels. The calculation formula is as follows: S0 is the reference time step, which is a preset parameter that can be set according to the typical duration of the abnormal behavior of the target. For example, for sudden running behavior in a subway station, the reference step S0 can be set to 2 frames. The symbol indicates rounding up. For the l-th... t Layers, starting from the first frame of the video sequence, are numbered in order of length (S). t (l t Video frames are extracted evenly at intervals of 2, forming a sub-video sequence for that time level. For example, if the step size of level 0 is 2, then frames 0, 2, 4, ... are extracted; if the step size of level 1 is 4, then frames 0, 4, 8, ... are extracted; and if the step size of level 2 is 8, then frames 0, 8, 16, ... are extracted. This results in a three-level set of video sub-sequences arranged from coarse to fine in the time dimension.

[0028] The spatial downsampling is performed as follows: For each frame in each sub-video sequence obtained through the above temporal sampling, spatial downsampling is performed. Let the spatial resolution of the original frame be H×W. For the l-th... s The spatial scale level is defined by its spatial downsampling factor as D. s (l s The factor settings also follow an exponentially decreasing rule to obtain a larger receptive field of global scene information at higher levels and retain richer local detail information at lower levels. The calculation formula is as follows: , where l s =0 represents the original scale, D s (0)=1; l s =1 represents the intermediate scale, D s (1) = 1 / √2; l s =2 represents the coarsest scale, D s (2) = 1 / 2. For the lth... s The layer scales the spatial dimensions of each frame in the temporal subsequence to H'×W' using bilinear interpolation, where H' = ⌊H⋅ D. s (l s )⌋,W' = ⌊W⋅ D s (l s H×W represents the spatial resolution of the original video frame (H is the height, W is the width), D s (l s ) is the lth s The spatial downsampling factor is defined at each time level; ⌊⋅⌋ represents the floor function; H'×W' is the target spatial resolution of the downsampled video frame. Thus, for each time level, three video subsequences at different spatial scales are obtained.

[0029] After completing the temporal and spatial layering, the video subsequences under each spatiotemporal combination need to be regularly divided into spatiotemporal blocks. The spatiotemporal block is the basic unit for local feature extraction. The size of the spatiotemporal block is set to h×w×d, where h and w represent the spatial height and width of the block, and d represents the temporal depth of the block, i.e., the number of consecutive frames it contains. As a preferred implementation, h=16, w=16, and d=8 can be set. During division, on each spatiotemporal level video subsequence, a sliding window with non-overlapping or partial overlap is used to cut the video subsequence into multiple cube blocks of uniform size, i.e., spatiotemporal blocks, in a spatially h×w window size and in a temporally d window length. For the end portion that is insufficient in spatial or temporal dimensions, mirror padding is used to supplement it to the specified size. Through the above process, the original video sequence is divided from bottom to top into multiple sets of standardized spatiotemporal blocks with different spatiotemporal granularities. Temporal hierarchical sampling allows higher-level blocks to cover a longer time span, which is beneficial for capturing slow or periodic anomaly patterns; spatial hierarchical downsampling gives higher-level blocks a larger spatial view, which is beneficial for perceiving global behavioral trends. This structured multi-scale representation provides a solid foundation for subsequent robust and refined anomaly detection.

[0030] Thus, the spatiotemporal pyramid structure of the preprocessed video sequence based on preset spatiotemporal scale parameters was constructed, and a multi-scale spatiotemporal block set was obtained.

[0031] Step S103: Feature extraction is performed on the spatiotemporal blocks of each layer of the spatiotemporal pyramid. An improved three-dimensional convolutional neural network is used to extract multi-scale spatiotemporal features of each layer's spatiotemporal subsequence. An attention mechanism is used to assign weights and enhance the features at each level to obtain enhanced spatiotemporal features at each level.

[0032] After obtaining the multi-scale spatiotemporal block set, features that can effectively characterize the spatiotemporal dynamics of the video content need to be extracted from it. Since spatiotemporal blocks at different levels contain information of different granularities—higher-level blocks contain macroscopic behavioral patterns, while lower-level blocks contain microscopic action details—and abnormal behaviors may exhibit differences in specific spatiotemporal regions, specialized feature extraction and adaptive enhancement are required for each level of spatiotemporal block. This step employs an improved 3D convolutional neural network as the basic feature extractor and integrates channel and spatial dual-path attention mechanisms to adaptively enhance features crucial for anomaly detection while suppressing irrelevant background interference. Specifically, for any specified level of spatiotemporal block set in the spatiotemporal pyramid, it is fed as input into a pre-trained 3D convolutional neural network model. This 3D convolutional neural network model is an improvement on the C3D network architecture and includes multiple 3D convolutional layers, 3D pooling layers, and fully connected layers. As a preferred implementation, the improved 3D CNN used in this embodiment comprises five convolutional blocks, each consisting of a 3D convolutional layer, a batch normalization layer, and a ReLU activation function layer in sequence, with a 3D max pooling layer added after some convolutional blocks to reduce the feature map size. This network can simultaneously capture spatial appearance information and temporal motion information. First, a spatiotemporal block of size h×w×d×C is input into the network, where h and w are the spatial height and width, d is the temporal depth, and C is the number of image channels. The network performs 3D convolution and pooling operations layer by layer, ultimately outputting a high-dimensional feature vector, which initially encodes the local spatiotemporal patterns within the spatiotemporal block. However, features extracted directly from the 3D CNN treat all channels and all spatial locations equally, which is not optimal for anomaly detection tasks. This is because anomalous behavior may only be manifested by certain key feature channels or in specific spatial regions of the video. Therefore, this step integrates a lightweight dual-path attention module on the feature map output by the last convolutional layer of the basic 3D CNN. This module computes channel attention weights and spatial attention weights in parallel to recalibrate the features. The channel attention path aims to model the importance of different feature channels. For the input feature map, global average pooling and global max pooling are first performed in the spatial dimension to generate two distinct channel descriptors. Next, these two descriptors are fed into a shared multilayer perceptron (MLP), which contains a dimensionality reduction layer and an dimensionality increase layer, with a ReLU activation function in between. The two outputs of the MLP are element-wise summed, and then the final channel attention weight vector is generated using a sigmoid activation function. This process can be formally represented as follows: Where F is the input feature map, AvgPool and MaxPool represent global average pooling and global max pooling in the spatial dimension, respectively, MLP represents a shared multilayer perceptron, σ represents the sigmoid activation function, and M cThis is the calculated channel attention weight vector. This weight vector will be multiplied element-wise with the original feature map along the channel dimension to amplify the feature responses of important channels.

[0033] The spatial attention path aims to focus on important spatial locations in the feature map. Global average pooling and global max pooling are performed on the input feature map along the channel dimension, resulting in two spatial feature maps. These two spatial feature maps are then concatenated along the channel dimension to form a two-channel feature map. Next, a standard convolutional layer is applied to this feature map to reduce the number of channels to 1, followed by activation with the sigmoid function to generate the spatial attention weight matrix. This process can be formally represented as: , where [;] represents a splicing operation along the channel dimension, f 7×7 M represents a standard convolution operation with a filter size of 7x7. s This is the calculated spatial attention weight matrix. This weight matrix will be multiplied element-wise with the feature map after channel attention weighting in the spatial dimension, thereby enhancing the features of key spatial regions.

[0034] Finally, the features calibrated and enhanced by the dual-path attention mechanism are aggregated into a fixed-dimensional feature vector through a global average pooling layer, serving as the final enhanced spatiotemporal feature for that spatiotemporal block. For each layer of the spatiotemporal pyramid, the same improved 3D CNN and attention module are used to process the features of each spatiotemporal block independently. Since the resolution and time span of spatiotemporal blocks at different levels vary, those skilled in the art will understand that fine-tuning of the network input size or shallow network structure at different levels is necessary to adapt to inputs of different scales. This is a conventional transfer learning technique and will not be elaborated upon here. Through this step, each level of the spatiotemporal pyramid obtains an adaptively enhanced hierarchical feature representation that highlights anomalously related spatiotemporal patterns, providing a high-quality feature foundation for subsequent multi-level feature fusion and final decision-making.

[0035] Thus, the feature extraction and enhancement of spatiotemporal blocks at each level of the spatiotemporal pyramid were completed, resulting in enhanced spatiotemporal features at each level.

[0036] Step S104: Perform cross-level fusion of enhanced spatiotemporal features of each layer of the pyramid from top to bottom to construct a global fused spatiotemporal feature map. Input the map into the preset anomaly detection model for feature matching and anomaly probability calculation to obtain preliminary anomaly detection results.

[0037] After obtaining the enhanced spatiotemporal features at each level of the spatiotemporal pyramid, these features characterize video content at different temporal granularities and spatial scales. However, a complete anomalous behavior typically spans multiple spatiotemporal scales; for example, behavior in a subway station includes both rapid limb waving details and sustained physical confrontation postures. To comprehensively utilize information from macroscopic behavioral patterns to microscopic action details, this step employs a top-down hierarchical feature fusion strategy, gradually fusing high-level semantic information with low-level detailed information to construct a global fusion feature map containing multi-scale contextual information. This map is then input into a pre-defined anomaly detection model for decision-making.

[0038] Specifically, the top-down cross-level fusion is performed according to the following process. First, the hierarchical order of the spatiotemporal pyramid is determined. The level with the coarsest temporal sampling step and the largest spatial downsampling factor, that is, the level covering the longest time span and the widest spatial field of view, is defined as the top level. The level with the original or near-original spatiotemporal resolution is defined as the bottom level. In this embodiment, as mentioned above, both time and space are divided into three levels, denoted as L2, L1, and L0, where L2 is the top level and L0 is the bottom level.

[0039] The fusion process begins at the top layer L2. The enhanced spatiotemporal feature map of the top layer L2 obtained in step S103 is denoted as F2. Since F2 has the smallest spatial size, it needs to be upsampled first to match the spatial size of the feature map of the next layer L1. Bilinear interpolation is used to spatially upsample F2, resulting in the upsampled feature map F2_up. Next, F2_up is fused with the enhanced spatiotemporal feature map F1 from the L1 layer. Before fusion, it is necessary to ensure that F1 and F2_up have the same number of channels, which can be achieved by adjusting the number of channels in F1 using a 1x1 convolutional layer. The fusion operation uses a combination of element-wise addition and channel concatenation to preserve and enhance features from different sources. This process can be represented by the following formula: Correspondingly, ;where Conv 1×1 The first sign represents a one-to-one convolution operation, used to adjust the number of channels in the feature map; BN represents batch normalization, used to stabilize the feature distribution; the plus sign indicates element-wise addition; the semicolon indicates concatenation along the channel dimension. The resulting F... fuse1 and F' fuse1 It can be further processed by a convolutional block to generate an intermediate fused feature map F that combines L2 and L1 information. mid1 .

[0040] Then, the intermediate fused feature map F mid1 Continue to merge towards the lower L0 level. Similarly, for F... mid1Spatial upsampling is performed to match its size with the feature map F0 of layer L0. Then, the upsampled F0 is processed in the same manner as described above. mid1 The _up function is fused with F0 (after adjusting the number of channels by a 1x1 convolution). This involves first adding elements-wise, then concatenating along the channel dimension, and finally generating the final global fused spatiotemporal feature map F through a convolutional block. global This top-down process allows high-level semantic context information to propagate hierarchically and guide the integration of low-level detailed features.

[0041] The global fusion spatiotemporal feature map F is obtained. global Then, it is input into a pre-defined anomaly detection model. This model is a classification network, such as a multilayer perceptron or a lightweight convolutional neural network, pre-trained on a large number of normal and abnormal behavior videos. The input to this model is F. global The model's output consists of two parts: a prediction of the abnormal behavior category and a probability value indicating that the predicted behavior is abnormal. The model matches the input feature map with feature patterns learned during training for both normal and various abnormal behaviors, calculates the probability of each category, and outputs the category with the highest probability value and its corresponding probability as the initial anomaly detection result. For example, in a subway station scenario, the model might output "abnormal" with a probability of 0.92, or "normal" with a probability of 0.85.

[0042] Thus, a global feature rich in multi-scale information was constructed through top-down cross-level feature fusion, and preliminary detection results containing behavior category and anomaly probability were obtained through calculation by a preset anomaly detection model.

[0043] Step S105: Post-process the preliminary anomaly detection results, remove time-isolated false detection results based on the feature similarity of adjacent frames, and verify the spatiotemporal rationality of the abnormal behavior by combining the prior information of the scene to be detected. Finally, output the structured detection results containing the abnormal behavior area, type, time interval and precise spatiotemporal positioning information.

[0044] After obtaining the preliminary detection results containing anomaly categories and probabilities, the model may misjudge complex backgrounds, lighting changes, or transient interferences in the monitored scene as anomalies. This result may contain isolated noise points and false alarms that do not conform to the scene's logic. To improve the reliability and practicality of the system in real-world applications, this step performs two levels of post-processing on the preliminary results: timing consistency verification and scene logic verification. This filters out false anomalies and accurately locates and structures the true anomaly events.

[0045] Specifically, the first step is to eliminate false detections based on temporal consistency. Genuine abnormal behavior typically exhibits a certain degree of persistence over time, manifesting as related features appearing consecutively or across multiple frames; while false detections often present as isolated points in time. Let the initial detection results be a time series, where the probability of frame t being identified as exhibiting a certain type of abnormal behavior is P. t Calculate the cosine similarity between the global fused spatiotemporal feature maps of adjacent frames in the sequence to measure their temporal continuity. For frames judged as anomalous by the model, i.e., P... t For frames with a probability greater than a preset primary probability threshold η, examine the frames within a time window W before and after the abnormal frame. If the number of consecutive frames within the window before and after the abnormal frame that satisfy the probability P greater than the threshold η is less than the preset minimum number of consecutive frames K, then this abnormality is determined to be a time outlier and is removed. This judgment logic can be formalized as: for frame t, if P satisfies... t >η, but satisfies condition {P t-w , ..., P t-1 , P t+1 , ..., P t+w If the number of consecutive frames greater than η is less than K, then the anomaly detection for frame t is cancelled. The window size W can be set based on the shortest reasonable duration of the target anomaly; for example, for "sudden running" behavior, W can be the number of frames corresponding to 1 second, and K can be 3 frames. This step effectively filters out momentary false alarms caused by instantaneous changes in light and shadow or brief occlusion by objects.

[0046] Secondly, a rationality verification based on scenario prior information is conducted. Different monitoring scenarios have different definitions of "abnormal behavior." Therefore, a scenario prior rule base needs to be constructed for each scenario to be detected. Taking the subway security check area as an example, its rule base may include: abnormal behavior usually occurs in specific areas of interest, such as near the security conveyor belt or turnstiles; certain behaviors have extremely low probability within a specific time period, such as a large number of people running at high speed on the platform when no trains are arriving; "going against the flow" behavior has high anomaly value in one-way passages, but lower value in open halls. From the preliminary and time-filtered results, the spatial location, start timestamp, and behavior type of the abnormal event are extracted. This information is then matched and verified with the scenario prior rule base. For example, if the system detects "personnel lingering" in the subway track area during non-operational hours, but the prior rules indicate that the area should be completely closed and unattended at this time, this detection result is logically unreasonable and is marked as a false anomaly and removed. As another example, if the detected "rapid movement" behavior is completely along the prescribed passenger movement direction and the speed is within the normal walking range, it is corrected to normal behavior.

[0047] Finally, for abnormal events confirmed as genuine through the above two-level verification, structured information is output. This includes: 1. The type of abnormal behavior, such as running, lingering, or falling; 2. The precise time interval of the abnormality, i.e., the timestamps of the start and end frames; 3. The spatial location of the abnormal behavior in the video, usually represented by the smallest bounding rectangle that can enclose the range of movement of the abnormal subject, and can be traced back to the specific location of the original video frame through the pyramid structure constructed in step S102; 4. The confidence score of the abnormal behavior, usually the final value of the probability value output by the model after post-processing calibration. This information is integrated into a structured alarm record for security personnel to review or trigger a linkage response. Through the above post-processing process, the system's final output is no longer a raw, noisy preliminary judgment, but a precise abnormal event report with complete and reliable information, verified by both time and spatial logic.

[0048] This completes the post-processing and refinement of the preliminary anomaly detection results, outputting a highly reliable final anomaly detection result with complete spatiotemporal description information.

[0049] This method employs a spatiotemporal pyramid structure to segment videos at multiple scales, simultaneously capturing both macroscopic evolution patterns and microscopic action details of abnormal behavior, thus enhancing the comprehensiveness of behavioral representation. An improved 3DCNN integrating a dual-path attention mechanism is used for feature extraction and enhancement, adaptively focusing on key spatiotemporal features related to anomalies and effectively suppressing interference from complex backgrounds. Furthermore, a top-down cross-level feature fusion strategy organically integrates spatiotemporal contextual information of different granularities, thereby constructing more discriminative global features. Finally, a post-processing workflow combining temporal consistency verification and scene prior knowledge effectively eliminates false alarms with isolated timeframes and logically flawed detection results, ultimately outputting accurate, reliable, and structured alarms with complete spatiotemporal positioning information, significantly improving the accuracy and practical application value of abnormal behavior detection.

[0050] like Figure 2 As shown, this embodiment of the invention also provides a video abnormal behavior detection system based on a spatiotemporal pyramid, used to implement the above-mentioned video abnormal behavior detection method based on a spatiotemporal pyramid. The system includes: The video sequence preprocessing module is used to acquire continuous monitoring video sequences of the scene to be detected, perform frame synchronization preprocessing on the video sequences, remove redundant noise frames, and unify frame resolution. The spatiotemporal pyramid construction module is used to construct a spatiotemporal pyramid structure on the preprocessed video sequence based on preset spatiotemporal scale parameters, and complete the bottom-up multi-scale spatiotemporal block division. The multi-scale feature extraction and enhancement module is used to extract features from each spatiotemporal block of the spatiotemporal pyramid. It uses an improved three-dimensional convolutional neural network to extract multi-scale spatiotemporal features and uses an attention mechanism to assign weights and enhance features at each level. The cross-level feature fusion and anomaly detection module is used to perform cross-level fusion of enhanced spatiotemporal features of each layer of the pyramid from top to bottom, construct a global fused spatiotemporal feature map, and input the preset anomaly detection model for feature matching and anomaly probability calculation. The post-processing module for detection results is used to post-process the preliminary anomaly detection results, remove time-isolated false detection results based on the feature similarity of adjacent frames, and verify the spatiotemporal rationality of abnormal behavior by combining the prior information of the scene to be detected.

[0051] The video sequence preprocessing module specifically includes: The frame synchronization processing unit is used to provide a unified time reference for all surveillance cameras by deploying a network time protocol server, and to align and reassemble video frames collected by different cameras according to timestamps to obtain a set of multiple video frames that are strictly synchronized on the timeline. The redundant frame removal unit is used to calculate the gray value difference of corresponding pixels between consecutive video frames using a method based on inter-frame difference measurement, count the proportion of pixels whose differences exceed a preset threshold, and when the proportion is lower than a preset sensitivity threshold, it is determined to be a redundant noise frame and removed. The resolution unification unit is used to scale all video frames to a uniform standard resolution using a bilinear interpolation algorithm, ensuring that the input data has a regular format in the spatial dimension.

[0052] The spatiotemporal pyramid construction module specifically includes: The spatiotemporal scale parameter setting unit is used to set the sampling level Lt in the time dimension and the scale level Ls in the spatial dimension of the spatiotemporal pyramid, and to define the time sampling step size and spatial downsampling factor of each layer. The time-level sampling unit is used to set the time sampling step size of each time sampling level according to the exponential increment rule. Starting from the starting frame of the video sequence, video frames are extracted evenly with the corresponding step size to form sub-video sequences of different time levels. The spatial hierarchical downsampling unit is used to set the spatial downsampling factor for each spatial scale according to the exponential decreasing rule, and to scale each frame of the temporal hierarchical subsequence to the target spatial resolution using the bilinear interpolation algorithm. The spatiotemporal block partitioning unit is used to cut the video subsequence at each spatiotemporal level into multiple cube blocks of the same size using a sliding window method, forming a standardized set of spatiotemporal blocks.

[0053] The multi-scale feature extraction and enhancement module specifically includes: The 3DCNN feature extraction unit is used to perform three-dimensional convolution and pooling operations on the input spatiotemporal block using an improved three-dimensional convolutional neural network as the basic feature extractor, and output a high-dimensional feature vector. The dual-path attention unit is used to compute channel attention weights and spatial attention weights in parallel, recalibrate the feature map, and enhance the feature response of important channels and key spatial regions. The feature enhancement output unit is used to aggregate the features that have been calibrated and enhanced by the dual-path attention mechanism into a fixed-dimensional enhanced spatiotemporal feature vector through a global average pooling layer.

[0054] The cross-level feature fusion and anomaly detection module specifically includes: The hierarchical feature fusion unit is used to gradually fuse high-level semantic information with low-level detailed information from top to bottom. It uses bilinear interpolation for spatial upsampling and achieves cross-level feature fusion by combining element-wise addition and channel concatenation. The global feature construction unit is used to process the fused features through convolutional blocks to generate a global fused spatiotemporal feature map containing multi-scale contextual information; The anomaly detection model unit is used to input the global fused feature map into a pre-trained classification network, calculate the probability of belonging to each category through feature matching, and output preliminary detection results containing behavior category and anomaly probability.

[0055] The post-processing module for the detection results specifically includes: The temporal consistency verification unit is used to calculate the cosine similarity between features of adjacent frames, check the number of consecutive frames greater than the probability threshold within the time window before and after the abnormal frame, and remove false detection results that are temporally isolated. The scenario prior verification unit is used to build a scenario prior rule base, match and verify the spatial location, start timestamp, and behavior type of abnormal events with the rule base, and remove detection results that do not conform to the scenario logic; The structured output unit is used to output structured detection results for confirmed abnormal events, including the type of abnormal behavior, the precise time interval, spatial location information, and confidence score.

[0056] This system embodiment ensures the quality and regularity of the input video data through a preprocessing module, laying a solid foundation for subsequent analysis. The collaborative work of the spatiotemporal pyramid construction module and the feature extraction enhancement module enables automated and refined extraction of multi-scale spatiotemporal features from the video content. The cross-level fusion and detection module is responsible for effectively aggregating scattered feature information and transforming it into preliminary decisions. In particular, the independent post-processing module for detection results improves the reliability and credibility of the system output through integrated verification rules. This systematic implementation enables the entire detection process to run stably and automatically, making it very suitable for large-scale monitoring scenarios such as subway stations. It enables real-time and accurate monitoring and alarm of abnormal behavior, greatly enhancing the intelligence level and practical effectiveness of the security system.

[0057] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0058] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A video anomaly behavior detection method based on spatiotemporal pyramids, characterized in that, include: Step S101: Obtain a continuous monitoring video sequence of the scene to be detected, perform frame synchronization preprocessing on the video sequence, remove redundant noise frames and unify the frame resolution; Step S102: Based on preset spatiotemporal scale parameters, a spatiotemporal pyramid structure is constructed on the preprocessed video sequence. The spatiotemporal pyramid includes multiple sampling levels in the time dimension and multiple scale levels in the spatial dimension. Normalized spatiotemporal blocks are obtained by hierarchical sampling and spatial downsampling, thus completing the bottom-up multi-scale spatiotemporal block division. Step S103: Feature extraction is performed on the spatiotemporal blocks of each layer of the spatiotemporal pyramid. An improved three-dimensional convolutional neural network based on the C3D network architecture, which includes multiple three-dimensional convolutional layers, three-dimensional pooling layers and fully connected layers, and integrates a dual-path attention module that calculates channel attention weights and spatial attention weights in parallel on the feature map output by the last convolutional layer, is used to extract multi-scale spatiotemporal features of each layer's spatiotemporal subsequence. The attention mechanism is used to assign weights and enhance the features at each level to obtain enhanced spatiotemporal features at each level. Step S104: Perform cross-level fusion of enhanced spatiotemporal features of each layer of the pyramid from top to bottom to construct a global fused spatiotemporal feature map. Input the map into the preset anomaly detection model for feature matching and anomaly probability calculation to obtain preliminary anomaly detection results. Step S105: Post-process the preliminary anomaly detection results, remove time-isolated false detection results based on the feature similarity of adjacent frames, and verify the spatiotemporal rationality of the abnormal behavior by combining the prior information of the scene to be detected. Finally, output the structured detection results containing the abnormal behavior area, type, time interval and precise spatiotemporal positioning information.

2. The method as described in claim 1, characterized in that, Step S101, which involves frame synchronization preprocessing of the video sequence, includes: By deploying a network time protocol server to provide a unified time reference for all surveillance cameras, each video frame is stamped with a high-precision timestamp based on this unified time reference during video acquisition. Based on the timestamp, the video frames acquired by different cameras are aligned and recombined to obtain a set of multiple video frames that are strictly synchronized on the timeline. The synchronized continuous monitoring video sequence is subjected to redundant noise frame removal operation. The method based on inter-frame difference measurement is used to calculate the gray value difference of corresponding pixels between consecutive video frames, and to count the proportion of pixels with differences exceeding a preset threshold to the total number of pixels. When the proportion is lower than the preset sensitivity threshold, the next frame is determined to be a redundant noise frame and is removed. The frame resolution of the deredundant video sequence is unified by using a bilinear interpolation algorithm to scale all video frames to a uniform standard resolution.

3. The method as described in claim 1, characterized in that, Step S102, which involves constructing the spacetime pyramid structure, includes: The spacetime pyramid is defined as containing L in the time dimension. t Each sampling level contains L in the spatial dimension. s Each scale level, where L t =3,L s =3; For the lth in the time dimension t Layer sampling level, where l t =0,1,2, defining the time sampling step S. t (l t Following the exponential increasing rule, the calculation formula is as follows: S0 is the base time step, set to 2 frames, starting from the first frame of the video sequence and incrementing by S. t (l t The video frames are extracted at uniform intervals to form a sub-video sequence of the time level after hierarchical sampling in this time dimension; For the lth spatial dimension s Layer scale level, where l s =0,1,2, define the spatial downsampling factor D. s (l s Following the exponentially decreasing rule, each frame of the temporal subsequence is scaled to the target spatial resolution using a bilinear interpolation algorithm; The video subsequence under each spatiotemporal combination is divided according to rules. The size of the spatiotemporal block is set as h×w×d, where h=16, w=16, and d=8. The video subsequence is cut into multiple spatiotemporal blocks of the same size using a sliding window method.

4. The method as described in claim 1, characterized in that, Step S103, which uses an improved 3D convolutional neural network for feature extraction, includes: The spatiotemporal block is input into a pre-trained 3D convolutional neural network model, which is an improvement on the C3D network architecture and includes multiple 3D convolutional layers, 3D pooling layers and fully connected layers. A dual-path attention module is integrated on the feature map output by the last convolutional layer to compute channel attention weights and spatial attention weights in parallel. The channel attention path generates channel descriptors through global average pooling and global max pooling. After being input into a shared multilayer perceptron, it is activated by Sigmoid to generate channel attention weight vectors, which are then multiplied element-wise with the original feature map in the channel dimension. The spatial attention path obtains spatial feature maps through global average pooling and global max pooling along the channel dimension. After concatenation, it generates a spatial attention weight matrix through 7×7 convolution and Sigmoid activation. The matrix is ​​then multiplied element-wise with the channel attention-weighted feature map in the spatial dimension. The calibrated and enhanced features are aggregated into a fixed-dimensional enhanced spatiotemporal feature vector through global average pooling.

5. The method as described in claim 1, characterized in that, Step S104, top-down cross-level fusion, includes: The enhanced spatiotemporal feature map F2 of the top layer L2 of the spatiotemporal pyramid is spatially upsampled and fused with the feature map F1 of the next layer L1. Before fusion, the number of channels of F1 is adjusted by 1×1 convolution. The fusion operation includes element-wise addition and channel concatenation to generate an intermediate fused feature map F. mid1 ; F mid1 Spatial upsampling is performed and fused with the feature map F0 of the underlying L0. Similarly, the number of channels is adjusted through 1×1 convolution. After addition and concatenation, a global fused spatiotemporal feature map F is generated through a convolutional block. global ; F global Input a preset anomaly detection model, which is a classification network pre-trained on normal and abnormal behavior videos, and output the abnormal behavior category and probability as the preliminary detection result.

6. The method as described in claim 1, characterized in that, Post-processing in step S105 includes: Based on time consistency verification, the cosine similarity between the global fusion spatiotemporal feature maps of adjacent frames is calculated. For abnormal frames with a probability Pt greater than the primary probability threshold, the number of consecutive frames greater than the probability threshold η within the time window W before and after the frame is checked. If the number of consecutive frames is less than the minimum number of continuous frames K, the abnormal judgment corresponding to the frame is removed. Based on scenario prior verification, a scenario prior rule base is constructed. The spatial region, start timestamp, and behavior type of abnormal events are matched with the rule base to eliminate detection results that do not conform to logic. The system outputs structured results for real-world abnormal events, including the type of abnormal behavior, the precise time interval, the spatial location bounding box, and the confidence score.

7. The method as described in claim 2, characterized in that, In the redundant noise frame removal operation, the preset sensitivity threshold is set to 0.001; in the frame resolution unification, the standard resolution is set to 640x480 pixels.

8. A video anomaly behavior detection system based on a spatiotemporal pyramid, used to implement the method described in any one of claims 1-7, characterized in that the system include: The video sequence preprocessing module is used to acquire continuous monitoring video sequences of the scene to be detected, perform frame synchronization preprocessing on the video sequences, remove redundant noise frames, and unify frame resolution. The spatiotemporal pyramid construction module is used to construct a spatiotemporal pyramid structure on the preprocessed video sequence based on preset spatiotemporal scale parameters, and complete the bottom-up multi-scale spatiotemporal block division. The multi-scale feature extraction and enhancement module is used to extract features from each spatiotemporal block of the spatiotemporal pyramid. It uses an improved three-dimensional convolutional neural network to extract multi-scale spatiotemporal features and uses an attention mechanism to assign weights and enhance features at each level. The cross-level feature fusion and anomaly detection module is used to perform cross-level fusion of enhanced spatiotemporal features of each layer of the pyramid from top to bottom, construct a global fused spatiotemporal feature map, and input the preset anomaly detection model for feature matching and anomaly probability calculation. The post-processing module for detection results is used to post-process the preliminary anomaly detection results, remove time-isolated false detection results based on the feature similarity of adjacent frames, and verify the spatiotemporal rationality of abnormal behavior by combining the prior information of the scene to be detected.

9. The system as described in claim 8, characterized in that, The video sequence preprocessing module includes: The frame synchronization processing unit is used to provide a unified time reference for all surveillance cameras by deploying a network time protocol server, and to align and reassemble video frames collected by different cameras according to timestamps to obtain a set of multiple video frames that are strictly synchronized on the timeline. The redundant frame removal unit is used to calculate the gray value difference of corresponding pixels between consecutive video frames using a method based on inter-frame difference measurement, count the proportion of pixels whose differences exceed a preset threshold, and when the proportion is lower than a preset sensitivity threshold, it is determined to be a redundant noise frame and removed. The resolution unification unit is used to scale all video frames to a uniform standard resolution using a bilinear interpolation algorithm; The spatiotemporal pyramid construction module includes: The spatiotemporal scale parameter setting unit is used to set the sampling level Lt in the time dimension and the scale level Ls in the spatial dimension of the spatiotemporal pyramid, and to define the time sampling step size and spatial downsampling factor of each layer. The time-level sampling unit is used to set the time sampling step size of each time sampling level according to the exponential increment rule. Starting from the starting frame of the video sequence, video frames are extracted evenly with the corresponding step size to form sub-video sequences of different time levels. The spatial hierarchical downsampling unit is used to set the spatial downsampling factor for each spatial scale according to the exponential decreasing rule, and to scale each frame of the temporal hierarchical subsequence to the target spatial resolution using the bilinear interpolation algorithm. The spatiotemporal block partitioning unit is used to divide the video subsequence at each spatiotemporal level into multiple sets of spatiotemporal blocks of the same size using a sliding window method.

10. The system as described in claim 8, characterized in that, The multi-scale feature extraction and enhancement module includes: The 3DCNN feature extraction unit is used to perform three-dimensional convolution and pooling operations on the input spatiotemporal block using an improved three-dimensional convolutional neural network as the basic feature extractor, and output a high-dimensional feature vector. The dual-path attention unit is used to compute channel attention weights and spatial attention weights in parallel, recalibrate the feature map, and enhance the feature response of important channels and key spatial regions. The feature enhancement output unit is used to aggregate the features that have been calibrated and enhanced by the dual-path attention mechanism into a fixed-dimensional enhanced spatiotemporal feature vector through a global average pooling layer. The cross-level feature fusion and anomaly detection module includes: The hierarchical feature fusion unit is used to gradually fuse high-level semantic information with low-level detailed information from top to bottom. It uses bilinear interpolation for spatial upsampling and achieves cross-level feature fusion by combining element-wise addition and channel concatenation. The global feature construction unit is used to process the fused features through convolutional blocks to generate a global fused spatiotemporal feature map containing multi-scale contextual information; The anomaly detection model unit is used to input the global fused feature map into a pre-trained classification network, calculate the probability of belonging to each category through feature matching, and output preliminary detection results containing behavior category and anomaly probability. The post-processing module for the detection results includes: The temporal consistency verification unit is used to calculate the cosine similarity between features of adjacent frames, check the number of consecutive frames greater than the probability threshold within the time window before and after the abnormal frame, and remove false detection results that are temporally isolated. The scenario prior verification unit is used to build a scenario prior rule base, match and verify the spatial location, start timestamp, and behavior type of abnormal events with the rule base, and remove detection results that do not conform to the scenario logic; The structured output unit is used to output structured detection results for confirmed abnormal events, including the type of abnormal behavior, the precise time interval, spatial location information, and confidence score.

Images