An edge computing-based video stream abnormal behavior detection method

Abstract

The application discloses a video stream abnormal behavior detection method based on edge computing, comprising the following steps: S1, video stream preprocessing and lightweight adaptation; through video stream preprocessing and lightweight adaptation, a'shallow lightweight + deep dynamic' feature extraction architecture and power dynamic scheduling, the application meets the low delay and lightweight deployment requirements of the edge node, avoids the decrease of detection accuracy caused by lightweight processing, reduces redundant calculation when multiple video streams are concurrently processed, strengthens the modeling capability of continuous long-time abnormal behavior chain through the combination of lightweight time sequence attention and behavior chain associated modeling, reduces the missed detection and false detection rate in the crowd dense and target shielding scene, and further improves the detection reliability and the performance consistency of multiple nodes by relying on edge-cloud collaborative review, multi-node collaborative optimization and incremental learning mechanism, and the application considers privacy protection and deployment flexibility, and can effectively adapt to the actual application requirements of large-scale video monitoring.

Description

Technical Field

[0001] This invention relates to the field of abnormal behavior detection technology in video streams, and in particular to a method for detecting abnormal behavior in video streams based on edge computing. Background Technology

[0002] With the widespread deployment of video surveillance systems, abnormal behavior detection in video streams has become a key technology for ensuring public safety and improving management efficiency. Edge computing technology, with its advantages of low latency and high privacy, is gradually replacing the traditional cloud-based centralized detection mode and becoming the mainstream architecture for abnormal video stream detection.

[0003] However, existing edge computing-based video stream abnormal behavior detection methods still have the following shortcomings, making it difficult to meet the needs of practical applications: 1. There is an irreconcilable contradiction between computing power, accuracy, and real-time performance: The computing power, memory, and power consumption of edge devices are strictly limited. High-precision anomaly detection models have a large number of parameters and high computational complexity, making them impossible to deploy directly on edge devices. If the model is pruned or quantized to reduce its weight, the temporal modeling capability and feature expression depth will be sacrificed, resulting in a significant increase in the false negative rate and false positive rate of complex abnormal behaviors and long-tailed anomalies. At the same time, when multiple video streams are running concurrently, redundant computation will further exacerbate the performance degradation. 2. Weak ability to capture long temporal dependencies: Most existing methods only focus on single-frame or short-temporal feature extraction, making it difficult to model continuous long-term abnormal behavior chains such as "holding a weapon → hitting → falling down" and "loitering → gathering → conflict". It is easy to split continuous anomalies into isolated frames for misjudgment or false negatives. Especially in dense crowds and target occlusion scenarios, the temporal consistency judgment is prone to failure. In summary, this application proposes a video stream abnormal behavior detection method based on edge computing. Summary of the Invention

[0004] Based on the technical problems existing in the background technology, this invention proposes a video stream abnormal behavior detection method based on edge computing.

[0005] This invention proposes a video stream abnormal behavior detection method based on edge computing, comprising the following steps: S1: Video stream preprocessing and lightweight adaptation: The edge node receives the raw video stream captured by the front-end camera, performs adaptive frame interval sampling, size normalization, illumination compensation, noise suppression and lightweight feature enhancement on the video stream, removes redundant data and improves the contrast of target features. S2: Hierarchical feature extraction and dynamic computing power scheduling: The feature extraction architecture of "shallow lightweight + deep dynamic" is adopted. The basic texture features of video frames are extracted through shallow lightweight network. The deep feature branches are dynamically activated according to the complexity of shallow features to extract fine-grained temporal features and context-related features. At the same time, the edge nodes monitor their own computing power load in real time and dynamically schedule computing power resources to balance detection accuracy and real-time performance. S3: Long Temporal Dependency Modeling and Feature Fusion: This method combines a lightweight temporal attention mechanism with behavioral chain association modeling to capture long temporal dependencies in video streams, extract temporal difference features, and fuse hierarchical features, temporal attention features, and temporal difference features to output a fused feature vector. S4: Real-time anomaly detection and cloud verification at the edge: A dual threshold detection mechanism is adopted to detect anomalies in the fused feature vector. High-confidence anomalies immediately trigger local alarms and are synchronized to the cloud. Suspicious anomalies are temporarily stored for cloud verification. The cloud performs high-precision verification of suspicious anomalies. At the same time, based on the labeled data uploaded by the edge nodes, the incremental learning algorithm is used to optimize the relevant models of the edge nodes. S5: Multi-edge node collaboration and dynamic updates: Multiple edge nodes achieve data sharing and collaborative detection through the edge gateway. The cloud aggregates multi-node detection logs and performs global model optimization to ensure consistent multi-node detection performance.

[0006] Preferably, in step S1, the adaptive frame interval sampling strategy is dynamically adjusted according to the video stream frame rate: when the frame rate is ≥30fps, the sampling interval is 2-3 frames; when the frame rate is <30fps, the sampling interval is 1 frame. Illumination compensation adopts the CLAHE algorithm, noise suppression adopts Gaussian filtering combined with median filtering, and lightweight feature enhancement is based on the feature enhancement branch of MobileNetV4-Lite. The specific logical steps are as follows: S101: The edge node receives the raw video stream captured by the front-end camera and parses it into a continuous video frame sequence: Simultaneously, the sampling interval and normalized target size basic parameters are initialized to adapt to the edge node computing power configuration, where T is the total number of frames in the original video stream. This is the T-th frame of the video image; S102: Reads the raw video stream frame rate via the edge node's built-in interface. The sampling interval k is dynamically determined based on the frame rate, and the formula is as follows: , in The number of frames per second in the original video stream. The sampling interval is used to further sample the frame sequence. Sampling yields an effective frame sequence. m is the number of valid frames after sampling, which completes the removal of redundant data; S103: Using bilinear interpolation, each sampled frame... Scale to target size The formula used is: Coordinate mapping: ; Pixel value calculation: ; in , To normalize the target width and height, , The sampled frame width and height. , These are the normalized frame coordinates. , These are floating-point coordinates mapped to the sampled frame. For interpolation weights, Rounding down x yields the normalized frame sequence. ,in This is the normalized image of the m-th frame; S104: The CLAHE algorithm is used to normalize each frame. The formula used for illumination compensation is: Sub-block histogram calculation: ; Contrast Limitation: ; Grayscale mapping: ; in Sub-blocks for normalized frames For sub-block index, w and h are the width and height of the sub-block, t is the grayscale value, and I ( ) is an indicator function. This is the grayscale histogram of the sub-blocks. The contrast threshold. This is the grayscale mapping function; The illumination-compensated frame sequence is obtained after bilinear interpolation fusion. ,in This is the image after illumination compensation for the m-th frame; S105: To remove interference from rain, snow, dust, and electronic noise in video frames, and to meet the low-latency requirements of edge nodes, a combined filtering method of "Gaussian filtering + median filtering" is adopted. First, Gaussian noise is removed by Gaussian filtering, and then impulse noise is removed by median filtering, finally obtaining the noise-suppressed video frame sequence. ; S106: Feature enhancement is achieved based on MobileNetV4-Lite depthwise separable convolutions, using the following formula: Depthwise convolution: ; Point convolution: ; ReLU6 activation: ; Where c is the number of input channels. Number of output channels , These are the depthwise convolution kernel and the bias, respectively. , These are the point convolution kernel and the bias, respectively. , These output feature maps for depthwise convolution and pointwise convolution, respectively. Finally, a preprocessed frame sequence adapted for edge computing is obtained. ,in This is the final frame after feature enhancement.

[0007] Preferably, in step S2, the shallow lightweight network adopts MobileNetV4-Lite with the number of parameters controlled within 5M, and the deep feature branch adopts a lightweight Transformer structure with the number of parameters controlled within 8M. The triggering condition for dynamic scheduling of computing power is as follows: when the edge node CPU utilization is ≥70% or the memory occupancy is ≥65%, the inference frequency of the deep feature branch is reduced; when the CPU utilization is <70% and the memory occupancy is <65%, the normal inference frequency is restored. The specific logical steps are as follows: S201: Receive the preprocessed final frame sequence: Configure the core parameters of shallow and deep networks and set the feature complexity threshold. 1. Set computing load threshold, initialize computing scheduling parameters, and adapt to edge node computing power limitations; Where m is the number of valid frames and C is the number of input channels; S202: MobileNetV4-Lite is used as a shallow, lightweight network. Basic texture features are extracted through depthwise separable convolutions. The formula used is as follows: ; in For the shallow feature map of frame i in coordinates (x, y) and channels eigenvalues ​​at that location ( ) is a lightweight activation function. It is a shallow 3×3 convolution kernel. The preprocessed image of the i-th frame in coordinates The pixel value at channel c. This is a shallow convolution bias. , This is the kernel offset; And output shallow feature maps: ; S203: Complexity is evaluated by combining feature information entropy and variance. The core formula is: Overall complexity assessment value: ; Dynamic activation judgment: ; in Let be the evaluation value of the shallow feature synthesis complexity of the i-th frame. =0.6, The shallow feature information entropy of the i-th frame is... , The minimum and maximum values ​​of the feature information entropy are... Let V be the shallow feature variance of the i-th frame. , These represent the minimum and maximum values ​​of the feature variance. Activate=1 indicates activation of deep branches, and Activate=0 indicates inactivation. S204: When Activate=1, the lightweight Transformer branch is activated. Taking shallow features as input, it extracts fine-grained temporal and contextual features through a self-attention mechanism. The formula used is: , in These are the query matrix, key matrix, and value matrix, respectively. For the attention head dimension, Scaling factor ( () is the normalization function, which outputs the deep feature map. When Activate=0, ; S205: Unifies and fuses the feature dimensions of shallow and deep layers through 1×1 convolution; the core formula is... ; in The fused feature map of the i-th frame is located at coordinates (x, y) and channels. eigenvalues ​​at that location This represents the number of feature channels after fusion. , These are 1×1 fusion convolutional kernels representing shallow and deep features, respectively. To fuse convolutional biases; Output fusion features: ; S206: Real-time sampling and monitoring of CPU utilization and memory usage, using the following formula: , ; in Let be the CPU utilization at time t. Let t be the CPU busy time. =50ms is the computing load sampling period. This represents the current memory usage. This represents the amount of memory already used. This represents the total memory size of the edge nodes; S207: Adjust the deep branch inference frequency according to the load, using the following formula: ; in Let be the inference frequency of the deep feature branch at time t. This represents the normal inference frequency of deep feature branches. The frequency adjustment factor is 70% and 65%, which are the load thresholds for CPU utilization and memory usage, respectively.

[0008] Preferably, the specific logical steps of S3 are as follows: S301: Receive the fused feature sequence output after hierarchical feature fusion. Where m is the number of valid frames, , , To fuse the width and height of the feature maps, To integrate the number of feature channels, we configure a lightweight temporal attention mechanism and core parameters for behavioral chain association modeling, set the temporal difference window size and behavioral chain template parameters, initialize feature fusion weights, adapt to the low computing power requirements of edge nodes, and ensure that the modeling and fusion process is efficient and has low latency. S302: Based on the fusion features of adjacent frames, temporal difference features are extracted to capture the motion change trend of targets in the video stream. The formula is as follows: ,in Let be the temporal difference feature map of the i-th frame. For the fused features of the i-th frame, The fused features are for the (i-1)th frame; And output the temporal difference feature sequence ; S303: Employs a lightweight temporal attention mechanism based on GRU to model the fused feature sequence, focusing on key frame features and capturing long-term temporal dependencies between frames. The formula used is as follows: ; ; ; in This represents the hidden state output by the GRU in the i-th frame. ( (This is a lightweight gated loop unit.) This is the hidden state from the previous frame. Let w be the attention weight for the i-th frame, w be the attention weight matrix, and b be the bias. This is the temporal attention feature vector; S304: Based on a pre-defined template of common abnormal behavior chains, association modeling is performed on temporal attention features to further strengthen long-term temporal dependencies. The formula used is as follows: ; ; in For the t-th preset behavior chain template, cos( () is the cosine similarity function. To achieve the maximum matching similarity, Feature vectors after modeling behavioral chain associations; S305: The hierarchical fusion features, temporal attention features, and temporal difference features are fused along a unified dimension to output the final fused feature vector. The formula is as follows: ; in The final output is the fused feature vector. This is a vector flattened from the hierarchical fusion features. This is the flattened vector of temporal difference features. , , These are the fusion weight matrices for the three types of features. For fusion bias.

[0009] Preferably, the specific logical steps of S4 are as follows: S401: Receives the final fused feature vector output after long-term dependency modeling and feature fusion. Where m is the number of valid frames, , For the final fusion feature dimension; High threshold with dual threshold judgment mechanism Low threshold Initialize the local alarm threshold and suspicious anomaly temporary storage cache at the edge, set the parameters of the high-precision review model and the core parameters of the incremental learning algorithm in the cloud, and ensure that the real-time performance of the edge and the review accuracy of the cloud are taken into account, so as to adapt to the edge-cloud collaborative architecture. S402: A lightweight SVM classifier is used to calculate the anomaly confidence score of the fused feature vector for each frame, combined with a dual threshold mechanism to determine the anomaly type. The formula used is as follows: ; ; in Let SVM be the anomaly confidence of the fused feature vector of the i-th frame. This is a lightweight support vector machine classifier. For high confidence threshold, The low confidence threshold The anomaly detection type for the i-th frame; S403: Based on the exception judgment result of S402, execute the corresponding processing logic, as follows: S4031: If it is a high-confidence anomaly: immediately trigger a local alarm at the edge, and simultaneously fuse the feature vector of this frame. Anomaly confidence level C The frame images are synchronized to the cloud for cloud recording and backup. S4032: If it is a suspicious anomaly: temporarily store the frame's fused feature vector, anomaly confidence, and frame image in the local cache at the edge, and periodically upload them to the cloud in batches, waiting for cloud review; S4033: If normal: No alarm or upload operation will be performed; only the detection log will be recorded for data accumulation for subsequent incremental learning. S404: The cloud receives suspicious anomaly data uploaded from the edge device, uses a 3D CNN+Transformer high-precision model for verification, and recalculates the anomaly confidence level. The formula used is as follows: ;

[0010] in This represents the confidence level for anomalies after cloud-based verification. ( This is a high-precision verification model in the cloud. For the context features of this frame, This is the final judgment result for any suspicious anomalies; The cloud will synchronize the review results back to the edge device, and the edge device will update its local detection records. S405: The edge end regularly uploads the fused feature vectors and annotation results corresponding to high-confidence anomalies and anomalies confirmed by the cloud to the cloud; based on the uploaded annotation data, the cloud uses an incremental learning algorithm to optimize the anomaly judgment model and feature extraction related models of the edge nodes.

[0011] Preferably, the specific logical steps of S5 are as follows: S501: Deploy multiple edge nodes, each establishing a communication connection through an edge gateway, completing the initialization of the collaboration protocol, configuring the edge gateway's data transmission rate and data sharing range, and setting the global model optimization cycle. The multi-node detection performance consistency threshold Δ is initialized to initialize the global model parameters in the cloud and the local model parameters of each edge node, adapting to the multi-edge-cloud collaborative architecture and balancing data transmission efficiency and detection real-time performance. S502: Each edge node uploads its own detection logs to the edge gateway in real time. The edge gateway performs unified aggregation, deduplication, and format standardization of data from multiple nodes to achieve data sharing among multiple nodes. S503: Each edge node performs consistency calibration on its local detection results based on standardized data shared by the edge gateway, correcting its own detection bias. The formula used is: ,in The anomaly confidence level of the k-th edge node after calibration in the i-th frame. Let the original anomaly confidence of the k-th edge node in the i-th frame be . Let be the average anomaly confidence score of all edge nodes in the i-th frame. =0.1-0.2 is a calibration coefficient used to balance local detection accuracy and multi-node consistency; After calibration, each edge node synchronously updates its local detection results to ensure consistent detection trends across multiple nodes; S504: The cloud periodically receives multi-node detection logs uploaded by the edge gateway, performs a global evaluation of the multi-node detection performance, and optimizes the global model using a federated averaging algorithm based on the aggregated detection logs and performance evaluation results. The formula used is as follows: ; ; in , For the optimized cloud-based global model weights and biases, , For the k-th edge node, the old weights and biases of the local model are given. The global learning rate, The gradient of the global loss function. Annotated detection data aggregated from multiple nodes; After optimization, the cloud distributes the global lightweight model to each edge node, and each node synchronously updates its local model parameters to ensure consistent detection performance across multiple nodes; simultaneously, the edge gateway updates the global mean of data sharing. Compared with global standard deviation This forms a collaborative optimization closed loop.

[0012] Preferably, in S3, the number of preset abnormal behavior chain templates K is 8-12, covering common abnormal behaviors such as falling, running, gathering, and climbing. The number of GRU units is set to 64. The temporal attention feature dimension Cattn is consistent with the number of fused feature channels Cf. Finally, the fused feature vector dimension Cfinal is set to 128 dimensions to adapt to the low computing power inference and subsequent anomaly judgment requirements at the edge.

[0013] Preferably, in S5, the global model optimization cycle The time limit is set to 1-2 hours, the multi-node detection performance consistency threshold Δ=5%, the data transmission rate of the edge gateway is controlled at 10-20Mbps, the data sharing scope only includes the detection results, standardized features and performance logs of each edge node, and the original video frames are not transmitted to reduce the data transmission bandwidth usage and adapt to low bandwidth scenarios of edge nodes.

[0014] Compared with existing technologies, the beneficial effects of this invention are: 1. By using lightweight adaptation of video stream preprocessing, a “shallow lightweight + deep dynamic” feature extraction architecture and dynamic scheduling of computing power, we can ensure low-latency deployment of edge nodes while avoiding the decrease in detection accuracy caused by lightweight processing, reducing redundant calculations when multiple video streams are running concurrently, and balancing lightweight deployment of edge devices with anomaly detection accuracy. 2. By combining lightweight temporal attention with behavioral chain association modeling, the modeling capability of continuous long-term abnormal behavior chains such as "holding a weapon → striking → falling" is enhanced, avoiding the misjudgment of splitting continuous anomalies into isolated frames. It can still maintain good temporal consistency judgment in dense crowds and target occlusion scenarios. At the same time, by combining edge-cloud collaborative verification, multi-node collaborative optimization and incremental learning, the detection reliability and multi-node performance consistency are further improved, taking into account privacy protection and deployment flexibility, and adapting to the actual needs of large-scale video surveillance. This invention, through lightweight adaptation of video stream preprocessing, a "shallow lightweight + deep dynamic" feature extraction architecture, and dynamic scheduling of computing power, meets the requirements of low latency and lightweight deployment of edge nodes while avoiding the decrease in detection accuracy caused by lightweight processing, reducing redundant computation when multiple video streams are running concurrently. By combining lightweight temporal attention with behavioral chain association modeling, it enhances the modeling capability of continuous long-term abnormal behavior chains, reducing the false negative and false positive rates in densely populated and target-occluded scenarios. At the same time, relying on edge-cloud collaborative verification, multi-node collaborative optimization, and incremental learning mechanisms, it further improves detection reliability and multi-node performance consistency, taking into account both privacy protection and deployment flexibility, and can effectively adapt to the actual application needs of large-scale video surveillance. Attached Figure Description

[0015] Figure 1 This is a flowchart of a video stream abnormal behavior detection method based on edge computing proposed in this invention. Detailed Implementation

[0016] The present invention will be further explained below with reference to specific embodiments.

[0017] Example Reference Figure 1 This embodiment proposes a video stream abnormal behavior detection method based on edge computing, including the following steps: S1: Video stream preprocessing and lightweight adaptation: The edge node receives the raw video stream captured by the front-end camera, performs adaptive frame interval sampling, size normalization, illumination compensation, noise suppression and lightweight feature enhancement on the video stream, removes redundant data and improves the contrast of target features. The adaptive frame interval sampling strategy is dynamically adjusted according to the video stream frame rate: when the frame rate is ≥30fps, the sampling interval is 2-3 frames; when the frame rate is <30fps, the sampling interval is 1 frame. Illumination compensation adopts the CLAHE algorithm, noise suppression adopts Gaussian filtering combined with median filtering, and lightweight feature enhancement is based on the feature enhancement branch of MobileNetV4-Lite. The specific logical steps are as follows: S101: The edge node receives the raw video stream captured by the front-end camera and parses it into a continuous video frame sequence: Simultaneously, the sampling interval and normalized target size basic parameters are initialized to adapt to the edge node computing power configuration, where T is the total number of frames in the original video stream. This is the T-th frame of the video image; S102: Reads the raw video stream frame rate via the edge node's built-in interface. The sampling interval k is dynamically determined based on the frame rate, and the formula is as follows: , in The number of frames per second in the original video stream. The sampling interval is used to further sample the frame sequence. Sampling yields an effective frame sequence. m is the number of valid frames after sampling, which completes the removal of redundant data; S103: Using bilinear interpolation, each sampled frame... Scale to target size The formula used is: Coordinate mapping: ; Pixel value calculation: ; in , To normalize the target width and height, , The sampled frame width and height. , These are the normalized frame coordinates. , These are floating-point coordinates mapped to the sampled frame. For interpolation weights, Rounding down x yields the normalized frame sequence. ,in This is the normalized image of the m-th frame; S104: The CLAHE algorithm is used to normalize each frame. The formula used for illumination compensation is: Sub-block histogram calculation: ; Contrast Limitation: ; Grayscale mapping: ; in Sub-blocks for normalized frames For sub-block index, w and h are the width and height of the sub-block, t is the grayscale value, and I ( ) is an indicator function. This is the grayscale histogram of the sub-blocks. The contrast threshold. This is the grayscale mapping function; The illumination-compensated frame sequence is obtained after bilinear interpolation fusion. ,in This is the image after illumination compensation for the m-th frame; S105: To remove interference from rain, snow, dust, and electronic noise in video frames, and to meet the low-latency requirements of edge nodes, a combined filtering method of "Gaussian filtering + median filtering" is adopted. First, Gaussian noise is removed by Gaussian filtering, and then impulse noise is removed by median filtering, finally obtaining the noise-suppressed video frame sequence. ; S106: Feature enhancement is achieved based on MobileNetV4-Lite depthwise separable convolutions, using the following formula: Depthwise convolution: ; Point convolution: ; ReLU6 activation: ; Where c is the number of input channels. Number of output channels , These are the depthwise convolution kernel and the bias, respectively. , These are the point convolution kernel and the bias, respectively. , These output feature maps for depthwise convolution and pointwise convolution, respectively. Finally, a preprocessed frame sequence adapted for edge computing is obtained. ,in This is the final frame after feature enhancement; S2: Hierarchical feature extraction and dynamic computing power scheduling: The feature extraction architecture of "shallow lightweight + deep dynamic" is adopted. The basic texture features of video frames are extracted through shallow lightweight network. The deep feature branches are dynamically activated according to the complexity of shallow features to extract fine-grained temporal features and context-related features. At the same time, the edge nodes monitor their own computing power load in real time and dynamically schedule computing power resources to balance detection accuracy and real-time performance. The shallow lightweight network uses MobileNetV4-Lite with fewer than 5M parameters, while the deep feature branches use a lightweight Transformer structure with fewer than 8M parameters. The dynamic scheduling of computing power is triggered as follows: when the edge node CPU utilization is ≥70% or the memory usage is ≥65%, the inference frequency of the deep feature branches is reduced; when the CPU utilization is <70% and the memory usage is <65%, the normal inference frequency is restored. The specific logical steps are as follows: S201: Receive the preprocessed final frame sequence: Configure the core parameters of shallow and deep networks and set the feature complexity threshold. 1. Set computing load threshold, initialize computing scheduling parameters, and adapt to edge node computing power limitations; Where m is the number of valid frames and C is the number of input channels; S202: MobileNetV4-Lite is used as a shallow, lightweight network. Basic texture features are extracted through depthwise separable convolutions. The formula used is as follows: ; in For the shallow feature map of frame i in coordinates (x, y) and channels eigenvalues ​​at that location ( ) is a lightweight activation function. It is a shallow 3×3 convolution kernel. The preprocessed image of the i-th frame in coordinates The pixel value at channel c. This is a shallow convolution bias. , This is the kernel offset; And output shallow feature maps: ; S203: Complexity is evaluated by combining feature information entropy and variance. The core formula is: Overall complexity assessment value: ; Dynamic activation judgment: ; in Let be the evaluation value of the shallow feature synthesis complexity of the i-th frame. =0.6, The shallow feature information entropy of the i-th frame is... , The minimum and maximum values ​​of the feature information entropy are... Let V be the shallow feature variance of the i-th frame. , These represent the minimum and maximum values ​​of the feature variance. Activate=1 indicates activation of deep branches, and Activate=0 indicates inactivation. S204: When Activate=1, the lightweight Transformer branch is activated. Taking shallow features as input, it extracts fine-grained temporal and contextual features through a self-attention mechanism. The formula used is: , in These are the query matrix, key matrix, and value matrix, respectively. For the attention head dimension, Scaling factor ( () is the normalization function, which outputs the deep feature map. When Activate=0, ; S205: Unifies and fuses the feature dimensions of shallow and deep layers through 1×1 convolution; the core formula is... ; in The fused feature map of the i-th frame is located at coordinates (x, y) and channels. eigenvalues ​​at that location This represents the number of feature channels after fusion. , These are 1×1 fusion convolutional kernels representing shallow and deep features, respectively. To fuse convolutional biases; Output fusion features: ; S206: Real-time sampling and monitoring of CPU utilization and memory usage, using the following formula: , ; in Let be the CPU utilization at time t. Let t be the CPU busy time. =50ms is the computing load sampling period. This represents the current memory usage. This represents the amount of memory already used. This represents the total memory size of the edge nodes; S207: Adjust the deep branch inference frequency according to the load, using the following formula: ; in Let be the inference frequency of the deep feature branch at time t. This represents the normal inference frequency of deep feature branches. This is the frequency adjustment factor; 70% and 65% are the load thresholds for CPU utilization and memory usage, respectively. S3: Long Temporal Dependency Modeling and Feature Fusion: This method combines a lightweight temporal attention mechanism with behavioral chain association modeling to capture long temporal dependencies in video streams, extract temporal difference features, and fuse hierarchical features, temporal attention features, and temporal difference features to output a fused feature vector. The specific logical steps are as follows: S301: Receive the fused feature sequence output after hierarchical feature fusion. Where m is the number of valid frames, , , To fuse the width and height of the feature maps, To integrate the number of feature channels, we configure a lightweight temporal attention mechanism and core parameters for behavioral chain association modeling, set the temporal difference window size and behavioral chain template parameters, initialize feature fusion weights, adapt to the low computing power requirements of edge nodes, and ensure that the modeling and fusion process is efficient and has low latency. S302: Based on the fusion features of adjacent frames, temporal difference features are extracted to capture the motion change trend of targets in the video stream. The formula is as follows: ,in Let be the temporal difference feature map of the i-th frame. For the fused features of the i-th frame, The fused features are for the (i-1)th frame; And output the temporal difference feature sequence ; S303: Employs a lightweight temporal attention mechanism based on GRU to model the fused feature sequence, focusing on key frame features and capturing long-term temporal dependencies between frames. The formula used is as follows: ; ; ; in This represents the hidden state output by the GRU in the i-th frame. ( (This is a lightweight gated loop unit.) This is the hidden state from the previous frame. Let w be the attention weight for the i-th frame, w be the attention weight matrix, and b be the bias. This is the temporal attention feature vector; S304: Based on a pre-defined template of common abnormal behavior chains, association modeling is performed on temporal attention features to further strengthen long-term temporal dependencies. The formula used is as follows: ; ; in For the t-th preset behavior chain template, cos( () is the cosine similarity function. To achieve the maximum matching similarity, Feature vectors after modeling behavioral chain associations; S305: The hierarchical fusion features, temporal attention features, and temporal difference features are fused along a unified dimension to output the final fused feature vector. The formula is as follows: ; in The final output is the fused feature vector. This is a vector flattened from the hierarchical fusion features. This is the flattened vector of temporal difference features. , , These are the fusion weight matrices for the three types of features. For fusion bias; In addition, the number of preset abnormal behavior chain templates K is 8-12, covering common abnormal behaviors such as falling, running, gathering, and climbing. The number of GRU units is set to 64. The temporal attention feature dimension Cattn is consistent with the number of fused feature channels Cf. Finally, the fused feature vector dimension Cfinal is set to 128 dimensions to adapt to the low computing power inference and subsequent anomaly judgment requirements at the edge. S4: Real-time anomaly detection and cloud verification at the edge: A dual threshold detection mechanism is adopted to detect anomalies in the fused feature vector. High-confidence anomalies immediately trigger local alarms and are synchronized to the cloud. Suspicious anomalies are temporarily stored for cloud verification. The cloud performs high-precision verification of suspicious anomalies. At the same time, based on the labeled data uploaded by the edge nodes, the incremental learning algorithm is used to optimize the relevant models of the edge nodes. The specific logical steps are as follows: S401: Receives the final fused feature vector output after long-term dependency modeling and feature fusion. Where m is the number of valid frames, , For the final fusion feature dimension; The high threshold of the dual threshold judgment mechanism Low threshold Initialize the local alarm threshold and suspicious anomaly temporary storage cache at the edge, set the parameters of the high-precision review model and the core parameters of the incremental learning algorithm in the cloud, and ensure that the real-time performance of the edge and the review accuracy of the cloud are taken into account, so as to adapt to the edge-cloud collaborative architecture. S402: A lightweight SVM classifier is used to calculate the anomaly confidence score of the fused feature vector for each frame, combined with a dual threshold mechanism to determine the anomaly type. The formula used is as follows: ; ; in Let SVM be the anomaly confidence of the fused feature vector of the i-th frame. This is a lightweight support vector machine classifier. For high confidence threshold, The low confidence threshold The anomaly detection type for the i-th frame; S403: Based on the exception judgment result of S402, execute the corresponding processing logic, as follows: S4031: If it is a high-confidence anomaly: immediately trigger a local alarm at the edge, and simultaneously fuse the feature vector of this frame. Anomaly confidence level C The frame images are synchronized to the cloud for cloud recording and backup. S4032: If it is a suspicious anomaly: temporarily store the frame's fused feature vector, anomaly confidence, and frame image in the local cache at the edge, and periodically upload them to the cloud in batches, waiting for cloud review; S4033: If normal: No alarm or upload operation will be performed; only the detection log will be recorded for data accumulation for subsequent incremental learning. S404: The cloud receives suspicious anomaly data uploaded from the edge device, uses a 3D CNN+Transformer high-precision model for verification, and recalculates the anomaly confidence level. The formula used is as follows: ;

[0018] in This represents the confidence level for anomalies after cloud-based verification. ( This is a high-precision verification model in the cloud. For the context features of this frame, This is the final judgment result for any suspicious anomalies; The cloud will synchronize the review results back to the edge device, and the edge device will update its local detection records. S405: The edge end regularly uploads the fused feature vectors and annotation results corresponding to high-confidence anomalies and anomalies confirmed by the cloud to the cloud; based on the uploaded annotation data, the cloud uses an incremental learning algorithm to optimize the anomaly judgment model and feature extraction related models of the edge nodes; S5: Multi-edge node collaboration and dynamic updates: Multiple edge nodes achieve data sharing and collaborative detection through the edge gateway. The cloud aggregates multi-node detection logs, performs global model optimization, and ensures the consistency of multi-node detection performance. The specific logical steps are as follows: S501: Deploy multiple edge nodes, each establishing a communication connection through an edge gateway, completing the initialization of the collaboration protocol, configuring the edge gateway's data transmission rate and data sharing range, and setting the global model optimization cycle. The multi-node detection performance consistency threshold Δ is initialized to initialize the global model parameters in the cloud and the local model parameters of each edge node, adapting to the multi-edge-cloud collaborative architecture and balancing data transmission efficiency and detection real-time performance. S502: Each edge node uploads its own detection logs to the edge gateway in real time. The edge gateway performs unified aggregation, deduplication, and format standardization of data from multiple nodes to achieve data sharing among multiple nodes. S503: Each edge node performs consistency calibration on its local detection results based on standardized data shared by the edge gateway, correcting its own detection bias. The formula used is: ,in The anomaly confidence level of the k-th edge node after calibration in the i-th frame. Let the original anomaly confidence of the k-th edge node in the i-th frame be . Let be the average anomaly confidence score of all edge nodes in the i-th frame. =0.1-0.2 is a calibration coefficient used to balance local detection accuracy and multi-node consistency; After calibration, each edge node synchronously updates its local detection results to ensure consistent detection trends across multiple nodes; S504: The cloud periodically receives multi-node detection logs uploaded by the edge gateway, performs a global evaluation of the multi-node detection performance, and optimizes the global model using a federated averaging algorithm based on the aggregated detection logs and performance evaluation results. The formula used is as follows: ; ; in , For the optimized cloud-based global model weights and biases, , For the k-th edge node, the old weights and biases of the local model are given. The global learning rate, The gradient of the global loss function. Annotated detection data aggregated from multiple nodes; After optimization, the cloud distributes the global lightweight model to each edge node, and each node synchronously updates its local model parameters to ensure consistent detection performance across multiple nodes; simultaneously, the edge gateway updates the global mean of data sharing. Compared with global standard deviation This forms a collaborative optimization closed loop; Additionally, the global model optimization cycle The time limit is set to 1-2 hours, the multi-node detection performance consistency threshold Δ=5%, the data transmission rate of the edge gateway is controlled at 10-20Mbps, the data sharing scope only includes the detection results, standardized features and performance logs of each edge node, and the original video frames are not transmitted to reduce the data transmission bandwidth usage and adapt to low bandwidth scenarios of edge nodes.

[0019] This embodiment achieves low-latency and lightweight deployment of edge nodes by employing lightweight video stream preprocessing adaptation, a "shallow lightweight + deep dynamic" feature extraction architecture, and dynamic scheduling of computing power. This avoids the decrease in detection accuracy caused by lightweight processing, reduces redundant computation when multiple video streams are running concurrently, and enhances the modeling capability of continuous long-term abnormal behavior chains by combining lightweight temporal attention with behavioral chain association modeling. This reduces the false negative and false positive rates in scenarios with dense crowds and target occlusion. Furthermore, relying on edge-cloud collaborative verification, multi-node collaborative optimization, and incremental learning mechanisms, it further improves detection reliability and multi-node performance consistency, balancing privacy protection and deployment flexibility. This approach effectively adapts to the practical application needs of large-scale video surveillance.

[0020] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting abnormal behavior in video streams based on edge computing, characterized in that, Includes the following steps: S1: Video stream preprocessing and lightweight adaptation: The edge node receives the raw video stream captured by the front-end camera, performs adaptive frame interval sampling, size normalization, illumination compensation, noise suppression and lightweight feature enhancement on the video stream, removes redundant data and improves the contrast of target features. S2: Hierarchical feature extraction and dynamic computing power scheduling: The feature extraction architecture of "shallow lightweight + deep dynamic" is adopted. The basic texture features of video frames are extracted through shallow lightweight network. The deep feature branches are dynamically activated according to the complexity of shallow features to extract fine-grained temporal features and context-related features. At the same time, the edge nodes monitor their own computing power load in real time and dynamically schedule computing power resources to balance detection accuracy and real-time performance. S3: Long Temporal Dependency Modeling and Feature Fusion: This method combines a lightweight temporal attention mechanism with behavioral chain association modeling to capture long temporal dependencies in video streams, extract temporal difference features, and fuse hierarchical features, temporal attention features, and temporal difference features to output a fused feature vector. S4: Real-time anomaly detection and cloud verification at the edge: A dual threshold detection mechanism is adopted to detect anomalies in the fused feature vector. High-confidence anomalies immediately trigger local alarms and are synchronized to the cloud. Suspicious anomalies are temporarily stored for cloud verification. The cloud performs high-precision verification of suspicious anomalies. At the same time, based on the labeled data uploaded by the edge nodes, the incremental learning algorithm is used to optimize the relevant models of the edge nodes. S5: Multi-edge node collaboration and dynamic updates: Multiple edge nodes achieve data sharing and collaborative detection through the edge gateway. The cloud aggregates multi-node detection logs and performs global model optimization to ensure consistent multi-node detection performance.

2. The video stream abnormal behavior detection method based on edge computing according to claim 1, characterized in that, In S1, the adaptive frame interval sampling strategy is dynamically adjusted according to the video stream frame rate: when the frame rate is ≥30fps, the sampling interval is 2-3 frames; when the frame rate is <30fps, the sampling interval is 1 frame. Illumination compensation adopts the CLAHE algorithm, noise suppression adopts Gaussian filtering combined with median filtering, and lightweight feature enhancement is based on the feature enhancement branch of MobileNetV4-Lite. The specific logical steps are as follows: S101: The edge node receives the raw video stream captured by the front-end camera and parses it into a continuous video frame sequence: Simultaneously, the sampling interval and normalized target size basic parameters are initialized to adapt to the edge node computing power configuration, where T is the total number of frames in the original video stream. This is the T-th frame of the video image; S102: Reads the raw video stream frame rate via the edge node's built-in interface. The sampling interval k is dynamically determined based on the frame rate, and the formula is as follows: ,in The number of frames per second in the original video stream. The sampling interval is used to further sample the frame sequence. Sampling yields an effective frame sequence. m is the number of valid frames after sampling, which completes the removal of redundant data; S103: Using bilinear interpolation, each sampled frame... Scale to target size The formula used is: Coordinate mapping: ; Pixel value calculation: ; in , To normalize the target width and height, , The sampled frame width and height. , These are the normalized frame coordinates. , These are floating-point coordinates mapped to the sampled frame. For interpolation weights, Rounding down x yields the normalized frame sequence. ,in This is the normalized image of the m-th frame; S104: The CLAHE algorithm is used to normalize each frame. The formula used for illumination compensation is: Sub-block histogram calculation: ; Contrast Limitation: ; Grayscale mapping: ; in Sub-blocks for normalized frames For sub-block index, w and h are the width and height of the sub-block, t is the grayscale value, and I ( ) is an indicator function. This is the grayscale histogram of the sub-blocks. The contrast threshold. This is the grayscale mapping function; The illumination-compensated frame sequence is obtained after bilinear interpolation fusion. ,in This is the image after illumination compensation for the m-th frame; S105: To remove interference from rain, snow, dust, and electronic noise in video frames, and to meet the low-latency requirements of edge nodes, a combined filtering method of "Gaussian filtering + median filtering" is adopted. First, Gaussian noise is removed by Gaussian filtering, and then impulse noise is removed by median filtering, finally obtaining the noise-suppressed video frame sequence. ; S106: Feature enhancement is achieved based on MobileNetV4-Lite depthwise separable convolutions, using the following formula: Depthwise convolution: ; Point convolution: ; ReLU6 activation: ; Where c is the number of input channels. Number of output channels , These are the depthwise convolution kernel and the bias, respectively. , These are the point convolution kernel and the bias, respectively. , These output feature maps for depthwise convolution and pointwise convolution, respectively. Finally, a preprocessed frame sequence adapted for edge computing is obtained. ,in This is the final frame after feature enhancement.

3. The video stream abnormal behavior detection method based on edge computing according to claim 2, characterized in that, In S2, the shallow lightweight network adopts MobileNetV4-Lite with the number of parameters controlled within 5M, and the deep feature branches adopt a lightweight Transformer structure with the number of parameters controlled within 8M. The triggering condition for dynamic scheduling of computing power is as follows: when the CPU utilization of the edge node is ≥70% or the memory occupancy is ≥65%, the inference frequency of the deep feature branches is reduced; when the CPU utilization is <70% and the memory occupancy is <65%, the normal inference frequency is restored. The specific logical steps are as follows: S201: Receive the preprocessed final frame sequence: Configure the core parameters of shallow and deep networks and set the feature complexity threshold.

1. Set computing load threshold, initialize computing scheduling parameters, and adapt to edge node computing power limitations; Where m is the number of valid frames and C is the number of input channels; S202: MobileNetV4-Lite is used as a shallow, lightweight network. Basic texture features are extracted through depthwise separable convolutions. The formula used is as follows: ; in For the shallow feature map of frame i in coordinates (x, y) and channels eigenvalues ​​at that location ( ) is a lightweight activation function. It is a shallow 3×3 convolution kernel. The preprocessed image of the i-th frame in coordinates The pixel value at channel c. This is a shallow convolution bias. , This is the kernel offset; And output shallow feature maps: ; S203: Complexity is evaluated by combining feature information entropy and variance. The core formula is: Overall complexity assessment value: ; Dynamic activation judgment: ; in Let be the evaluation value of the shallow feature synthesis complexity of the i-th frame. =0.6, The shallow feature information entropy of the i-th frame is... , The minimum and maximum values ​​of the feature information entropy are... Let V be the shallow feature variance of the i-th frame. , These represent the minimum and maximum values ​​of the feature variance. Activate=1 indicates activation of deep branches, and Activate=0 indicates inactivation. S204: When Activate=1, the lightweight Transformer branch is activated. Taking shallow features as input, it extracts fine-grained temporal and contextual features through a self-attention mechanism. The formula used is: ,in These are the query matrix, key matrix, and value matrix, respectively. For the attention head dimension, Scaling factor ( () is the normalization function, which outputs the deep feature map. When Activate=0, ; S205: Unifies and fuses the feature dimensions of shallow and deep layers through 1×1 convolution; the core formula is... ； in The fused feature map of the i-th frame is located at coordinates (x, y) and channels. eigenvalues ​​at that location This represents the number of feature channels after fusion. , These are 1×1 fusion convolutional kernels representing shallow and deep features, respectively. To fuse convolutional biases; Output fusion features: ; S206: Real-time sampling and monitoring of CPU utilization and memory usage, using the following formula: , ; in Let be the CPU utilization at time t. Let t be the CPU busy time. =50ms is the computing load sampling period. This represents the current memory usage. This represents the amount of memory already used. This represents the total memory size of the edge nodes; S207: Adjust the deep branch inference frequency according to the load, using the following formula: ; in Let be the inference frequency of the deep feature branch at time t. This represents the normal inference frequency of deep feature branches. The frequency adjustment factor is 70% and 65%, which are the load thresholds for CPU utilization and memory usage, respectively.

4. The video stream abnormal behavior detection method based on edge computing according to claim 3, characterized in that, The specific logical steps of S3 are as follows: S301: Receive the fused feature sequence output after hierarchical feature fusion. Where m is the number of valid frames, , , To fuse the width and height of the feature maps, To integrate the number of feature channels, we configure a lightweight temporal attention mechanism and core parameters for behavioral chain association modeling, set the temporal difference window size and behavioral chain template parameters, initialize feature fusion weights, adapt to the low computing power requirements of edge nodes, and ensure that the modeling and fusion process is efficient and has low latency. S302: Based on the fusion features of adjacent frames, temporal difference features are extracted to capture the motion change trend of targets in the video stream. The formula is as follows: ,in Let be the temporal difference feature map of the i-th frame. For the fused features of the i-th frame, The fused features are for the (i-1)th frame; And output the temporal difference feature sequence ; S303: Employs a lightweight temporal attention mechanism based on GRU to model the fused feature sequence, focusing on key frame features and capturing long-term temporal dependencies between frames. The formula used is as follows: ； ； ； in This represents the hidden state output by the GRU in the i-th frame. ( (This is a lightweight gated loop unit.) This is the hidden state from the previous frame. Let w be the attention weight for the i-th frame, w be the attention weight matrix, and b be the bias. This is the temporal attention feature vector; S304: Based on a pre-defined template of common abnormal behavior chains, association modeling is performed on temporal attention features to further strengthen long-term temporal dependencies. The formula used is as follows: ； ； in For the t-th preset behavior chain template, cos( () is the cosine similarity function. To achieve the maximum matching similarity, Feature vectors after modeling behavioral chain associations; S305: The hierarchical fusion features, temporal attention features, and temporal difference features are fused along a unified dimension to output the final fused feature vector. The formula is as follows: ； in The final output is the fused feature vector. This is a vector flattened from the hierarchical fusion features. This is the flattened vector of temporal difference features. , , These are the fusion weight matrices for the three types of features. For fusion bias.

5. The video stream abnormal behavior detection method based on edge computing according to claim 4, characterized in that, The specific logical steps of S4 are as follows: S401: Receives the final fused feature vector output after long-term dependency modeling and feature fusion. Where m is the number of valid frames, , For the final fusion feature dimension; High threshold with dual threshold judgment mechanism Low threshold Initialize the local alarm threshold and suspicious anomaly temporary storage cache at the edge, set the parameters of the high-precision review model and the core parameters of the incremental learning algorithm in the cloud, and ensure that the real-time performance of the edge and the review accuracy of the cloud are taken into account, so as to adapt to the edge-cloud collaborative architecture. S402: A lightweight SVM classifier is used to calculate the anomaly confidence score of the fused feature vector for each frame, combined with a dual threshold mechanism to determine the anomaly type. The formula used is as follows: ； ； in Let SVM be the anomaly confidence of the fused feature vector of the i-th frame. This is a lightweight support vector machine classifier. For high confidence threshold, The low confidence threshold The anomaly detection type for the i-th frame; S403: Based on the exception judgment result of S402, execute the corresponding processing logic, as follows: S4031: If it is a high-confidence anomaly: immediately trigger a local alarm at the edge, and simultaneously fuse the feature vector of this frame. Anomaly confidence level C The frame images are synchronized to the cloud for cloud recording and backup. S4032: If it is a suspicious anomaly: temporarily store the frame's fused feature vector, anomaly confidence, and frame image in the local cache at the edge, and periodically upload them to the cloud in batches, waiting for cloud review; S4033: If normal: No alarm or upload operation will be performed; only the detection log will be recorded for data accumulation for subsequent incremental learning. S404: The cloud receives suspicious anomaly data uploaded from the edge device, uses a 3D CNN+Transformer high-precision model for verification, and recalculates the anomaly confidence level. The formula used is as follows: ； ； in This represents the confidence level for anomalies after cloud-based verification. ( This is a high-precision verification model in the cloud. For the context features of this frame, This is the final judgment result for any suspicious anomalies; The cloud will synchronize the review results back to the edge device, and the edge device will update its local detection records. S405: The edge end regularly uploads the fused feature vectors and annotation results corresponding to high-confidence anomalies and anomalies confirmed by the cloud to the cloud; based on the uploaded annotation data, the cloud uses an incremental learning algorithm to optimize the anomaly judgment model and feature extraction related models of the edge nodes.

6. The video stream abnormal behavior detection method based on edge computing according to claim 5, characterized in that, The specific logical steps of S5 are as follows: S501: Deploy multiple edge nodes, each establishing a communication connection through an edge gateway, completing the initialization of the collaboration protocol, configuring the edge gateway's data transmission rate and data sharing range, and setting the global model optimization cycle. The multi-node detection performance consistency threshold Δ is initialized to initialize the global model parameters in the cloud and the local model parameters of each edge node, adapting to the multi-edge-cloud collaborative architecture and balancing data transmission efficiency and detection real-time performance. S502: Each edge node uploads its own detection logs to the edge gateway in real time. The edge gateway performs unified aggregation, deduplication, and format standardization of data from multiple nodes to achieve data sharing among multiple nodes. S503: Each edge node performs consistency calibration on its local detection results based on standardized data shared by the edge gateway, correcting its own detection bias. The formula used is: ,in The anomaly confidence level of the k-th edge node after calibration in the i-th frame. Let the original anomaly confidence of the k-th edge node in the i-th frame be . Let be the average anomaly confidence score of all edge nodes in the i-th frame. =0.1-0.2 is a calibration coefficient used to balance local detection accuracy and multi-node consistency; After calibration, each edge node synchronously updates its local detection results to ensure consistent detection trends across multiple nodes; S504: The cloud periodically receives multi-node detection logs uploaded by the edge gateway, performs a global evaluation of the multi-node detection performance, and optimizes the global model using a federated averaging algorithm based on the aggregated detection logs and performance evaluation results. The formula used is as follows: ； ； in , For the optimized cloud-based global model weights and biases, , For the k-th edge node, the old weights and biases of the local model are given. The global learning rate, The gradient of the global loss function. Annotated detection data aggregated from multiple nodes; After optimization, the cloud distributes the global lightweight model to each edge node, and each node synchronously updates its local model parameters to ensure consistent detection performance across multiple nodes; simultaneously, the edge gateway updates the global mean of data sharing. Compared with global standard deviation This forms a collaborative optimization closed loop.

7. The video stream abnormal behavior detection method based on edge computing according to claim 6, characterized in that, In S3, the number of preset abnormal behavior chain templates K is 8-12, covering common abnormal behaviors such as falling, running, gathering, and climbing. The number of GRU units is set to 64. The temporal attention feature dimension Cattn is consistent with the number of fused feature channels Cf. Finally, the fused feature vector dimension Cfinal is set to 128 dimensions to adapt to the low computing power inference and subsequent anomaly judgment requirements at the edge.

8. The video stream abnormal behavior detection method based on edge computing according to claim 7, characterized in that, In S5, the global model optimization cycle The time limit is set to 1-2 hours, the multi-node detection performance consistency threshold Δ=5%, the data transmission rate of the edge gateway is controlled at 10-20Mbps, the data sharing scope only includes the detection results, standardized features and performance logs of each edge node, and the original video frames are not transmitted to reduce the data transmission bandwidth usage and adapt to low bandwidth scenarios of edge nodes.

Images