Security monitoring video intelligent analysis method based on multi-algorithm cooperation and unified architecture

Abstract

The application discloses a security monitoring video intelligent analysis method based on a multi-algorithm cooperation and unified architecture, relates to the technical field of intelligent video monitoring, and comprises the following steps: collecting security monitoring videos for pretreatment, performing space-time feature extraction and fusion on the collected security monitoring videos by using a CNN-LSTM space-time fusion engine, and outputting a fusion feature map; performing target detection, behavior recognition and abnormality detection on the fusion feature map to obtain multi-algorithm analysis results; performing weighted integration processing on the multi-algorithm analysis results by using a cross-module fusion mechanism based on an attention mechanism to obtain a fusion event representation vector; and performing event type identification and risk level evaluation on the fusion event representation vector to obtain event classification results and an event risk level; and the application improves the front-end sensing capability for abnormal behaviors in complex scenes by using the CNN-LSTM space-time fusion engine, and achieves the capability of improving the event detection accuracy and the anti-interference capability.

Description

Technical Field

[0001] This invention relates to the field of intelligent video surveillance technology, and in particular to an intelligent analysis method for security surveillance videos based on multi-algorithm collaboration and a unified architecture. Background Technology

[0002] With the development of video surveillance technology and artificial intelligence algorithms, intelligent video analytics has gradually become one of the core application methods in fields such as public safety, traffic management, and industrial monitoring. Traditional video surveillance systems mostly rely on manual patrols and rule-driven event detection mechanisms, which suffer from low efficiency and slow response in practical applications. In recent years, deep learning-based visual algorithms such as Convolutional Neural Networks (CNN) and Long Short-Term Memory Networks (LSTM) have been widely used in tasks such as object detection, behavior recognition, and anomaly detection, effectively improving the automation and intelligence level of video analytics.

[0003] Existing technologies typically employ a single algorithm or a serial structure to analyze targets and behaviors in surveillance videos, lacking the ability to collaboratively optimize multi-source analysis results. In particular, there is room for improvement in the deep information fusion and unified decision-making among multi-task results. Especially when facing dynamic and variable target behaviors, the system cannot fully integrate spatial information, time-series features, and cross-task semantic cues, resulting in limited performance in comprehensively judging event types and risk levels. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a security monitoring video intelligent analysis method based on multi-algorithm collaboration and unified architecture to solve the problems of inaccurate event identification and unclear risk assessment caused by the lack of deep integration and unified decision-making mechanism.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for intelligent analysis of security surveillance videos based on multi-algorithm collaboration and a unified architecture, comprising, Security monitoring videos are collected, preprocessed, and then spatiotemporal features are extracted and fused using the CNN-LSTM spatiotemporal fusion engine to output a fused feature map. The fused feature map is subjected to object detection, behavior recognition, and anomaly detection to obtain multi-algorithm analysis results; By utilizing a cross-module fusion mechanism based on attention, the analysis results of multiple algorithms are weighted and integrated to obtain a fused event representation vector; The event type identification and risk level assessment are performed on the fused event representation vector to obtain the event classification results and event risk level. The system makes alarm decisions based on the event classification results and event risk levels, obtains alarm information, stores and visualizes it, and obtains structured security monitoring video analysis results.

[0007] As a preferred embodiment of the intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and unified architecture described in this invention, the security monitoring video includes multiple camera video sources, real-time frame data, timestamps, and camera identifiers. The preprocessing includes noise reduction, white balance correction, and illumination equalization.

[0008] As a preferred embodiment of the intelligent analysis method for security surveillance videos based on multi-algorithm collaboration and unified architecture described in this invention, the specific steps for outputting the fused feature map are as follows: Based on preprocessed historical security surveillance videos, the CNN-LSTM spatiotemporal fusion engine is jointly fine-tuned and trained using an end-to-end supervised learning method to obtain the trained CNN-LSTM spatiotemporal fusion engine. The pre-processed security monitoring video is received by the trained CNN-LSTM spatiotemporal fusion engine, and frame rate segmentation is performed to generate a continuous frame sequence. Multi-scale convolution and pooling operations are performed on consecutive frame sequences to extract image texture, edge and target shape information, forming a spatial feature sequence; By capturing the inter-frame dependencies of spatial feature sequences through forget gates and input gates, the temporal feature sequences of target motion patterns and behavioral evolution are extracted. Spatial feature sequences and temporal feature sequences are concatenated and integrated along the channel dimension to output a fused feature map.

[0009] As a preferred embodiment of the intelligent analysis method for security surveillance videos based on multi-algorithm collaboration and a unified architecture described in this invention, the specific steps for performing target detection, behavior recognition, and anomaly detection on the fused feature map to obtain multi-algorithm analysis results are as follows. The object detection method based on CNN identifies each object in the fused feature map, locates the bounding box and category, and outputs the object detection results. Using an LSTM-based behavior recognition method, the target detection results and fused feature map are used to track target position changes and parse action sequences, outputting behavior recognition results. Based on the behavior recognition results and the fused feature map, the error of the behavior features is calculated and compared with the preset anomaly threshold, and the multi-algorithm analysis results are output.

[0010] As a preferred embodiment of the intelligent analysis method for security surveillance videos based on multi-algorithm collaboration and a unified architecture described in this invention, the method utilizes a cross-module fusion mechanism based on an attention mechanism to perform weighted integration processing on the analysis results of multiple algorithms to obtain a fused event representation vector. The specific steps are as follows: The deep semantic features of the intermediate layer are extracted from the multi-algorithm analysis results and concatenated with the logarithmic prediction value of the output layer to obtain the multi-source perception fusion feature vector. By using a cross-module fusion mechanism based on attention, attention weights are calculated on the multi-source perception fusion feature vectors to generate feature dimension attention weights. Based on the attention weights of the feature dimensions, the multi-source perception fusion feature vectors are weighted and summed to obtain the fusion event representation vector.

[0011] As a preferred embodiment of the intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and a unified architecture described in this invention, the specific steps for performing event type identification and risk level assessment on the fused event representation vector to obtain event classification results and event risk levels are as follows. High-level semantic features are extracted from the fused event representation vector as event feature expressions, and event type identification and classification are performed on the event feature expressions to output the event classification results; By combining event classification results, event characteristic expressions, and historical risk data, a risk level assessment is performed, and the event risk level is output.

[0012] As a preferred embodiment of the intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and a unified architecture described in this invention, the specific steps for making alarm decisions based on event classification results and event risk levels to obtain alarm information are as follows: The classification results and risk levels of historical events are summarized, compared and abstracted to extract the mapping relationship between event characteristics and risk levels, and to construct an alarm decision rule base. Based on the event classification results and event risk level, the corresponding alarm decision rule is matched in the alarm decision rule base, the alarm triggering logic is executed, and the alarm information is output.

[0013] As a preferred embodiment of the intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and a unified architecture described in this invention, the specific steps for obtaining the structured security monitoring video analysis results are as follows: The alarm information is formatted to form a structured alarm record and then stored. The structured alarm records are displayed in real time, and the results of structured security monitoring video analysis are obtained.

[0014] As a preferred embodiment of the intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and unified architecture described in this invention, the target detection result includes target bounding box coordinates, category label, category confidence score, and detection timestamp. The behavior recognition results include behavior category, behavior probability distribution, recognition confidence, action start time, end time, and target motion trajectory information.

[0015] As a preferred embodiment of the intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and unified architecture described in this invention, the alarm information includes a unique event identifier, event time information, event location information, event category label, event risk level label, and event core feature expression vector.

[0016] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the intelligent analysis method for security monitoring video based on multi-algorithm collaboration and unified architecture as described in the first aspect of the present invention.

[0017] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the intelligent analysis method for security monitoring video based on multi-algorithm collaboration and unified architecture as described in the first aspect of the present invention.

[0018] The beneficial effects of this invention are as follows: By using multi-scale convolution and long short-term memory frame-dependent capture and fusion, a deep joint representation of spatial texture features and temporal dynamic features in video frames is achieved. This can accurately depict the appearance details and motion trajectory of targets, ensuring reliable capture of key behavioral information even in environments with slow movement, partial occlusion, or changing lighting. This enhances the front-end perception capability of abnormal behavior in complex scenes, thereby improving the accuracy of event detection and the ability to resist interference. By using a cross-module attention mechanism to integrate multi-source features according to importance, the key information of high-confidence sub-algorithms can be automatically amplified and noise or redundant results can be suppressed, reducing false alarm rates, reducing computational resource waste, and enhancing output interpretability. The combination of these two methods improves real-time performance and scalability, providing strong technical support for intelligent monitoring in large-scale multi-channel deployment environments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a security monitoring video intelligent analysis method based on multi-algorithm collaboration and a unified architecture.

[0021] Figure 2 A flowchart for generating fusion event representation vectors.

[0022] Figure 3 A flowchart for obtaining the risk level of an event.

[0023] Figure 4 This is a flowchart for outputting structured security surveillance video analysis results. Detailed Implementation

[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0025] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0026] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0027] Reference Figures 1-4 This is one embodiment of the present invention, which provides a method for intelligent analysis of security surveillance videos based on multi-algorithm collaboration and a unified architecture, including the following steps: S1. Collect and preprocess security monitoring videos.

[0028] S1.1 The security monitoring video includes multiple camera video sources, real-time frame data, timestamps, and camera identifiers.

[0029] Furthermore, multiple camera video sources are deployed within the security monitoring area. The video data acquisition functions of these cameras are invoked, and each camera channel is accessed via a unified video data acquisition protocol. An exemplary acquisition frame rate of 25 frames per second is set. Each image frame is continuously acquired via acquisition commands. During acquisition, the camera's internal chip or the access control interface simultaneously obtains the real-time time information corresponding to the image frame and converts it into a unified timestamp format, such as "2025-07-30 09:15:27.120". Camera identification information is extracted by calling the camera's inherent hardware identifier or the unique encoded parameters in the network connection address. During image frame caching, each image frame, its corresponding timestamp, and the camera identifier are combined one-to-one to form a complete image frame. This is then packaged sequentially in the image cache or transmission protocol, forming a security monitoring video containing multiple camera video sources, real-time frame data, timestamps, and camera identification. S1.2 Preprocessing includes noise reduction, white balance correction, and illumination equalization.

[0030] Furthermore, denoising processing is performed on each real-time frame data of the security monitoring video. Based on the non-local mean filtering method, the exemplary search window size is set to 21×21, and the exemplary similarity window size is set to 7×7. By calculating the similarity between the grayscale mean difference of the current pixel block and the neighboring pixel blocks, pixel blocks with a similarity higher than the exemplary 0.03 are selected to participate in the weighted average, and pixel blocks with a similarity lower than the exemplary 0.03 are set to 0, thus completing the denoising processing. White balance correction is performed on the denoised image. The pixel mean of the three channels is extracted based on the gray world algorithm. The full image mean of the red, green and blue channels is obtained respectively. The standard gray value is set to 128. The gain coefficient of each channel is adjusted by the ratio between the channel mean and the standard gray value. The pixel values ​​of the three channels are remapped to obtain a color-balanced image frame. Illumination equalization is performed on color-balanced image frames. Based on the histogram equalization method, the gray-level histogram of the brightness channel of the color-balanced image frame is extracted. The number of gray levels is set to 256. By counting the number of pixels at each gray level in the brightness channel of the gray-level histogram, the proportion of the cumulative number of pixels corresponding to each gray level value to the total number of pixels is obtained. Each gray value in the brightness channel of the gray-level histogram is re-mapped to a new gray-level range, realizing the redistribution of brightness levels in the entire gray-level histogram brightness channel, making the brightness values ​​more evenly distributed in the gray-level histogram, thus completing illumination equalization.

[0031] S2. Use the CNN-LSTM spatiotemporal fusion engine to extract and fuse spatiotemporal features, and output a fused feature map.

[0032] S2.1 Based on the preprocessed historical security monitoring video, the CNN-LSTM spatiotemporal fusion engine is jointly fine-tuned and trained using an end-to-end supervised learning method to obtain the trained CNN-LSTM spatiotemporal fusion engine.

[0033] Furthermore, based on the preprocessed historical security surveillance videos, a training sample set containing input frame sequences and event labels is constructed. Each segment of the historical security surveillance video is continuously segmented at an exemplary frame rate of 25 frames per second, and each segment of image frame sequence with an exemplary length of 16 frames is obtained as input. Combined with the behavior category label or event type label corresponding to the image frame sequence as a supervision signal, the convolutional neural network structure for spatial feature extraction in the CNN-LSTM spatiotemporal fusion engine is called to perform convolution kernel sliding extraction on each image frame to extract texture, edge, and shape features, generating an exemplary spatial feature map sequence with dimensions of 224×224×64. The spatial feature map sequence is input in frame order by calling the long short-term memory neural network structure used for time-dependent capture in the CNN-LSTM spatiotemporal fusion engine. The memory unit state update step size is set to an exemplary 1 frame. Inter-frame feature changes and continuous action features are extracted. The supervision error is backpropagated to the parameters of each layer of the convolutional neural network and the long short-term memory neural network in combination with the labels of the input training sample set. The parameters are iteratively updated based on an exemplary learning rate of 0.001. After each training round, the number of training rounds is selected based on the validation set loss as an indicator and the training is terminated. The joint fine-tuning training of the CNN-LSTM spatiotemporal fusion engine is completed, and the trained CNN-LSTM spatiotemporal fusion engine is obtained.

[0034] S2.2. The preprocessed security monitoring video is received through the trained CNN-LSTM spatiotemporal fusion engine, and the frame rate is cut to generate a continuous frame sequence.

[0035] Furthermore, the preprocessed security monitoring video is received through the trained CNN-LSTM spatiotemporal fusion engine, and the image input interface is called to load continuous frame images, with an exemplary frame rate of 25 frames per second. Each preprocessed security monitoring video is divided into timelines according to an exemplary frame rate. An exemplary 16-frame sequence of images is extracted from the security monitoring video timeline at exemplary intervals of 0.64 seconds as a continuous frame sequence. The image frame sequence is numbered in chronological order to ensure that the image frames are strictly aligned with their corresponding timestamps. The input adaptation interface of the trained CNN-LSTM spatiotemporal fusion engine is called to input each continuous frame sequence into the spatial feature extraction network structure of the trained CNN-LSTM spatiotemporal fusion engine and complete the format conversion to generate a continuous frame sequence.

[0036] S2.3 Perform multi-scale convolution and pooling operations on the continuous frame sequence to extract image texture, edge and target shape information, forming a spatial feature sequence.

[0037] Furthermore, multi-scale convolution and pooling operations are performed on the continuous frame sequence, and the spatial feature extraction network structure in the trained CNN-LSTM spatiotemporal fusion engine is invoked. Each image frame is adjusted to an exemplary uniform input size of 224×224 pixels, and the convolution kernel sizes of the three exemplary sets of different receptive fields are set to 3×3, 5×5 and 7×7 respectively. Sliding calculations are performed on image frames to extract local texture, edge, and structural information at the corresponding scale. Channel independence is maintained in the convolution output, and scanning is performed with a stride of 1 pixel (example). Max pooling is performed on each group of convolution results, with the pooling window size set to 2×2 (example) and the stride set to 2 pixels (example). The maximum pixel value within the window is used to preserve the significant feature response of the region, reducing the spatial size and enhancing the expression of key region information. The multi-scale pooling results of each image frame are merged along the channel dimension to form a spatial feature map with an example dimension of 224×224×64. The above operation is repeated for each image frame in a continuous frame sequence to generate a spatial feature sequence arranged in chronological order.

[0038] S2.4. By capturing the inter-frame dependencies of spatial feature sequences through forget gates and input gates, the temporal feature sequences of target motion patterns and behavior evolution are extracted.

[0039] Furthermore, by capturing the inter-frame dependencies of spatial feature sequences through forget gates and input gates, the long short-term memory neural network structure in the trained CNN-LSTM spatiotemporal fusion engine is invoked, and the spatial feature sequence of each image frame is used as input. The state update step size of the memory unit is set to an exemplary 1 frame, and the temporal dependency model is performed on the spatial features input in each frame. The input gate controls the weight of the current input information. An example input gate activation threshold of 0.5 is set, and input spatial feature sequences greater than the input gate activation threshold are weighted and passed. The forget gate controls the influence of the memory from the previous moment. An example forget gate threshold of 0.5 is set, and memory information less than the forget gate threshold is discarded, while important temporal dimension information is retained to capture motion patterns and behavioral evolution between consecutive frames. The temporal feature sequence of the target motion law and behavioral evolution is extracted to form a temporal feature sequence. It should be noted that the process of setting the input gate activation threshold and forget gate threshold is as follows: Using an exemplary method, the frequency distribution curves of the activation values ​​corresponding to the input gate and forget gate in a large training sample set are first calculated. The boundary point that can effectively distinguish important information from noise is selected as the threshold selection. For example, 0.5 means that features with activation values ​​higher than 0.5 are considered important information and are retained, while features with activation values ​​lower than 0.5 are considered irrelevant information and are ignored. The input gate activation threshold and forget gate threshold are determined to be 0.5.

[0040] S2.5. The spatial feature sequence and the temporal feature sequence are concatenated and integrated along the channel dimension to output a fused feature map.

[0041] Furthermore, the fusion network structure used to integrate multi-source features in the trained CNN-LSTM spatiotemporal fusion engine is invoked. The spatial feature sequence with an exemplary dimension of 224×224×64 output by the spatial feature extraction network structure and the temporal feature sequence with an exemplary dimension of 224×224×64 output by the long short-term memory neural network structure are tensor-concatenated in the channel dimension to generate a spatiotemporal joint feature map sequence with an exemplary dimension of 224×224×128. The convolution kernel size of the fusion network structure is set to an exemplary 3×3 and the stride is set to an exemplary 1 pixel. A sliding convolution operation is performed on the concatenated spatiotemporal joint feature map sequence to extract the coupling information between channels. ReLU is used as the activation operation to enhance the nonlinear expression capability. After the convolution output, a max pooling operation is performed. The pooling window size is set to an exemplary 2×2 and the stride is set to an exemplary 2 pixels. Based on the preprocessed security monitoring video, a continuous frame sequence within a fixed time window is extracted. In each image frame, the Sobel operator is used to calculate the horizontal and vertical gradient values ​​of each pixel in the image frame to generate an image grayscale gradient map. The gradient magnitude of each direction in the local area is statistically analyzed to form a grayscale gradient direction histogram, which serves as a spatial texture feature. Based on consecutive frame images, the dense optical flow method is used to estimate the motion vector of each pixel in two adjacent image frames, and the direction distribution and average amplitude of motion vectors in local regions are extracted as inter-frame motion features. The continuous frame images are processed. In each image frame, the pixel difference image between the current frame and the previous frame is calculated using the inter-frame difference method. By setting the pixel difference threshold, the significantly changed areas are extracted, and the spatial location and contour information of the target area are preserved. Within each region of significant change, calculate the grayscale gradient orientation histogram of the region of significant change and the magnitude and orientation of the inter-frame motion vector; The spatial texture features of significantly changed regions are concatenated with inter-frame motion features to form a local spatiotemporal response vector; all significantly changed regions are traversed and local spatiotemporal feature response sequences are constructed in chronological order to form a compressed fusion feature map with an exemplary dimension of 112×112×128. It should be noted that the pixel difference threshold is set as follows: the distribution pattern of pixel differences in the pixel difference image is obtained, and the boundary point that can effectively distinguish the significantly changed area of ​​the target from the background noise is selected as the basis for setting. The exemplary difference threshold is between 0.1 and 0.3. S3. Perform object detection, behavior recognition, and anomaly detection on the fused feature map to obtain multi-algorithm analysis results.

[0042] S3.1 Using a CNN-based object detection method, identify each object in the fused feature map, locate the bounding box and category, and output the object detection results.

[0043] Furthermore, based on the compressed fused feature map with an exemplary dimension of 112×112×128, a CNN-based object detection method is called, with the convolution kernel size set to an exemplary 3×3 and the stride set to an exemplary 1 pixel. Sliding convolution is performed on the fused feature map to extract the feature response of the local region, and candidate box information containing bounding box coordinates, class probability and confidence is output at each sliding window position. After generating a set of candidate boxes at all sliding positions, non-maximum suppression is used to eliminate candidate boxes with high overlap. The intersection-union ratio (IUR) of two candidate boxes is calculated and compared with the overlap of the example 0.5. Low-confidence candidate boxes with IUR greater than the example 0.5 are deleted, and only the candidate boxes with the highest confidence are retained as output. For each retained candidate box, the corresponding target bounding box position coordinates and class label are output as the final target detection result. It should be noted that the expression for calculating the intersection-union ratio of two candidate boxes is: ; in, It is the intersection-union ratio of two candidate boxes. It is the area where the predicted bounding box intersects with the ground truth bounding box. It is the height of the overlapping area. It is the area of ​​the prediction box. It is the area of ​​the actual bounding box.

[0044] S3.2 Utilize the LSTM-based behavior recognition method to perform target position change tracking and action sequence parsing on the target detection results and fused feature map, and output the behavior recognition results.

[0045] Furthermore, based on the target detection results, the position coordinates of the target bounding boxes are serialized in chronological order. The bounding box coordinates of the same target in consecutive time frames are associated through a spatial overlap matching method. The matching process uses intersection-union ratio (IURR) calculation. The IURR of two bounding boxes is compared with the example value of 0.3. Bounding boxes with an IURR greater than the example value of 0.3 are determined to be continuous positions of the same target, thus constructing a target position change trajectory sequence. The target position change trajectory sequence and the corresponding fused feature map are used as input. The LSTM-based behavior recognition method is called. The memory unit state update step size is set to 1 frame for example. The target position and feature changes are processed recursively through time steps to capture the temporal dependence of key moments in the action sequence. Error backpropagation is performed in combination with input label information, and the LSTM weight parameters are adjusted to complete the training of the LSTM-based behavior recognition method. Using a trained LSTM-based behavior recognition method, the target position change trajectory is input into the LSTM-based behavior recognition method in chronological order. The internal memory unit retains and forgets trajectory information at consecutive time points. A gating mechanism is used to capture the temporal dependence and dynamic change features of the action. At the same time, combined with the spatial information in the input fusion feature map, the spatiotemporal evolution pattern of the target behavior is mined. The LSTM model is used to recursively model the temporal features of the target position change trajectory and the fusion feature map. The gating mechanism is used to extract the behavior evolution pattern. The softmax function of the output layer is used to obtain the probability distribution of behavior categories. Categories with a probability distribution greater than 0.5 are considered as valid behavior categories, and the behavior recognition result is obtained.

[0046] S3.3 Based on the behavior recognition results and fused feature maps, calculate the error of the behavior features and compare it with the preset anomaly threshold, and output the multi-algorithm analysis results.

[0047] Furthermore, based on the behavior recognition results and the fused feature map, a behavior feature vector is extracted. The behavior feature vector includes the action category probability, action confidence, and spatial temporal features in the fused feature map. The mean square error is used as the behavior feature error measure to obtain the error value between the current behavior feature vector and the historical normal behavior feature benchmark vector. Error values ​​are calculated by statistically analyzing a large number of behavioral feature error samples. For example, frequency intervals are divided as [0,0.1), [0.1, 0.2), [0.2, 0.3), [0.3, 0.4), [0.4, 0.5), etc. The width of each interval is set to 0.1 for example. The number of occurrences of error samples in each interval is counted, the frequency proportion of each interval is calculated, and a frequency distribution curve is plotted by combining the frequency proportions of all intervals. The boundary point that can effectively distinguish between normal and abnormal behavior is selected as the abnormal threshold. The abnormal threshold is set as the boundary point. The calculation error value is compared with the abnormal threshold. When the calculation error value is greater than the abnormal threshold, it is judged as abnormal behavior. When the error value is not greater than the abnormal threshold, it is judged as normal behavior. Based on the judgment result, the multi-algorithm analysis results are output, including event recognition results and event feature information. It should be noted that the historical normal behavior feature baseline vector is derived from: after identifying the behavior of normal behavior data in a large amount of preprocessed historical security monitoring videos, a set of behavior feature vectors is extracted, and by statistically analyzing these normal behavior feature vectors, representative statistics such as the mean or median are obtained to form a stable and representative baseline vector.

[0048] S4. Using an attention-based cross-module fusion mechanism, the analysis results of multiple algorithms are weighted and integrated to obtain a fused event representation vector.

[0049] S4.1 Extract the deep semantic features of the intermediate layer from the multi-algorithm analysis results and concatenate them with the logarithmic prediction values ​​of the output layer to obtain the multi-source perception fusion feature vector.

[0050] Furthermore, based on the analysis results of multiple algorithms, deep semantic feature vectors of the intermediate layers are extracted from both the CNN-based object detection method and the LSTM-based action recognition method. For the CNN-based object detection method, feature maps output from multiple convolutional layers or high-dimensional feature vectors from the last fully connected layer are selected. These high-dimensional feature vectors can express the spatial texture, shape, and contextual information of the target. For the LSTM-based action recognition method, temporal feature representations of the hidden layers of the internal LSTM are extracted. These temporal feature representations reflect the dynamic evolution of the target's actions over time. For the output layer, the category prediction results of both the CNN-based object detection method and the LSTM-based action recognition method are obtained, and the category confidence distribution is logarithmically transformed to generate logarithmic prediction values ​​to enhance the numerical stability and discriminative power of the category probabilities. The aforementioned deep semantic feature vectors and their corresponding logarithmic prediction values ​​are processed through a unified dimensional transformation to ensure compatibility between features from different sources during concatenation. Finally, they are fused in the channel dimension using a tensor concatenation method to form a multi-source perceptual fusion feature vector with an exemplary fusion dimension of 256. The deep semantic feature vectors and corresponding logarithmic prediction values ​​of the CNN-based object detection method and the LSTM-based behavior recognition method are processed to unify their dimensions. The deep semantic feature vectors and logarithmic prediction values ​​are concatenated in the channel dimension using the tensor concatenation method to generate a multi-source perception fusion feature vector with an exemplary fusion dimension of 256. The concatenation process maintains the integrity of each feature vector without information loss and is used as the output of the multi-source perception fusion feature vector.

[0051] S4.2. Through a cross-module fusion mechanism based on attention mechanism, attention weights are calculated on the multi-source perception fusion feature vector to generate feature dimension attention weights.

[0052] Furthermore, by using the training data of multi-source perception fusion feature vectors and corresponding event category labels, the samples are input into the cross-module fusion mechanism based on the attention mechanism in batches according to the exemplary batch size of 32. For the multi-source perception fusion feature vector in each batch, the query mapping function in the attention-based cross-module fusion mechanism is called to generate the query vector, the key mapping function is called to generate the key vector, and the value mapping function is called to generate the value vector. Calculate the dot product similarity between the query vector and the key vector, and for example, adjust the similarity using a temperature scaling factor of 0.07; perform softmax normalization on the adjusted similarity to obtain the attention weights of the feature dimensions; The attention weights are summed after element-wise multiplication of the value vectors to obtain a weighted feature representation; the weighted feature representation is then input into the classification head to output the predicted event category distribution. The loss value is calculated using the cross-entropy loss function based on the predicted distribution and the actual event category labels. The loss value is passed to the weight parameters of the query mapping function, key mapping function, and value mapping function through the backpropagation algorithm, and the parameters are updated using the Adam optimizer with an example learning rate of 0.0001. The above iterative steps are repeated for 50 rounds until the accuracy of the validation set no longer improves significantly. After training, the trained attention-based cross-module fusion mechanism is saved. Based on the trained attention-based cross-module fusion mechanism, the multi-source perception fusion feature vector is input into the attention-based cross-module fusion mechanism. The corresponding vector is generated using the query, key, and value mapping function. The similarity between the query vector and the key vector is calculated and the attention weight is obtained by softmax normalization. The weight is used to weight and sum the value vector to generate the feature dimension attention weight. It should be noted that the expression for calculating the loss value using the cross-entropy loss function is as follows: ； in, It is the loss value. It is the first in the true target distribution The value of the class, The probability distribution of the output of the attention-based cross-module fusion mechanism is as follows: Predicted probability of class It is an index variable of the category.

[0053] S4.3. Based on the attention weights of the feature dimensions, the multi-source perception fusion feature vectors are weighted and summed to obtain the fusion event representation vector.

[0054] Furthermore, based on the feature dimension attention weights output by the multi-source perception fusion feature vector and the cross-module fusion mechanism based on the attention mechanism, for each feature channel, the corresponding attention coefficient value in the feature dimension attention weight is read. All channels of the multi-source perception fusion feature vector are traversed, and all feature values ​​of each channel are multiplied with the corresponding attention coefficients element by element to obtain the fusion channel features weighted by the feature dimension. The weighted fusion channel features are summed by the channel dimension to obtain the fusion event representation vector.

[0055] S5. Perform event type identification and risk level assessment on the fused event representation vector to obtain event classification results and event risk level.

[0056] S5.1 Extract high-level semantic features from the fused event representation vector as event feature expressions, and perform event type identification and classification on the event feature expressions to output the event classification results.

[0057] Furthermore, all feature values ​​are read from the fused event representation vector in dimensional order, and the fused event representation vector is input into the classification head part of the trained attention-based cross-module fusion mechanism. A linear transformation is performed on the input fused event representation vector and the transformation result is output. The activation function is called to perform non-linear processing on the linear transformation result to generate an intermediate representation. An exemplary dropout operation is used to randomly mask some intermediate representation features at a ratio of 0.5 to reduce the risk of overfitting. The intermediate representation after the dropout operation is input into the linear transformation function again to output the event category prediction distribution. The softmax function is called to perform a normalization operation on the prediction distribution to obtain the event classification probability. The normalized event classification probability vector is traversed to obtain the index corresponding to the maximum value. The index corresponding to the maximum value is mapped one-to-one with the event category label to output the event classification result.

[0058] S5.2 Combine the event classification results, event characteristic expressions, and historical risk data to perform a risk level assessment and output the event risk level.

[0059] Furthermore, the category label of the event is read from the event classification results, the feature values ​​corresponding to all feature dimensions are read from the event feature expression to form the event feature expression vector, historical event samples that are consistent with the event category label are retrieved from the historical risk data according to the event category, and the risk level label and the corresponding event feature expression vector of each historical event sample are extracted. Obtain the cosine similarity between the event feature expression vector and the feature expression vector of each historical event sample. Select the top 20 historical event samples with the highest similarity as a highly relevant sample set. Perform statistical operations on the risk level label corresponding to each sample in the highly relevant sample set, count the frequency of each risk level category in the highly relevant sample set, and calculate the frequency distribution of each risk level category. The risk level category with the highest frequency value in the frequency distribution is taken as the event risk level of the current event. If there are multiple risk level categories with the same frequency value in the frequency distribution, the risk level category corresponding to the historical event sample with the highest cosine similarity to the current event feature expression vector is selected as the event risk level. It should be noted that the expression for calculating the frequency distribution of each risk level category is as follows: ; in, It is the first The frequency of each risk level category It is an index variable for risk level categories. It is the first The number of samples in the highly relevant sample set for each risk level category It is the total number of samples in the highly relevant sample set.

[0060] S6. Make alarm decisions based on the event classification results and event risk levels to obtain alarm information.

[0061] S6.1 Summarize, compare and abstract the classification results and risk levels of historical events, extract the mapping relationship between event characteristics and risk levels, and construct an alarm decision rule base.

[0062] Furthermore, the classification results and risk levels of historical events are grouped by event category labels. All historical event samples under the same event category are organized into a dataset according to the event feature expression vector and corresponding risk level information. Statistical methods are used to perform feature selection and dimensionality reduction on the event feature expression vector in each event category to extract the core dimensions of the event category features. Different risk level categories are distinguished according to the risk level label. The distribution characteristics and boundary range of each event risk level category in the core feature space are calculated. Based on the feature distribution and boundary range, an exemplary feature space distance threshold is set to distinguish different event risk level categories. The event category, core feature dimension range, risk level, and feature space distance threshold information are formed into mapping relationship entries. All mapping relationship entries of event categories are organized and summarized into an alarm decision rule base. It should be noted that the expression for calculating the distribution characteristics and boundary range of each event risk level category in the core feature space is as follows: ; in, It means the first Distribution characteristics of risk levels in the core feature space It is the risk level category number of the event. It means the first The total number of event samples in each risk level category It indicates that it belongs to the first Class of risk level The core feature vector of an event It is the first Index of event samples within the risk level category; ; in, It is the first The boundary range of the core feature space of risk level.

[0063] S6.2 Based on the event classification results and event risk level, match the corresponding alarm decision rules in the alarm decision rule base, execute the alarm triggering logic, and output alarm information.

[0064] Furthermore, the event category label and event risk level label are read from the event classification results and event risk level. The mapping relationship entries that match the event category and event category label are retrieved from the alarm decision rule base. The feature values ​​of the corresponding core feature dimension in the event feature expression vector are extracted to form the event core feature expression. The distance between the event core feature expression and the center of the core feature space distribution corresponding to different risk level categories in the mapping relationship entries is obtained. Based on the exemplary feature space distance threshold, it is determined which risk level category center has the smallest distance and is less than or equal to the feature space distance threshold. If multiple risk level categories meet the feature space distance threshold, they are sorted according to the distance from smallest to largest and the risk level category corresponding to the smallest value is selected. The matching result is the corresponding alarm decision rule in the alarm decision rule base. According to the alarm level configuration and output field definition in the matching result, alarm information is generated by combining them. It should be noted that the process of setting the feature space distance threshold is as follows: based on the labeled normal and abnormal events, the distance distribution is statistically analyzed in the feature space, and the boundary value that can effectively distinguish the two types of events is selected as the feature space distance threshold. An example value is 0.15.

[0065] S7. Obtain the structured security monitoring video analysis results.

[0066] S7.1 Format the alarm information to form a structured alarm record and store it.

[0067] Furthermore, the values ​​of each field in the structured alarm information are filled into the corresponding field position information positions in the structured alarm record format according to the field order of the alarm record. The exemplary structured alarm record format includes an event unique identifier field, an event time information field, an event location information field, an event category label field, an event risk level label field, an event core feature expression dimension field and its corresponding value field, an alarm level field, and an alarm generation time field. The alarm generation time is recorded in an exemplary timestamp format and is used as a component of the event unique identifier field. The event unique identifier field, event time information field, event location information field, event category label field, event risk level label field, event core feature expression dimension field and their corresponding value fields are concatenated into a standard format record structure and written into the storage form, and then saved to disk using a structured data storage method.

[0068] S7.2. The structured alarm records are displayed in real time to obtain the structured security monitoring video analysis results.

[0069] Furthermore, a periodic polling mechanism is configured in the front-end display interface to request the latest structured alarm records from the back-end service interface once every 1 second at an exemplary time interval. In the service interface, a database query statement is called to sort and filter the structured alarm records within the past minute in descending order by the event time information field, and the query results are encapsulated in JSON format and returned to the display interface. The display interface parses the returned structured alarm record data and extracts the event time information field, event location information field, event category label field, event risk level label field, event core feature expression dimension field and corresponding value field, and alarm level field for structured display rendering. A linkage mechanism is configured in the display area to spatially associate and label the event location information field with an exemplary electronic map component, and to match the event time information field with the exemplary video retrieval interface using time index to request the corresponding monitoring video segment. The video content with the same time as the alarm event is synchronously loaded in the monitoring video display area and the location of the event is highlighted in the frame with an exemplary red box. Finally, a structured security monitoring video analysis result containing structured alarm record information and corresponding monitoring video content is formed in the unified display page.

[0070] S8. The target detection results include the target bounding box coordinates, category label, category confidence score, and detection timestamp; S8.1 The behavior recognition results include behavior category, behavior probability distribution, recognition confidence, action start time, end time and target motion trajectory information.

[0071] S9. Behavioral alarm information includes a unique event identifier, event time information, event location information, event category label, event risk level label, and event core feature expression vector.

[0072] This embodiment also provides a computer device applicable to the intelligent analysis method of security monitoring video based on multi-algorithm collaboration and unified architecture, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the intelligent analysis method of security monitoring video based on multi-algorithm collaboration and unified architecture proposed in the above embodiment.

[0073] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0074] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements the intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and a unified architecture, as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0075] In summary, this invention achieves a deep joint representation of spatial texture features and temporal dynamic features in video frames through multi-scale convolution and long short-term memory frame-dependent capture and fusion. This accurately depicts the appearance details and motion trajectory of targets, ensuring reliable capture of key behavioral information even in environments with slow movement, partial occlusion, or changing lighting. It enhances the front-end perception capability of abnormal behavior in complex scenes, thereby improving event detection accuracy and anti-interference capabilities. By utilizing a cross-module attention mechanism to weight and integrate multi-source features according to importance, it can automatically amplify key information of high-confidence sub-algorithms and suppress noise or redundant results, reducing false alarm rates, minimizing computational resource waste, and enhancing output interpretability. The combination of these two approaches improves real-time performance and scalability, providing strong technical support for intelligent monitoring in large-scale multi-channel deployment environments.

[0076] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for intelligent analysis of security surveillance videos based on multi-algorithm collaboration and a unified architecture, characterized by: include, Security surveillance videos are collected and preprocessed, and then spatiotemporal features are extracted and fused using a CNN-LSTM spatiotemporal fusion engine to output a fused feature map. The specific steps are as follows. Based on preprocessed historical security surveillance videos, the CNN-LSTM spatiotemporal fusion engine is jointly fine-tuned and trained using an end-to-end supervised learning method to obtain the trained CNN-LSTM spatiotemporal fusion engine. The pre-processed security monitoring video is received by the trained CNN-LSTM spatiotemporal fusion engine, and frame rate segmentation is performed to generate a continuous frame sequence. Multi-scale convolution and pooling operations are performed on consecutive frame sequences to extract image texture, edge and target shape information, forming a spatial feature sequence; By capturing the inter-frame dependencies of spatial feature sequences through forget gates and input gates, the temporal feature sequences of target motion patterns and behavioral evolution are extracted. The spatial feature sequence and the temporal feature sequence are concatenated and integrated along the channel dimension to output a fused feature map. The steps include: based on the preprocessed security monitoring video, extracting the continuous frame sequence within a fixed time window; in each image frame, using the Sobel operator to calculate the horizontal and vertical gradient values ​​of each pixel in the image frame to generate an image grayscale gradient map; and statistically analyzing the gradient magnitudes in each direction in the local area to form a grayscale gradient direction histogram as a spatial texture feature. The fused feature map is subjected to object detection, behavior recognition, and anomaly detection to obtain multi-algorithm analysis results; A cross-module fusion mechanism based on attention is used to weightedly integrate the analysis results of multiple algorithms to obtain a fused event representation vector. The specific steps are as follows. The deep semantic features of the intermediate layer are extracted from the multi-algorithm analysis results and concatenated with the logarithmic prediction value of the output layer to obtain the multi-source perception fusion feature vector. By using a cross-module fusion mechanism based on attention, attention weights are calculated on the multi-source perception fusion feature vectors to generate feature dimension attention weights. Based on the attention weights of the feature dimensions, the multi-source perception fusion feature vectors are weighted and summed to obtain the fusion event representation vector. The method includes the following steps: Based on the analysis results of multiple algorithms, deep semantic feature vectors of the intermediate layers are extracted from the CNN-based object detection method and the LSTM-based action recognition method, respectively. For the CNN-based object detection method, feature maps output from multiple convolutional layers or high-dimensional feature vectors from the last fully connected layer are selected. These high-dimensional feature vectors can express the spatial texture, shape, and contextual information of the target. For the LSTM-based action recognition method, temporal feature representations of the hidden layers of the internal LSTM are extracted. Temporal feature representations reflect the dynamic evolution of the target's actions over time. For the output layer, the class prediction results of the CNN-based object detection method and the LSTM-based action recognition method are obtained respectively, and the class confidence distribution is logarithmically transformed to generate logarithmic prediction values ​​to enhance the numerical stability and discriminative power of the class probabilities. The event type identification and risk level assessment are performed on the fused event representation vector to obtain the event classification results and event risk level. The system makes alarm decisions based on the event classification results and event risk levels, obtains alarm information, stores and visualizes it, and obtains structured security monitoring video analysis results.

2. The intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and unified architecture as described in claim 1, characterized in that: The security monitoring video includes multiple camera video sources, real-time frame data, timestamps, and camera identifiers. The preprocessing includes noise reduction, white balance correction, and illumination equalization.

3. The intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and unified architecture as described in claim 2, characterized in that: The process involves performing object detection, behavior recognition, and anomaly detection on the fused feature map to obtain multi-algorithm analysis results. The specific steps are as follows: The object detection method based on CNN identifies each object in the fused feature map, locates the bounding box and category, and outputs the object detection results. Using an LSTM-based behavior recognition method, the target detection results and fused feature map are used to track target position changes and parse action sequences, outputting behavior recognition results. Based on the behavior recognition results and the fused feature map, the error of the behavior features is calculated and compared with the preset anomaly threshold, and the multi-algorithm analysis results are output.

4. The intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and unified architecture as described in claim 1, characterized in that: The process of performing event type identification and risk level assessment on the fused event representation vector yields event classification results and event risk levels. The specific steps are as follows: High-level semantic features are extracted from the fused event representation vector as event feature expressions, and event type identification and classification are performed on the event feature expressions to output the event classification results; By combining event classification results, event characteristic expressions, and historical risk data, a risk level assessment is performed, and the event risk level is output.

5. The intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and unified architecture as described in claim 4, characterized in that: The specific steps for making alarm decisions based on event classification results and event risk levels to obtain alarm information are as follows. The classification results and risk levels of historical events are summarized, compared and abstracted to extract the mapping relationship between event characteristics and risk levels, and to construct an alarm decision rule base. Based on the event classification results and event risk level, the corresponding alarm decision rule is matched in the alarm decision rule base, the alarm triggering logic is executed, and the alarm information is output.

6. The intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and unified architecture as described in claim 1, characterized in that: The specific steps for obtaining the structured security surveillance video analysis results are as follows: The alarm information is formatted to form a structured alarm record and then stored. The structured alarm records are displayed in real time, and the results of structured security monitoring video analysis are obtained.

7. The intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and unified architecture as described in claim 3, characterized in that: The target detection results include target bounding box coordinates, category label, category confidence score, and detection timestamp; The behavior recognition results include behavior category, behavior probability distribution, recognition confidence, action start time, end time, and target motion trajectory information.

8. The intelligent analysis method for security monitoring videos based on multi-algorithm collaboration and unified architecture as described in claim 5, characterized in that: The alarm information includes a unique event identifier, event time information, event location information, event category label, event risk level label, and event core feature expression vector.

Images