Equipment security monitoring method and equipment based on artificial intelligence, and medium

Abstract

The invention discloses an artificial intelligence-based equipment security and protection monitoring method, equipment and a medium, and relates to the technical field of equipment security and protection monitoring, and the method comprises the steps: constructing a deep learning optical flow estimation model based on image frame sequences, carrying out the optical flow motion vector calculation of adjacent image frames through employing the model, and extracting the motion characteristics of smoke diffusion; textural features are extracted from the image frames, normalization and time alignment are carried out on the textural features and smoke diffusion characterization features, and a space-time joint feature vector sequence is constructed; performing smoke event classification prediction on the sequence through a time sequence classification model containing an attention mechanism to obtain an event type and confidence; and triggering a security response according to the smoke event type and the confidence coefficient. According to the invention, the dynamic characteristics of smoke diffusion are effectively extracted, and the detection precision is improved; the capturing capability of time sequence continuity and space consistency in the smoke diffusion process is enhanced through spatio-temporal joint modeling, and missing report and false report are reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of equipment security monitoring, and in particular to an equipment security monitoring method based on artificial intelligence, equipment and medium. BACKGROUND

[0002] In the field of equipment security monitoring, traditional monitoring techniques mainly rely on static detection methods based on image frames. These methods usually detect abnormal events through pixel-level differences in images or motion changes in images. With the development of deep learning technology, more and more security monitoring methods based on artificial intelligence have been applied to real-time event detection. In particular, in monitoring video analysis, deep learning models can automatically extract features from complex backgrounds and scenes to recognize various abnormal events. These methods train deep neural networks to capture temporal and spatial patterns in video sequences to achieve more accurate abnormal event recognition.

[0003] However, existing equipment security monitoring methods based on artificial intelligence still have certain challenges in handling complex dynamic scenes and identifying fine-grained abnormal events. On the one hand, although deep learning methods can automatically extract features, they often struggle to accurately model the complex dynamic changes in time series for abnormal events such as smoke, which have fuzzy boundaries and strong diffusion. This is especially true in low-light and high-noise scenarios. On the other hand, existing methods rely heavily on single image frames for detection, lacking sufficient use of temporal information and failing to effectively capture the spatiotemporal continuity of events, which limits their performance in continuous monitoring scenarios. For smoke detection, the diffusion of smoke has a time span, and its morphological changes are irregular, so single-frame image information cannot effectively capture these dynamic characteristics. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides an equipment security monitoring method based on artificial intelligence to solve the problems of lacking abnormal event recognition in complex dynamic backgrounds and lacking spatiotemporal feature modeling in the prior art.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides an equipment security monitoring method based on artificial intelligence, which includes collecting monitoring video streams of a device area to be monitored, standardizing the monitoring video streams, and obtaining a sequence of continuous standardized image frames.

[0008] A deep learning optical flow estimation model is constructed based on a standardized image frame sequence. The deep learning optical flow estimation model is used to calculate the predicted optical flow motion vector of adjacent image frame pairs and extract the smoke diffusion characterization features.

[0009] Image texture features are extracted from a standardized image frame sequence, and the image texture features are normalized and time-aligned with the smoke diffusion characterization features to construct a spatiotemporal joint feature vector sequence.

[0010] The spatiotemporal joint feature vector sequence is used to classify and predict smoke events through a temporal classification model that includes an attention mechanism, predicting the type and confidence level of smoke events.

[0011] The system determines whether a smoke event is triggered based on the type and confidence level of the smoke event, and then triggers different levels of security response strategies.

[0012] As a preferred embodiment of the AI-based device security monitoring method of the present invention, the steps of acquiring the monitoring video stream of the area to be monitored, standardizing the monitoring video stream, and obtaining a continuous standardized image frame sequence are as follows:

[0013] When collecting surveillance video streams of the area to be monitored, use a camera to acquire a continuous video stream covering the area to be monitored;

[0014] Image frames are extracted from the acquired surveillance video stream at fixed time intervals.

[0015] Gaussian filtering is applied to the image frame sequence to remove noise, gamma transform is applied to the image frame sequence to enhance low-brightness areas in the image, and pixel normalization is applied to the image frame sequence to obtain an image frame sequence with consistent brightness distribution.

[0016] Cropping removes non-target regions from the image frame sequence, retains the target region, and obtains a continuous, standardized image frame sequence.

[0017] As a preferred embodiment of the AI-based device security monitoring method of the present invention, the specific steps of constructing a deep learning optical flow estimation model based on standardized image frame sequences are as follows:

[0018] Image frame pairs are constructed as training samples using a standardized sequence of image frames.

[0019] A weakly supervised training dataset was constructed using standardized image frame sequences and pseudo-labeled optical flow maps to create training samples and a training dataset.

[0020] The RAFT-based deep learning optical flow estimation model was trained using a weakly supervised training dataset, and the training weights were saved.

[0021] The deep learning optical flow estimation model consists of an image feature extraction layer, a correlation volume generation layer, an optical flow update iteration layer, and an optical flow upsampling reconstruction layer.

[0022] As a preferred embodiment of the AI-based device security monitoring method of the present invention, the steps of using a deep learning optical flow estimation model to calculate the predicted optical flow motion vectors of adjacent image frame pairs and extracting smoke diffusion characterization features are as follows:

[0023] Image frame pairs are extracted from a standardized image frame sequence in chronological order and used as the input image frame pair set for a deep learning optical flow estimation model.

[0024] A deep learning optical flow estimation model is used to perform forward inference on standardized image frames, and the predicted optical flow motion vector map atlas is output.

[0025] Local temporal stability statistics are performed on the predicted optical flow motion vector diagram to generate an optical flow stability matrix as the first smoke diffusion characterization feature.

[0026] The orientation histogram of each pixel is calculated based on the predicted optical flow motion vector diagram, and the orientation distribution histogram is used as a second smoke diffusion characterization feature.

[0027] As a preferred embodiment of the AI-based device security monitoring method of the present invention, the steps of extracting image texture features from a standardized image frame sequence, normalizing and temporally aligning the image texture features with smoke diffusion characterization features, and constructing a spatiotemporal joint feature vector sequence are as follows:

[0028] Local binary encoding is performed on standardized image frames to obtain image texture feature maps;

[0029] The image texture feature map is subjected to average pooling to generate a vector of image texture features;

[0030] Normalize the image texture feature vector and the smoke diffusion representation feature respectively;

[0031] By using a time window, the image texture feature vector and the smoke diffusion characterization feature are aligned according to the frame-level timestamp, forming time-aligned image texture feature sequences and smoke diffusion characterization feature sequences, respectively.

[0032] The texture feature vector and smoke diffusion representation feature of each frame image are concatenated to construct a spatiotemporal joint feature vector sequence.

[0033] As a preferred embodiment of the AI-based device security monitoring method of the present invention, the temporal classification model containing the attention mechanism is composed of a bidirectional temporal modeling layer, an attention mechanism layer, and a fully connected layer.

[0034] As a preferred embodiment of the AI-based device security monitoring method of the present invention, the specific steps of classifying and predicting smoke events by using a temporal classification model containing an attention mechanism to classify and predict the smoke event type and confidence level are as follows:

[0035] The spatiotemporal joint feature vector sequence is used as input, and the bidirectional gated loop structure in the temporal classification model containing the attention mechanism is used to encode the features step by step to generate the bidirectional GRU output sequence.

[0036] The bidirectional GRU output sequence is assigned attention weights for each frame through the attention mechanism in the temporal classification model to generate a global attention-weighted feature representation;

[0037] The global attention-weighted feature representation is passed through a fully connected layer to obtain the predicted probability distribution vector of the smoke event type;

[0038] The smoke event type with the highest probability is extracted from the predicted probability distribution vector as the final predicted smoke event type, and the highest probability of the final predicted smoke event type is used as the confidence level.

[0039] As a preferred embodiment of the AI-based device security monitoring method of the present invention, the specific steps of determining smoke event triggering based on smoke event type and confidence level, and triggering different levels of security response strategies, are as follows:

[0040] By analyzing the confidence levels of predictions in historical training data, a set of confidence thresholds for smoke event response levels is set.

[0041] Determine the response level for a smoke event based on the predicted type and confidence level.

[0042] The security response strategy type is determined based on the smoke event response level, and the corresponding security response strategy is triggered.

[0043] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements any step of the artificial intelligence-based device security monitoring method described in the first aspect of the present invention.

[0044] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the artificial intelligence-based device security monitoring method described in the first aspect of the present invention.

[0045] The beneficial effects of this invention are as follows: by performing dense pixel-level motion vector calculation based on a deep learning optical flow estimation model, fine-grained motion features of smoke diffusion are extracted, improving the detection accuracy of smoke events. In particular, it can accurately capture the dynamic change features of smoke in complex dynamic backgrounds. Through spatiotemporal joint modeling, effective modeling of spatiotemporal information between image frames is achieved, enhancing the model's ability to capture the temporal continuity and spatial consistency in the smoke diffusion process, and reducing the occurrence of false alarms and missed alarms. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Fig. 1 This is a flowchart of an AI-based device security monitoring method.

[0048] Fig. 2 A flowchart for extracting characteristics of smoke diffusion.

[0049] Fig. 3 A flowchart for constructing a spatiotemporal joint feature vector sequence.

[0050] Fig. 4 A flowchart for temporal classification that includes attention mechanisms. Detailed Implementation

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0052] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0053] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0054] Reference Figs. 1-4This is one embodiment of the present invention, which provides an artificial intelligence-based device security monitoring method, including the following steps:

[0055] S1. Collect the monitoring video stream of the area to be monitored, perform standardization processing on the monitoring video stream, and obtain a continuous sequence of standardized image frames.

[0056] When collecting surveillance video streams of the area to be monitored, use a camera to acquire a continuous video stream covering the area to be monitored.

[0057] Furthermore, in areas such as data center server rooms where equipment is to be inspected, high-definition wide-angle cameras with IMX415 image sensor chips are deployed at key monitoring points to continuously collect continuous monitoring video streams of the area where equipment is to be inspected.

[0058] Image frames are extracted from the acquired surveillance video stream at fixed time intervals.

[0059] Furthermore, the continuous monitoring video stream is decoded and frames are extracted, and an image frame sequence with frame numbers is generated by using time intervals.

[0060] The image frame sequence is subjected to Gaussian filtering to remove noise, gamma transform to enhance low-brightness areas in the image, and pixel normalization to obtain an image frame sequence with consistent brightness distribution.

[0061] Furthermore, each frame in the image frame sequence is denoised using a 5×5 Gaussian kernel to eliminate CMOS thermal noise and compression artifacts.

[0062] After denoising, each frame of the image undergoes gamma transformation using a gamma correction function to enhance details in dark areas.

[0063] Based on the image frame sequence after gamma transformation, the mean and standard deviation of each image in the image frame sequence are obtained. Contrast normalization is performed based on the mean and standard deviation to obtain an image frame sequence with consistent brightness distribution.

[0064] Cropping removes non-target regions from the image frame sequence, retains the target region, and obtains a continuous, standardized image frame sequence.

[0065] Furthermore, to avoid interference with smoke detection caused by the structure of the device under test, shadows in the corners of walls, or out-of-focus areas at the edges of the camera in the monitoring screen, the field of view of each frame with consistent brightness distribution is optimized to obtain a continuous, standardized image frame sequence.

[0066] Specifically, the field of view optimization involves extracting stable regions of each frame image by calculating the average pixel value of consecutive frames; marking long-term static edge structures, such as device borders and corners, as non-interest regions and setting them as zero-response occlusion regions; then, focusing on the central region of the image frame, cropping pixel regions closer to the image edge in both the horizontal and vertical directions, retaining only the central region.

[0067] For example, by cropping out approximately 30% of the pixel area from each edge of the image, taking a 1920×1080 image as an example, the final retained area is 600 to 1320 pixels horizontally and 300 to 780 pixels vertically, meaning there is an effective central area of ​​720×480 pixels.

[0068] It should be noted that by performing occlusion removal and boundary cropping, pseudo-motion recognition caused by environmental changes at image edges can be reduced; the subsequent deep learning optical flow estimation model can focus on the main diffusion path area of ​​smoke, improving detection efficiency; at the same time, the computational complexity of the deep learning optical flow estimation model is reduced, improving the real-time performance and resource utilization of actual deployment.

[0069] S2. Construct a deep learning optical flow estimation model based on standardized image frame sequences. Use the deep learning optical flow estimation model to calculate the predicted optical flow motion vector for adjacent image frame pairs and extract the smoke diffusion characterization features.

[0070] Image frame pairs are constructed as training samples using a standardized sequence of image frames.

[0071] Furthermore, training sample pairs are constructed using the acquired standardized image frame sequences, with each sample pair consisting of two image frames.

[0072] Weakly supervised training datasets were constructed using standardized image frame sequences, training samples, and pseudo-labeled optical flow maps.

[0073] Furthermore, a weakly supervised training dataset is constructed by generating pseudo-label optical flow maps for each training sample pair using the classical optical flow estimation algorithm as a supervision signal.

[0074] Specifically, the pseudo-label optical flow map is generated using the classical optical flow estimation algorithm. The standardized image frame sequence is constructed into a training sample pair set according to the principle of temporal adjacency, and the training sample pair is processed using the TV-L1 optical flow estimation algorithm.

[0075] The goal of the TV-L1 optical flow algorithm is to solve the pseudo-tag optical flow map. Therefore, the objective optimization function of the TV-L1 optical flow algorithm is expressed as:

[0076] ;

[0077] ;

[0078] ;

[0079] in, Indicates the first The pseudo-label optical flow map corresponding to the training samples of the frame. This represents the set of coordinates of all pixels in the image of the training samples. This represents a penalty function for the pixel differences between images. Indicates pixel differences, Represents the optical flow gradient. Indicates the first Frame training samples at pixel positions brightness value, Indicates the first Pixel values ​​of training samples after optical flow displacement. Represents the optical flow vector field at the pixel position Spatial gradient at that location This represents the offset of the optical flow vector in the horizontal direction. This represents the offset of the optical flow vector in the vertical direction.

[0080] The objective optimization function of the TV-L1 optical flow algorithm is iteratively solved and normalized to obtain the pseudo-label optical flow map corresponding to each training sample.

[0081] It should be noted that the Huber loss function is used as the penalty function for the pixel differences between images.

[0082] It should be noted that each training sample corresponds to a pseudo-label optical flow map. Each pixel in the optical flow map of a pseudo-tag contains two channels, representing displacement components in the horizontal and vertical directions, respectively.

[0083] The RAFT-based deep learning optical flow estimation model was trained using a weakly supervised training dataset, and the training weights were saved.

[0084] Furthermore, the deep learning optical flow estimation model adopts a RAFT-based architecture and recursively optimizes the full-frame pixel matching structure of the weakly supervised training dataset.

[0085] The structured loss function of the deep learning optical flow estimation model is expressed as:

[0086] ;

[0087] in, Represents the structured loss function. Indicates the width of the image. Indicates the height of the image. Indicates the first Frame at pixel position The predicted optical flow motion vector at that location, This represents the second-order smoothing value of the predicted optical flow.

[0088] It should be noted that the predicted optical flow vector is... ,but .

[0089] The second-order smoothing value of the predicted optical flow is then defined as:

[0090] ;

[0091] in, Indicates the image number 1 Frame at pixel position Horizontal motion vector components, Indicates the image number 1 Frame at pixel position The vertical motion vector component, This represents the two-dimensional discrete Laplace operator.

[0092] The deep learning optical flow estimation model consists of an image feature extraction layer, a correlation volume generation layer, an optical flow update iteration layer, and an optical flow upsampling reconstruction layer.

[0093] Furthermore, the training process of the deep learning optical flow estimation model specifically involves using a set of training sample pairs as training input and a pseudo-labeled optical flow map as training supervision signal. The training sample pairs are input into the RAFT-structured deep learning optical flow estimation model. A two-dimensional image feature tensor is extracted through an image feature extraction layer. Then, a correlation generation layer is used in the feature space to calculate the similarity representation between all pixels, and the predicted optical flow vector is initialized. In the optical flow update iteration layer, multiple iterations are performed, combining the optical flow estimation state with contextual features. Finally, an optical flow upsampling reconstruction layer restores the original resolution to obtain the final predicted optical flow vector. The predicted optical flow motion vector is compared pixel-by-pixel with the pseudo-labeled optical flow map. Backpropagation and gradient updates are performed on all trainable parameters of the RAFT structure based on a structured loss function. The Adam optimizer is used for iterative parameter training until the structured loss function converges or reaches a preset number of iterations, completing the training. The parameter weights of the trained deep learning optical flow estimation model are then saved.

[0094] Image frame pairs are extracted from a standardized image frame sequence in chronological order and used as the input image frame pair set for a deep learning optical flow estimation model.

[0095] Furthermore, adjacent frames in a continuous, standardized image sequence are combined pairwise to form image frame pairs, which are then used as input to the deep learning estimation model.

[0096] A deep learning optical flow estimation model is used to perform forward inference on standardized image frames, and the predicted optical flow motion vector map atlas is output.

[0097] Furthermore, using a deep learning optical flow estimation model trained with pseudo-labeled optical flow maps, forward inference is performed on image frame pairs to obtain the optical flow estimation results for each pixel from... Frame to The two-dimensional motion vectors of the frame are used to construct a vector diagram for predicting optical flow motion.

[0098] The two-dimensional motion vectors are the horizontal motion vector component and the vertical motion vector component, respectively.

[0099] Local temporal stability statistics are performed on the predicted optical flow motion vector diagram to generate an optical flow stability matrix as the first smoke diffusion characterization feature.

[0100] Furthermore, within the time window, the predicted optical flow vector map calculates the standard deviation of the optical flow change at each pixel position in consecutive image frames using a local statistical calculator. The standard deviation of the predicted optical flow vector map is then used as a component of the optical flow stability matrix to generate the optical flow stability matrix.

[0101] The optical flow stability matrix is ​​used as the first characteristic of smoke diffusion.

[0102] The standard deviation is calculated as follows:

[0103] ;

[0104] in, Indicates the first Pixel position in the optical flow stability matrix of a frame The standard deviation, Indicates the timing window length. Indicates the first Image frames of time-to-time predicted optical flow motion vector diagrams at pixel positions The predicted optical flow motion vector value at that location, Indicates the first Pre-frame Frame at pixel position The average predicted optical flow motion vector value, This represents the Euclidean norm.

[0105] The orientation histogram of each pixel is calculated based on the predicted optical flow motion vector diagram, and the orientation distribution histogram is used as a second smoke diffusion characterization feature.

[0106] Furthermore, based on the predicted optical flow motion vector, the arctangent function is used to calculate the gradient direction angle of each pixel for the horizontal and vertical motion vector components in the predicted optical flow motion vector. The gradient direction angles of all pixels are counted within the neighborhood window centered on each pixel, and the gradient direction angles are quantized into several directional intervals to construct a direction histogram. The direction histogram represents the dominant direction distribution characteristics in the neighborhood of a pixel, and contains two directional indicators, namely the dominant direction weight and the direction information entropy.

[0107] Normalize the orientation histogram to obtain the orientation distribution histogram.

[0108] Furthermore, the dominant direction weight is used to reflect whether the current pixel region has a clear dominant motion direction; the direction information entropy is used to measure the degree of dispersion of the diffusion direction.

[0109] Specifically, to obtain the two directional indicators, the direction histogram is normalized to obtain the direction vector in the form of a probability distribution. Based on the normalized direction histogram, two directional indicators are calculated, namely the dominant direction weight and the direction information entropy; where the dominant direction weight is the direction vector with the highest probability.

[0110] Directional information entropy, expressed as:

[0111] ;

[0112] in, Indicates the first Frame-time pixel coordinates The directional information entropy at the location, This indicates the number of directional intervals for gradient direction angle quantization. Indicates the first Frame-time pixel coordinates First Normalized directional histogram values ​​for each directional interval. This represents a very small constant.

[0113] S3. Extract image texture features from the standardized image frame sequence, normalize and time-align the image texture features with the smoke diffusion characterization features, and construct a spatiotemporal joint feature vector sequence.

[0114] Local binary encoding is performed on standardized image frames to obtain image texture feature maps.

[0115] Furthermore, based on the acquisition of a standardized image frame sequence, image texture features are extracted based on each standardized image frame.

[0116] Among them, the image texture feature extraction adopts the local binary mode operator to perform pixel-level processing on each standardized image frame and extract the image texture feature map.

[0117] The image texture feature map is subjected to average pooling to generate a vector of image texture features.

[0118] The image texture feature vector and the smoke diffusion representation feature are normalized respectively.

[0119] By using a time window, the image texture feature vector and the smoke diffusion characterization feature are aligned according to the frame-level timestamp, forming time-aligned image texture feature sequences and smoke diffusion characterization feature sequences, respectively.

[0120] Furthermore, select the time window length. As the smallest unit of time alignment, the image texture feature vector and the smoke diffusion feature map are normalized respectively, and the time-aligned image texture feature set and smoke diffusion representation feature sequence are constructed.

[0121] The texture feature vector and smoke diffusion representation feature of each frame image are concatenated to construct a spatiotemporal joint feature vector sequence.

[0122] Furthermore, within each time window, the image texture feature vector in the image texture feature set is concatenated with the smoke diffusion representation feature vector in the smoke diffusion representation feature sequence to form a spatiotemporal joint feature vector, and a complete spatiotemporal joint feature vector sequence is constructed.

[0123] S4. The spatiotemporal joint feature vector sequence is used to classify and predict smoke events through a temporal classification model that includes an attention mechanism, predicting the type and confidence level of smoke events.

[0124] The spatiotemporal joint feature vector sequence is used as input, and the bidirectional gated loop structure in the temporal classification model containing the attention mechanism is used to encode the features step by step, generating a bidirectional GRU output sequence.

[0125] The bidirectional GRU output sequence is assigned attention weights for each frame through the attention mechanism in the temporal classification model to generate a global attention-weighted feature representation.

[0126] Furthermore, the spatiotemporal joint feature vector sequence is processed at the frame level to ensure sequence length consistency, and the spatiotemporal joint feature vector sequence is set as the input sequence of the temporal classification model that includes an attention mechanism.

[0127] A bidirectional gated loop structure incorporating an attention mechanism is constructed as a temporal classification model; the temporal classification model consists of a bidirectional temporal modeling layer, an attention mechanism layer, and a fully connected layer.

[0128] It should be noted that the main body of the time-series classification model consists of a bidirectional GRU structure and an attention-weighted convergence layer. The specific structural steps are as follows: A1-A2:

[0129] A1: In the temporal classification model, the spatiotemporal feature joint feature vector sequence is encoded by a bidirectional GRU structure.

[0130] Specifically, the input to the bidirectional GRU structure is set to a sequence of joint spatiotemporal feature vectors, denoted as . ,in, For the first The spatiotemporal joint feature vector of a frame, The time frame length, For feature dimensions.

[0131] A bidirectional gated loop structure is constructed, consisting of a forward GRU and a backward GRU, which respectively handle the forward and reverse order of the spatiotemporal feature joint feature vector sequence.

[0132] Ascending order processing, represented as:

[0133] ;

[0134] in, Indicates the first The forward-order GRU output of frames, Indicates the first The frame is output in ascending order by the GRU.

[0135] Reverse order processing, represented as:

[0136] ;

[0137] in, Indicates the first The reverse-order GRU output of frames. Indicates the first The reverse GRU output of the frames.

[0138] The forward and reverse GRU outputs obtained from the forward and reverse processing are concatenated to obtain a bidirectional context feature vector with a dimension of 2. , is represented as:

[0139] ;

[0140] in, Indicates the first The bidirectional contextual feature vector of a frame.

[0141] Finally, the bidirectional contextual feature vectors from all time points are combined into a bidirectional GRU output sequence, which is then transmitted to the attention-weighted convergence layer.

[0142] A2: An attention mechanism is used to assign weights to the bidirectional GRU output sequence at all time points, and an attention-weighted representation vector is calculated.

[0143] Specifically, an attention mechanism is constructed, which uses learnable weights to assign weights to the bidirectional GRU outputs in the bidirectional GRU output sequence at each time step, and calculates the attention score at each time step.

[0144] The attention score is calculated and represented as follows:

[0145] ;

[0146] in, The first in the attention mechanism Frame attention score, Represents the column vector of the science department. The mapping matrix representing the attention weights, This represents the learnable bias during the attention shift process. This represents the hyperbolic tangent activation function.

[0147] in, For learnable parameters, For attention hidden layer dimensions.

[0148] Softmax normalization is applied to all attention scores to obtain the attention weight at each time step. The bidirectional GRU output sequences at all time steps are weighted and summed according to their corresponding attention weights to obtain the global attention-weighted feature representation.

[0149] By passing the globally attention-weighted feature representation through a fully connected layer, the predicted probability distribution vector of the smoke event type is obtained.

[0150] Furthermore, the global attention-weighted feature representation is used as the input to the fully connected layer, which contains a weight matrix and a bias vector. A linear transformation is performed to generate a score vector corresponding to each smoke event type. The score vector is then passed through the Softmax function to calculate the predicted probability value of each smoke event type, resulting in a predicted probability distribution vector for the smoke event type. The value of each dimension in the predicted probability distribution vector for the smoke event type represents the confidence level of the corresponding smoke event type.

[0151] Specifically, in constructing the weight matrix, let the dimension of the global attention-weighted feature representation be... The number of categories of smoke events is The weight matrix size of the fully connected layer is then set to... A two-dimensional matrix.

[0152] Specifically, the bias vector is constructed by setting its dimension to the number of categories of smoke event types. The bias vector is constructed using an all-zero initialization strategy, which means that each dimension of the bias vector is assigned a value of zero.

[0153] It should be noted that smoke incidents are categorized into two types: non-alarm and alarm. Non-alarm incidents include normal operation without smoke, dust and smoke interference, and abnormal white smoke leakage. Alarm incidents include black smoke emission or overheating smoke, initial smoke from open flames, and severe smoke obstruction.

[0154] Among them, normal operation without smoke means that no smoke is detected in the monitored area and the equipment is working normally; dust and smoke interference means that dust disturbance caused by fans, air conditioners, etc., creates a false alarm risk of smoke-like images; abnormal white smoke leakage means that steam-like white smoke is produced by leakage from the cooling system or high-pressure pump, indicating abnormal cooling; black smoke emission or overheating smoke means that the power supply, cable or circuit board is in the early stage of overheating and burning, releasing tar black smoke; open flame initial smoke means that combustibles are in the early stage of combustion, accompanied by local temperature rise and gray-black smoke, which is a precursor to fire; severe smoke obstruction means that the smoke has obviously obscured the monitoring lens, the specific status of the equipment is limited, and an emergency response is required.

[0155] The smoke event type with the highest probability is extracted from the predicted probability distribution vector as the final predicted smoke event type, and the highest probability of the final predicted smoke event type is used as the confidence level.

[0156] It should be noted that the training process of the temporal classification model specifically involves using the constructed spatiotemporal joint feature vector sequence as input samples and manually labeled smoke event types as supervision signals to perform end-to-end training on a bidirectional gated loop structure that includes an attention mechanism. During training, the cross-entropy loss function is used to compare the predicted smoke events with the true labels, and the parameters of the temporal classification model are continuously updated through backpropagation and gradient descent until the loss function converges or the maximum number of iterations is reached, thus completing the training.

[0157] S5. Determine the trigger of a smoke event based on the type and confidence level of the smoke event, and trigger different levels of security response strategies.

[0158] Define a set of confidence thresholds for smoke event response levels.

[0159] Furthermore, based on the smoke event type and corresponding confidence level output by the time series classification model, a set of confidence level thresholds is set; wherein, the set of confidence level thresholds includes low-level thresholds, medium-level thresholds, and high-level thresholds.

[0160] It should be noted that the confidence threshold is determined by setting the confidence level based on the prediction confidence level in historical training data.

[0161] The response level for a smoke event is determined based on the predicted type and confidence level of the smoke event.

[0162] Furthermore, the response level for the smoke event is determined using a response decision function.

[0163] Smoke incident response levels include low-level response, medium-level response, and high-level response.

[0164] The response determination function is expressed as:

[0165] ;

[0166] in, Indicates the type of smoke event. This represents the prediction confidence level corresponding to the type of smoke event. Indicates an ignored response level. Indicates a low-level response. Indicates a medium-level response. Indicates a high-level response. Indicates non-alarm type. Indicates alarm type, Indicates a low-level threshold. Indicates the medium-level threshold. This indicates a high-level threshold.

[0167] The security response strategy type is determined based on the smoke event response level, and the corresponding security response strategy is triggered.

[0168] Furthermore, based on the response level of the smoke event triggered by the response determination function, a security response strategy is determined.

[0169] Specifically, the safety response strategy is as follows: if the smoke event response level is ignored, no action is taken; if the smoke event response level is low, alarm image marking and log recording are performed; if the smoke event response level is medium, a warning notification is sent to the management terminal and a high-frequency screenshot sequence is saved while executing the low-level response strategy; if the smoke event response level is high, the fire alarm controller is immediately activated, local fire suppression equipment is started, and a sequence of monitoring image frames from the past 10 minutes is retrieved for manual review.

[0170] This embodiment also provides a computer device applicable to the device security monitoring method based on artificial intelligence, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the device security monitoring method based on artificial intelligence as proposed in the above embodiment.

[0171] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0172] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the artificial intelligence-based device security monitoring method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0173] In summary, this invention achieves fine-grained motion feature extraction of smoke diffusion by performing dense pixel-level motion vector calculation based on a deep learning optical flow estimation model, thus improving the detection accuracy of smoke events. Particularly in complex dynamic backgrounds, it can accurately capture the dynamic changes of smoke. Furthermore, through spatiotemporal joint modeling, it effectively models the spatiotemporal information between image frames, enhancing the model's ability to capture the temporal continuity and spatial consistency of the smoke diffusion process, and reducing false positives and false negatives.

[0174] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A device security monitoring method based on artificial intelligence, characterized in that: include, The system collects surveillance video streams of the area to be monitored, performs standardization processing on the surveillance video streams, and obtains a continuous sequence of standardized image frames. A deep learning optical flow estimation model is constructed based on a standardized image frame sequence. The deep learning optical flow estimation model is used to calculate the predicted optical flow motion vector of adjacent image frame pairs and extract the smoke diffusion characterization features. Image texture features are extracted from a standardized image frame sequence, and the image texture features are normalized and time-aligned with the smoke diffusion characterization features to construct a spatiotemporal joint feature vector sequence. The spatiotemporal joint feature vector sequence is used to classify and predict smoke events through a temporal classification model that includes an attention mechanism, predicting the type and confidence level of smoke events. The system determines whether a smoke event is triggered based on the type and confidence level of the smoke event, and then triggers different levels of security response strategies.

2. The device security monitoring method based on artificial intelligence as described in claim 1, characterized in that: The process involves collecting the monitoring video stream of the area to be monitored, standardizing the video stream, and obtaining a continuous sequence of standardized image frames. The specific steps are as follows: When collecting surveillance video streams of the area to be monitored, use a camera to acquire a continuous video stream covering the area to be monitored; Image frames are extracted from the acquired surveillance video stream at fixed time intervals. Gaussian filtering is applied to the image frame sequence to remove noise, gamma transform is applied to the image frame sequence to enhance low-brightness areas in the image, and pixel normalization is applied to the image frame sequence to obtain an image frame sequence with consistent brightness distribution. Cropping removes non-target regions from the image frame sequence, retains the target region, and obtains a continuous, standardized image frame sequence.

3. The device security monitoring method based on artificial intelligence as described in claim 2, characterized in that: The specific steps for constructing a deep learning optical flow estimation model based on standardized image frame sequences are as follows: Image frame pairs are constructed as training samples using a standardized sequence of image frames. A weakly supervised training dataset was constructed using standardized image frame sequences and pseudo-labeled optical flow maps to create training samples and a training dataset. The RAFT-based deep learning optical flow estimation model was trained using a weakly supervised training dataset, and the training weights were saved. The deep learning optical flow estimation model consists of an image feature extraction layer, a correlation volume generation layer, an optical flow update iteration layer, and an optical flow upsampling reconstruction layer.

4. The device security monitoring method based on artificial intelligence as described in claim 3, characterized in that: The method employs a deep learning optical flow estimation model to calculate the predicted optical flow motion vectors for adjacent image frame pairs and extracts smoke diffusion characterization features. The specific steps are as follows: Image frame pairs are extracted from a standardized image frame sequence in chronological order and used as the input image frame pair set for a deep learning optical flow estimation model. A deep learning optical flow estimation model is used to perform forward inference on standardized image frames, and the predicted optical flow motion vector map atlas is output. Local temporal stability statistics are performed on the predicted optical flow motion vector diagram to generate an optical flow stability matrix as the first smoke diffusion characterization feature. The orientation histogram of each pixel is calculated based on the predicted optical flow motion vector diagram, and the orientation distribution histogram is used as a second smoke diffusion characterization feature.

5. The device security monitoring method based on artificial intelligence as described in claim 4, characterized in that: The steps involve extracting image texture features from a standardized image frame sequence, normalizing and temporally aligning these features with smoke diffusion characterization features, and constructing a spatiotemporal joint feature vector sequence. Local binary encoding is performed on standardized image frames to obtain image texture feature maps; The image texture feature map is subjected to average pooling to generate a vector of image texture features; Normalize the image texture feature vector and the smoke diffusion representation feature respectively; By using a time window, the image texture feature vector and the smoke diffusion characterization feature are aligned according to the frame-level timestamp, forming time-aligned image texture feature sequences and smoke diffusion characterization feature sequences, respectively. The texture feature vector and smoke diffusion representation feature of each frame image are concatenated to construct a spatiotemporal joint feature vector sequence.

6. The device security monitoring method based on artificial intelligence as described in claim 5, characterized in that: The temporal classification model incorporating an attention mechanism consists of a bidirectional temporal modeling layer, an attention mechanism layer, and a fully connected layer.

7. The device security monitoring method based on artificial intelligence as described in claim 6, characterized in that: The specific steps for classifying and predicting smoke events by applying a temporal classification model incorporating an attention mechanism to the spatiotemporal joint feature vector sequence, and predicting the smoke event type and confidence level, are as follows: The spatiotemporal joint feature vector sequence is used as input, and the bidirectional gated loop structure in the temporal classification model containing the attention mechanism is used to encode the features step by step to generate the bidirectional GRU output sequence. The bidirectional GRU output sequence is assigned attention weights for each frame through the attention mechanism in the temporal classification model to generate a global attention-weighted feature representation; The global attention-weighted feature representation is passed through a fully connected layer to obtain the predicted probability distribution vector of the smoke event type; The smoke event type with the highest probability is extracted from the predicted probability distribution vector as the final predicted smoke event type, and the highest probability of the final predicted smoke event type is used as the confidence level.

8. The device security monitoring method based on artificial intelligence as described in claim 7, characterized in that: The specific steps for determining smoke event triggering based on smoke event type and confidence level, and triggering different levels of security response strategies, are as follows: By analyzing the confidence levels of predictions in historical training data, a set of confidence thresholds for smoke event response levels is set. Determine the response level for a smoke event based on the predicted type and confidence level. The security response strategy type is determined based on the smoke event response level, and the corresponding security response strategy is triggered.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the artificial intelligence-based device security monitoring method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the artificial intelligence-based device security monitoring method according to any one of claims 1 to 8.

Images