Improved I3D network model and abnormal flame identification method based on same

Abstract

The invention provides an improved I3D network model and an abnormal flame identification method based on the improved I3D network model, which are used for solving the technical problem that the accuracy and the reliability of an existing flame detection method are difficult to support because more complicated video data cannot be analyzed and processed in real time due to the fact that analysis data are mainly pictures. According to the improved I3D network model, a space-time adaptive defogging module is added at the front end of a traditional I3D network model to carry out defogging processing on an input video so as to reduce haze interference; meanwhile, an attention mechanism model integrating a space attention mechanism, a time attention mechanism and a highlight perception channel attention mechanism is constructed at the position, close to the rear end, of the traditional I3D network model, so that features in the video are better captured, and the sensitivity of the traditional I3D network model to flame changes is enhanced.

Description

Technical Field

[0001] The present invention relates to an abnormal flame identification method, and in particular to an improved I3D network model and an abnormal flame identification method based on the model. Background Art

[0002] When it comes to flame detection, traditional image processing algorithms typically rely on color and motion features, which can be difficult to detect in complex scenes with diverse backgrounds. To improve the accuracy and real-time performance of flame detection, the industry is exploring the use of advanced technologies and deep learning models for flame detection.

[0003] Li et al. used spatiotemporal feature fusion to address the problem of identifying dynamic changes in flames against complex backgrounds. See Li Z, Gavrilyuk K, Gavves E, et al., “VideoLSTM convolves, attends and flows for action recognition,” Computer Vision and Image Understanding, vol. 166, pp. 4150, 2018. Kim et al. (2019) used a long short-term memory (LSTM) network to summarize features within a bounding box in a short time sequence of consecutive frames to determine the presence of a fire and classify it. They then merged most consecutive short-term decisions together and determined the final result through long-term voting. See Kim B, Lee J, “A video-based fire detection using deep learning models,” Applied Sciences, vol. 9, no. 14, pp. 2862-2881, 2019. Dimitropoulos et al. proposed a method that combines spatiotemporal flame modeling and dynamic texture analysis. See Dimitropoulos K, Barmpoutis P, Grammalidis N, “Spatio-temporal flame modeling and dynamic texture analysis for automatic video-based fire detection,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 2, pp. 339-351, 2015. DOI: 10.1109 / TCSVT.2014.2339592. Yuming et al. focused more on simplifying the process of fire and smoke detection. They proposed a DETR-based object detection model that combines a convolutional neural network (CNN) and a transformer.To improve small object detection performance and speed up model training, a normalized attention module was added to the feature extraction stage, and multi-scale deformable attention was used in the encoder-decoder architecture. See Yuming L, Wei Z, Yanyan L, et al., “An efficient fire and smoke detection algorithm based on an end-to-end structured network,” Engineering Applications of Artificial Intelligence, vol. 116, 2022. Hikmat et al. proposed a modified YOLOv5s model that integrates a Stem module, replaces larger kernels with smaller kernels in the SPP (neck), and adds a P6 module to the head. See Hikmat Y, Ahmad ZK, Min UFU, et al., “A modified YOLOv5 architecture for efficient fire detection in smart cities,” Expert Systems With Applications, vol. 231, 2023. This design aims to reduce model complexity and size while being able to detect both small and large fire areas in images. In addition, they provided a medium-sized fire dataset consisting of three categories: vehicle fires, building fires, and indoor electrical fires. Each of the above methods has its own advantages and disadvantages, but the data they analyze is mainly based on images. In reality, more complex video data needs to be analyzed and processed in real time, which increases the complexity and difficulty of processing, making it difficult to support the accuracy and reliability of these methods. Summary of the Invention

[0004] The purpose of the present invention is to solve the technical problem that the existing flame detection methods have the problem that the analysis data are mainly based on images, which cannot be analyzed and processed in real time with more complex video data, making it difficult to support their accuracy and reliability. An improved I3D network model and an abnormal flame recognition method based on the model are provided.

[0005] In order to achieve the above object, the technical solution provided by the present invention is as follows: An improved I3D network model, which is special in that it includes a spatiotemporal adaptive dehazing module, an I3D network model, and an attention mechanism model; The input end of the spatiotemporal adaptive defogging module serves as the input end of the improved I3D network model, and is used to receive external input video and combine the time information of the video with the traditional atmospheric scattering model to obtain a dynamic atmospheric scattering model to perform defogging on the input video; The input end of the I3D network model is connected to the output end of the spatiotemporal adaptive defogging module; The attention mechanism model is set in the middle of the I3D network model, including a spatial attention mechanism module, a temporal attention mechanism module, a highlight perception channel attention mechanism module and a fusion module; the spatial attention mechanism module is used to adaptively assign weights to each spatial position; the temporal attention mechanism module is used to adaptively assign weights to different time frames; the highlight perception channel attention mechanism module is used to fuse channel features with brightness features to dynamically adjust channel weights; the input end of the fusion module is respectively connected to the output end of the spatial attention mechanism module, the temporal attention mechanism module and the highlight perception channel attention mechanism module, and is used to perform feature fusion on the output features of the spatial attention mechanism module, the temporal attention mechanism module and the highlight perception channel attention mechanism module; The output end of the I3D network model serves as the output end of the improved I3D network model and is used to output feature information.

[0006] Furthermore, the I3D network model includes a first three-dimensional convolutional layer, a second three-dimensional convolutional layer, a third three-dimensional convolutional layer, a fourth three-dimensional convolutional layer, a fifth three-dimensional convolutional layer, a first three-dimensional pooling layer, a second three-dimensional pooling layer, a third three-dimensional pooling layer, a fourth three-dimensional pooling layer, a first mixed pooling layer, a second mixed pooling layer, a third mixed pooling layer, an average pooling layer, a batch normalization layer and an output layer; The first three-dimensional convolution layer, the first three-dimensional pooling layer, the second three-dimensional convolution layer, the third three-dimensional convolution layer, the second three-dimensional pooling layer, the first mixed pooling layer, the third three-dimensional pooling layer, the second mixed pooling layer, the fourth three-dimensional pooling layer, and the third mixed pooling layer are connected in sequence in the order of input and output; the input end of the first three-dimensional convolution layer is connected to the output end of the spatiotemporal adaptive dehazing module, and the output end of the third mixed pooling layer is connected to the input ends of the spatial attention mechanism module, the temporal attention mechanism module, and the highlight perception channel attention mechanism module respectively; The fourth three-dimensional convolution layer, batch normalization layer, average pooling layer, fifth three-dimensional convolution layer and output layer are connected in sequence in the order of input and output; the input end of the fourth three-dimensional convolution layer is connected to the output end of the fusion module; the output layer serves as the output end of the I3D network model.

[0007] Furthermore, the spatiotemporal adaptive defogging module includes an input layer, a spatiotemporal pooling layer, a long short-term memory neural network layer, and a model generation and output layer, which are sequentially connected in input and output order; The input layer serves as the input end of the spatiotemporal adaptive defogging module and is used to receive external input video; The spatiotemporal pooling layer is used to perform an average pooling operation on the spatial dimension of the input video to retain channel and time information; The long short-term memory neural network layer is used to capture temporal dynamic features; The model generation and output layer is used to predict the atmospheric scattering parameters at each time step, and generate a dynamic atmospheric scattering model based on the atmospheric scattering parameters at each time step, and then output the dehazed video. Its output end is connected to the input end of the first three-dimensional convolutional layer.

[0008] Furthermore, the expression of the dynamic atmospheric scattering model is as follows:

[0009] in, This is the video after dehazing. is the foggy image sequence of the input video, is the global atmospheric light value, is the scattering coefficient, For depth estimation, is the two-dimensional space coordinate, is the time step index, ε is the numerical guard, ε=1×10 −6 .

[0010] Furthermore, the highlight perception channel attention mechanism module includes an attention input layer, a global average pooling layer, a global maximum pooling layer, a brightness feature convolution layer, a averaging layer, a splicing layer, a first fully connected layer, a second fully connected layer, and an expansion layer; The input end of the attention input layer is connected to the output end of the third hybrid pooling layer, and is used to receive the feature data output by the third hybrid pooling layer; the input ends of the global average pooling layer, the global maximum pooling layer, and the brightness feature convolution layer are respectively connected to the output end of the attention input layer, and are used to perform global average pooling, global maximum pooling, and brightness feature convolution on the feature data output by the third hybrid pooling layer; the input end of the average layer is connected to the output end of the brightness feature convolution layer, and is used to average the features after the brightness feature convolution along the time, height, and width dimensions to generate a global brightness feature; The input end of the splicing layer is connected to the output end of the global average pooling layer, the global maximum pooling layer and the average layer respectively, and is used to splice the features after global average pooling, the features after global maximum pooling and the global brightness features to form a comprehensive feature; The input end of the first fully connected layer is connected to the output end of the splicing layer, which is used to compress and nonlinearly fuse the comprehensive features; the input end of the second fully connected layer is connected to the output end of the first fully connected layer, which is used to generate channel weights based on the comprehensive features after compression and nonlinear fusion; the input end of the expansion layer is connected to the output end of the second fully connected layer, which is used to dimensionalize the generated channel weights, and its output end is connected to the input end of the fusion module.

[0011] Furthermore, the activation function of the first fully connected layer adopts a ReLU function; The activation function of the second fully connected layer adopts the Sigmoid function.

[0012] Furthermore, the first three-dimensional pooling layer, the second three-dimensional pooling layer, the third three-dimensional pooling layer, and the fourth three-dimensional pooling layer are all three-dimensional maximum pooling layers; The activation function of the batch normalization layer adopts the ReLU function; The loss function of the I3D network model adopts the cross entropy loss function.

[0013] Furthermore, the convolution kernel size of the first three-dimensional convolution layer is 7×7×7, the convolution kernel size of the second three-dimensional convolution layer is 1×1×1, the convolution kernel sizes of the third three-dimensional convolution layer and the fourth three-dimensional convolution layer are both 3×3×3, and the convolution kernel size of the fifth three-dimensional convolution layer is 1×1×1.

[0014] In addition, the present invention also provides an abnormal flame identification method based on the improved P3D network model, which is special in that it includes the following steps: Step 1: Collect flame videos of different categories and construct training sets and test sets based on the collected flame videos; Step 2: Build the above-mentioned improved I3D network model; Step 3: Input the training set constructed in step 1 into the improved I3D network model constructed in step 2 for training to obtain a trained improved I3D network model; test the trained improved I3D network model using the test set established in step 1 to obtain a tested improved I3D network model; Step 4: Input the video image of the target to be identified into the tested improved I3D network model, and realize abnormal flame recognition based on the output result of the tested improved I3D network model.

[0015] Furthermore, in step 1, the flame videos in the training set cover four types of flame situations.

[0016] The beneficial effects of the present invention compared to the prior art are as follows: 1. The improved I3D network model provided by the present invention adds a spatiotemporal adaptive dehazing module to the front end of the traditional I3D network model to dehaze the input video to reduce haze interference; at the same time, an attention mechanism model integrating spatial attention mechanism, temporal attention mechanism and highlight perception channel attention mechanism is constructed near the back end of the traditional I3D network model to better capture the features in the video and enhance the sensitivity of the traditional I3D network model to flame changes.

[0017] 2. The spatiotemporal adaptive dehazing module in this invention combines the traditional atmospheric scattering model with time information, greatly enhancing the effectiveness of the subsequent attention mechanism model.

[0018] 3. The highlight perception channel attention mechanism module in the present invention is an attention mechanism based on highlight perception designed for flame characteristics. The brightness features extracted by it have an enhancing effect on the channel attention highlight area, which is beneficial to improving the accuracy of network recognition.

[0019] 4. Compared with conventional abnormal flame recognition methods based on image information, the abnormal flame recognition method based on the improved I3D network model provided by the present invention innovatively uses video information directly for recognition from the perspective of computer vision. At the same time, based on the above-mentioned improved I3D network model, it can meet the needs of abnormal fire recognition in complex backgrounds, with an identification accuracy of 97.78%, and can be used for real-time monitoring in forests and urban areas.

[0020] 5. When constructing the training set for the improved I3D network model, the present invention collected 8,410 videos of flames under different combustion conditions based on four types of flame situations. This can accurately train the improved I3D network model and improve the accuracy of fire warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Schematic diagram of the improved 13D network model in step 2 of an embodiment of an abnormal flame identification method based on an improved 13D network model of the present invention; Figure 2 Schematic block diagram of the spatiotemporal adaptive defogging module in step 2 of an embodiment of the present invention; Figure 3 This is a schematic block diagram of the highlight perception channel attention mechanism module in step 2 of an embodiment of the present invention.

[0022] The specific reference numerals are as follows: 1. First 3D convolution layer; 2. Second 3D convolution layer; 3. Third 3D convolution layer; 4. Fourth 3D convolution layer; 5. Fifth 3D convolution layer; 6. First 3D pooling layer; 7. Second 3D pooling layer; 8. Third 3D pooling layer; 9. Fourth 3D pooling layer; 10. First mixed pooling layer; 11. Second mixed pooling layer; 12. Third mixed pooling layer; 13. Average pooling layer; 14. Batch normalization layer; 15. Output layer; 16. Attention mechanism model; 161. Spatial attention mechanism module; 162. Temporal attention mechanism module; 163. Highlight perception channel attention mechanism module; 1630. Attention input layer; 1631. Global average pooling layer; 1632. Global maximum pooling layer; 1633. Brightness feature convolution layer; 1634. Average layer; 1635. Splicing layer; 1636. First fully connected layer; 1637. Second fully connected layer; 1638. Extension layer; 164. Fusion module; 17. Spatiotemporal adaptive dehazing module; 171. Input layer; 172. Spatiotemporal pooling layer; 173. Long short-term memory neural network layer; 174. Model generation and output layer. DETAILED DESCRIPTION

[0023] In order to make the advantages and features of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] An abnormal flame recognition method based on an improved I3D network model specifically includes the following steps: Step 1: Construct training and test sets.

[0025] In this example, 8,410 videos of flames under different combustion conditions were collected. These videos cover four types of fire scenarios: abnormal fire / smoke in the wild, abnormal fire / smoke in residential areas, normal fire / smoke, and no fire / smoke. Based on these 8,410 flame videos, training and test sets for the improved I3D network model were constructed. This allows for precise training of the improved I3D network model and improves fire warning accuracy.

[0026] Step 2: Build an improved I3D network model.

[0027] The I3D network model excels in video analysis tasks because it effectively captures temporal and spatial features in videos. Its compact structure and powerful functionality enable it to maintain high efficiency and accuracy even when processing large video datasets. The I3D network model's design enhances generalization and nonlinear representation capabilities. Its 3D convolutional layer is based on a 2D convolutional network (such as Inception-v1), expanding the 2D convolution kernels to 3D ones through inflation to process consecutive frames of video data. It comprises multiple 3D Inception modules, each containing multiple 3D convolution kernels of varying sizes to capture multi-scale spatiotemporal features. Inception is a classic convolutional neural network (CNN) architecture, and the 3D Inception module is an extension of the Inception architecture to 3D CNNs. However, the I3D network model has a relatively high number of parameters and computational complexity, so model compression or optimization may be necessary when deployed in resource-constrained environments. However, given sufficient computing resources, the I3D network model can provide powerful video analysis capabilities and demonstrate strong robustness when it comes to flame recognition in complex and noisy scenes. Given that the focus of this paper is video analysis, specifically the identification of flame dynamics for fire detection, and that the video database contains thousands of video clips, this paper uses a CNN-based I3D network model as its foundational model.

[0028] To further improve the accuracy of flame recognition, the present invention has improved the I3D network model, referred to as the improved I3D network model. First, to reduce haze interference, a spatiotemporal adaptive dehazing module (ST-Dehazed) 17 is added to the front-end of the I3D network model to dehaze the input video. Furthermore, the present invention constructs an attention mechanism model 16 near the back-end of the I3D network model that integrates spatial attention, temporal attention, and highlight-aware channel attention mechanisms to better capture video features and enhance the I3D network model's sensitivity to flame changes. This spatiotemporal adaptive dehazing module 17 combines traditional atmospheric scattering models with temporal information, thereby enhancing the effectiveness of the subsequent attention mechanism model 16.

[0029] like Figure 1As shown, the improved I3D network model includes a spatiotemporal adaptive dehazing module 17, an I3D network model, and an attention mechanism model 16. The I3D network model includes a first 3D convolutional layer 1, a second 3D convolutional layer 2, a third 3D convolutional layer 3, a fourth 3D convolutional layer 4, a fifth 3D convolutional layer 5, a first 3D pooling layer 6, a second 3D pooling layer 7, a third 3D pooling layer 8, a fourth 3D pooling layer 9, a first mixed pooling layer 10, a second mixed pooling layer 11, a third mixed pooling layer 12, an average pooling layer 13, a batch normalization layer 14, and an output layer 15. The first 3D convolutional layer 1, the first 3D pooling layer 6, the second 3D convolutional layer 2, the third 3D convolutional layer 3, the second 3D pooling layer 7, the first mixed pooling layer 10, the third 3D pooling layer 8, the second mixed pooling layer 11, the fourth 3D pooling layer 9, and the third mixed pooling layer 12 are connected in sequence according to the order of input and output. The input end of the first three-dimensional convolution layer 1 is connected to the output end of the spatiotemporal adaptive dehazing module 17, and the output end of the third hybrid pooling layer 12 is connected to the input end of the attention mechanism model 16; the fourth three-dimensional convolution layer 4, the batch normalization layer 14, the average pooling layer 13, the fifth three-dimensional convolution layer 5 and the output layer 15 are connected in sequence in the order of input and output; the input end of the fourth three-dimensional convolution layer 4 is connected to the output end of the attention mechanism model 16; the output layer 15 serves as the output end of the I3D network model for outputting feature information.

[0030] The first three-dimensional convolution layer 1, the second three-dimensional convolution layer 2, the third three-dimensional convolution layer 3, the fourth three-dimensional convolution layer 4, and the fifth three-dimensional convolution layer 5 are used to perform three-dimensional convolution on the input video; in this embodiment, the convolution kernel size of the first three-dimensional convolution layer 1 is 7×7×7, the convolution kernel size of the second three-dimensional convolution layer 2 is 1×1×1, the convolution kernel sizes of the third three-dimensional convolution layer 3 and the fourth three-dimensional convolution layer 4 are both 3×3×3, and the convolution kernel size of the fifth three-dimensional convolution layer 5 is 1×1×1. The first 3D pooling layer 6, the second 3D pooling layer 7, the third 3D pooling layer 8, and the fourth 3D pooling layer 9 are all 3D max pooling layers, used to cooperate with global max pooling to reduce the dimensionality of the feature map. The first hybrid pooling layer 10, the second hybrid pooling layer 11, the third hybrid pooling layer 12, and the average pooling layer 13 are used to cooperate with global average pooling. In this embodiment, the window size of the first 3D pooling layer 6, the second 3D pooling layer 7, the third 3D pooling layer 8, the fourth 3D pooling layer 9, the first hybrid pooling layer 10, the second hybrid pooling layer 11, the third hybrid pooling layer 12, and the average pooling layer 13 is all 3×3×3. The batch normalization layer 14 is used to accelerate the training process of the improved I3D network model and improve its stability. The activation function of the batch normalization layer 14 uses the ReLU function. In addition, the choice of loss function is also crucial. Based on experimental results, the present invention selects the cross-entropy loss function to provide better video recognition robustness for the improved I3D network model.

[0031] The spatiotemporal adaptive defogging module 17 is used to receive external input videos, combine the time information of the videos with the traditional atmospheric scattering model, perform deep learning, and obtain a dynamic atmospheric scattering model to defog the input videos. Figure 2 As shown, the spatiotemporal adaptive defogging module 17 includes an input layer 171, a spatiotemporal pooling layer 172, a long short-term memory neural network ( Layer 173 and model generation and output layer 174.

[0032] Input layer 171 serves as the input of spatiotemporal adaptive defogging module 17, i.e., the input of the improved 13D network model, and is used to receive external input videos. When the improved 13D network model is being trained, input layer 171 receives the video input of the training set constructed in step 1. When the improved 13D network model is being used to identify abnormal flames, input layer 171 receives the video of the target to be identified.

[0033] The spatiotemporal pooling layer 172 is used to perform an average pooling operation on the spatial dimension of the input video to retain the channel and time information, as shown in formula (1).

[0034] (1) in, is the feature sequence obtained after spatial dimension average pooling, with a size of , H is the spatial height of the input feature map, W is the spatial width of the input feature map, is the eigenvalue of the original input video at the Cth channel, Tth frame, hth row, and wth column, R is the real number domain, B is the batch size (i.e., the number of samples), C is the number of channels, and T is the time dimension.

[0035] The long short-term memory neural network layer 173 is used to capture the dynamic characteristics of time series. After transposition, the transposed data is input into the long short-term memory neural network layer 173, and then: (2) in, 173 pairs of feature sequences for the long short-term memory neural network layer The output feature sequence at each time step has the size , is the hidden state vector corresponding to the Tth time step (the last frame), is the memory unit (cell state) vector corresponding to the Tth time step, and the hidden layer dimension of the long short-term memory neural network layer 173 is 64.

[0036] The model generation and output layer 174 is used to predict the atmospheric scattering parameters at each time step, and generate a dynamic atmospheric scattering model based on the atmospheric scattering parameters at each time step, and then output the defogging video. The atmospheric scattering parameters include the scattering coefficient , depth estimation and global atmospheric light values ; The scattering coefficient Defined as , where σ is the Sigmoid function, is the weight matrix of the scattering coefficient. Depth estimation and global atmospheric light values The normalized atmospheric light intensity is simulated, where , W d is the weight matrix for depth estimation, W A is the weight matrix of the global atmospheric light value, then the transmittance , the dehazed video output after model generation and output layer 174 for: (3) in, is the foggy image sequence of the input video, is the two-dimensional space coordinate, is the time step index, and ε is the numerical guard, which is used to prevent division by zero errors and ensure computational stability. Usually ε=1×10 −6 .

[0037] The output video tensor performs dehazing and restoration for each sample, each channel, each frame, and each pixel.

[0038] The spatiotemporal adaptive dehazing module 17 performs dehazing processing on the input video. The dehazed video features reduce the subsequent haze interference in the attention mechanism model 16, enhance the key features, that is, the spatiotemporal key areas are more prominent, and provide high signal-to-noise ratio input for subsequent processes, so that the attention mechanism model 16 can more intuitively capture the spatiotemporal key areas, thereby improving the accuracy of video target recognition.

[0039] The attention mechanism model 16 of the present invention includes a spatial attention mechanism module 161, a temporal attention mechanism module 162, a highlight-aware channel attention mechanism module 163, and a fusion module 164. The spatial attention mechanism module 161 is used to adaptively assign weights to each spatial position; the temporal attention mechanism module 162 is used to adaptively assign weights to different time frames; and the highlight-aware channel attention mechanism module (CHAM) 163 is used to fuse channel features with brightness features to dynamically adjust channel weights. Both the spatial attention mechanism module 161 and the temporal attention mechanism module 162 utilize existing attention mechanisms, while the highlight-aware channel attention mechanism module 163 is a highlight-aware attention mechanism designed for flame characteristics. The inputs of the spatial attention mechanism module 161, the temporal attention mechanism module 162, and the highlight-aware channel attention mechanism module 163 are all connected to the output of the third hybrid pooling layer 12 to implement feature filtering in the spatial, temporal, and brightness aspects of the dehazed video. Let the video input to the attention mechanism model 16 be X, and X∈R B×C×T×H×W , the expressions for extracting features by the spatial attention mechanism module 161, the temporal attention mechanism module 162, and the highlight perception channel attention mechanism module 163 are as follows: F avg =GAP(X), F max =GMP(X), F light =C 1×1×1 (X) (4) Among them, F avg represents the features output by the spatial attention mechanism module 161, GAP(X) represents the global average pooling of the video X in the spatial dimension, and F max represents the features output by the temporal attention mechanism module 162, GMP(X) represents the global maximum pooling of video X in the temporal dimension, and F light represents the features output by the highlight perception channel attention mechanism module 163, C 1×1×1 (X) represents global average pooling of the brightness features of video X in the spatial dimension, that is, a 1×1×1 convolution operation.

[0040] The brightness features extracted by the highlight perception channel attention mechanism module 163 have an enhancing effect on the channel attention highlight area, which is conducive to improving the accuracy of network recognition. The following focuses on the highlight perception channel attention mechanism module 163 in the present invention: like Figure 3As shown, the highlight perception channel attention mechanism module 163 includes an attention input layer 1630, a global average pooling layer 1631, a global maximum pooling layer 1632, a brightness feature convolution layer 1633, an averaging layer 1634, a splicing layer 1635, a first fully connected layer 1636, a second fully connected layer 1637 and an expansion layer 1638.

[0041] The attention input layer 1630 serves as the input end of the highlight perception channel attention mechanism module 163, and its input end is connected to the output end of the third hybrid pooling layer 12, and is used to receive the feature data output by the third hybrid pooling layer 12. The input ends of the global average pooling layer 1631, the global maximum pooling layer 1632, and the brightness feature convolution layer 1633 are respectively connected to the output end of the attention input layer 1630, and are used to perform global average pooling, global maximum pooling, and brightness feature convolution on the feature data output by the third hybrid pooling layer 12; the input end of the averaging layer 1634 is connected to the output end of the brightness feature convolution layer 1633, and is used to average the features after brightness feature convolution along the time, height, and width dimensions to generate global brightness features, that is, perform global average pooling on the brightness features of video X in the spatial dimension to obtain F light ∈R B×1 The input end of the splicing layer 1635 is connected to the output end of the global average pooling layer 1631, the global maximum pooling layer 1632, and the averaging layer 1634, respectively, and is used to compress and splice the features after global average pooling, the features after global maximum pooling, and the global brightness features to generate comprehensive features. In this embodiment, the compression multiple of the features after global average pooling, the features after global maximum pooling, and the global brightness features before splicing is three times, that is, a squeeze×3 operation is performed. The input end of the first fully connected layer 1636 is connected to the output end of the splicing layer 1635, and is used to compress and nonlinearly fuse the comprehensive features. Its activation function uses the ReLU function. The input end of the second fully connected layer 1637 is connected to the output end of the first fully connected layer 1636, and is used to generate channel weights based on the compressed and nonlinearly fused comprehensive features. Its activation function uses the Sigmoi function. The input end of the expansion layer 1638 is connected to the output end of the second fully connected layer 1637, and is used to expand the dimension of the generated channel weights, that is, to perform an unsqueeze×3 operation.

[0042] The input end of the fusion module 164 is respectively connected to the output end of the expansion layer 1638 in the spatial attention mechanism module 161, the temporal attention mechanism module 162 and the highlight perception channel attention mechanism module 163, and its output end is connected to the input end of the fourth three-dimensional convolutional layer 4, which is used to perform feature fusion on the output features of the spatial attention mechanism module 161, the temporal attention mechanism module 162 and the highlight perception channel attention mechanism module 163, that is, F combined=Concat(F avg ,F max ,F light )∈R B×(2C+1) (5) Among them, F combined It represents the joint feature vector obtained by fusing each feature in the channel dimension. Concat means concatenating multiple feature vectors in sequence along the channel dimension.

[0043] W c =σ(W2⋅ReLU(W1⋅F combined )) (6) Among them, W c is the channel weight output by the attention mechanism model 16, W c ∈R B×C×1×1×1 , W1 and W2 are the weight matrix parameters of the first fully connected layer 1636 and the second fully connected layer 1637, respectively, W1∈R (2C+1)×(C / r) , W2∈R (C / r)×C , σ is the Sigmoid function.

[0044] In step 3, the training set constructed in step 1 is input into the improved I3D network model constructed in step 2 for training, thereby obtaining a trained improved I3D network model. The test set constructed in step 1 is then input into the trained improved I3D network model for testing, thereby obtaining a tested improved I3D network model.

[0045] The specific working process is as follows: the training set and the test set are organized and formed into a read directory Txt file, and the parameters of the improved I3D network model are set at the same time. In this embodiment, the number of training iterations is set to 30 times, and the number of samples used in each training iteration is 4, that is, the batch size is 4; the amount of test set data is too large, so the forward calculation is performed in batches, the batch size of the test set is 10, and the step size of the model parameter update during training is 0.01.

[0046] In step 4, the video image of the target to be identified is input into the tested improved I3D network model. The improved I3D network model reads the video in the video each time according to the batch size of the training iteration. The read video is defogged by the spatiotemporal adaptive defogging module 17, and then a series of convolution and pooling operations are performed. The calculation results are input into the attention mechanism model 16, and spatial attention, temporal attention, and highlight perception channel attention are calculated respectively. After the three attention calculation results are feature fused, three-dimensional convolution, average pooling, elimination of redundant dimensions and alignment of feature shapes are continued to output classification features to complete abnormal flame recognition.

[0047] The above description is only used to illustrate the technical solution of the present invention, rather than to limit it. For ordinary professional and technical personnel in this field, the specific technical solutions recorded in the above embodiments can be modified, or some of the technical features therein can be replaced by equivalents. These modifications or replacements do not cause the essence of the corresponding technical solution to deviate from the scope of the technical solution protected by the present invention.

Claims

1. An improved I3D network model, characterized by: Including spatiotemporal adaptive dehazing module (17), I3D network model and attention mechanism model (16); The input end of the spatiotemporal adaptive defogging module (17) serves as the input end of the improved I3D network model, and is used to receive an external input video, and combine the time information of the video with the traditional atmospheric scattering model to obtain a dynamic atmospheric scattering model to perform defogging on the input video; The input end of the I3D network model is connected to the output end of the spatiotemporal adaptive defogging module (17); The attention mechanism model (16) is set in the middle of the I3D network model, including a spatial attention mechanism module (161), a temporal attention mechanism module (162), a highlight perception channel attention mechanism module (163) and a fusion module (164); the spatial attention mechanism module (161) is used to adaptively assign weights to each spatial position; the temporal attention mechanism module (162) is used to adaptively assign weights to different time frames; the highlight perception channel attention mechanism module (163) is used to fuse channel features with brightness features to dynamically adjust channel weights; The input end of the fusion module (164) is respectively connected to the output end of the spatial attention mechanism module (161), the temporal attention mechanism module (162) and the highlight perception channel attention mechanism module (163), and is used to perform feature fusion on the output features of the spatial attention mechanism module (161), the temporal attention mechanism module (162) and the highlight perception channel attention mechanism module (163); The output end of the I3D network model serves as the output end of the improved I3D network model and is used to output feature information.

2. The improved I3D network model according to claim 1, characterized in that: The I3D network model includes a first three-dimensional convolution layer (1), a second three-dimensional convolution layer (2), a third three-dimensional convolution layer (3), a fourth three-dimensional convolution layer (4), a fifth three-dimensional convolution layer (5), a first three-dimensional pooling layer (6), a second three-dimensional pooling layer (7), a third three-dimensional pooling layer (8), a fourth three-dimensional pooling layer (9), a first mixed pooling layer (10), a second mixed pooling layer (11), a third mixed pooling layer (12), an average pooling layer (13), a batch normalization layer (14) and an output layer (15); The first three-dimensional convolution layer (1), the first three-dimensional pooling layer (6), the second three-dimensional convolution layer (2), the third three-dimensional convolution layer (3), the second three-dimensional pooling layer (7), the first mixed pooling layer (10), the third three-dimensional pooling layer (8), the second mixed pooling layer (11), the fourth three-dimensional pooling layer (9), and the third mixed pooling layer (12) are connected in sequence according to the order of input and output; the input end of the first three-dimensional convolution layer (1) is connected to the output end of the spatiotemporal adaptive defogging module (17), and the output end of the third mixed pooling layer (12) is connected to the input ends of the spatial attention mechanism module (161), the temporal attention mechanism module (162), and the highlight perception channel attention mechanism module (163), respectively; The fourth three-dimensional convolution layer (4), the batch normalization layer (14), the average pooling layer (13), the fifth three-dimensional convolution layer (5) and the output layer (15) are connected in sequence according to the order of input and output; the input end of the fourth three-dimensional convolution layer (4) is connected to the output end of the fusion module (164); and the output layer (15) serves as the output end of the I3D network model.

3. The improved I3D network model according to claim 2, characterized in that: The spatiotemporal adaptive defogging module (17) includes an input layer (171), a spatiotemporal pooling layer (172), a long short-term memory neural network layer (173), and a model generation and output layer (174) connected in sequence according to input and output order; The input layer (171) serves as an input end of the spatiotemporal adaptive defogging module (17) and is used to receive external input video; The spatiotemporal pooling layer (172) is used to perform an average pooling operation on the spatial dimension of the input video to retain channel and time information; The long short-term memory neural network layer (173) is used to capture temporal dynamic features; The model generation and output layer (174) is used to predict the atmospheric scattering parameters at each time step, and generate a dynamic atmospheric scattering model based on the atmospheric scattering parameters at each time step, and then output the dehazed video, the output end of which is connected to the input end of the first three-dimensional convolution layer (1).

4. The improved I3D network model according to claim 3, characterized in that: The expression of the dynamic atmospheric scattering model is as follows: ； in, This is the video after dehazing. is the foggy image sequence of the input video, is the global atmospheric light value, is the scattering coefficient, For depth estimation, is the two-dimensional space coordinate, is the time step index, ε is the numerical guard, ε=1×10 −6 .

5. The improved I3D network model according to claim 4, characterized in that: The highlight perception channel attention mechanism module (163) includes an attention input layer (1630), a global average pooling layer (1631), a global maximum pooling layer (1632), a brightness feature convolution layer (1633), an averaging layer (1634), a splicing layer (1635), a first fully connected layer (1636), a second fully connected layer (1637), and an expansion layer (1638); The input end of the attention input layer (1630) is connected to the output end of the third mixed pooling layer (12) for receiving the feature data output by the third mixed pooling layer (12); the input ends of the global average pooling layer (1631), the global maximum pooling layer (1632), and the brightness feature convolution layer (1633) are respectively connected to the output end of the attention input layer (1630) for performing global average pooling, global maximum pooling, and brightness feature convolution on the feature data output by the third mixed pooling layer (12); the input end of the average layer (1634) is connected to the output end of the brightness feature convolution layer (1633) for averaging the features after brightness feature convolution along the time, height, and width dimensions to generate a global brightness feature; The input end of the splicing layer (1635) is respectively connected to the output end of the global average pooling layer (1631), the global maximum pooling layer (1632) and the average layer (1634), and is used to splice the features after global average pooling, the features after global maximum pooling and the global brightness features to form a comprehensive feature; The input end of the first fully connected layer (1636) is connected to the output end of the splicing layer (1635) for compressing and nonlinearly fusing the comprehensive features; the input end of the second fully connected layer (1637) is connected to the output end of the first fully connected layer (1636) for generating channel weights according to the comprehensive features after compression and nonlinear fusion; The input end of the expansion layer (1638) is connected to the output end of the second fully connected layer (1637) for dimensional expansion of the generated channel weights, and the output end is connected to the input end of the fusion module (164).

6. The improved I3D network model according to claim 5, characterized in that: The activation function of the first fully connected layer (1636) adopts the ReLU function; The activation function of the second fully connected layer (1637) adopts the Sigmoid function.

7. The improved I3D network model according to claim 2, characterized in that: The first three-dimensional pooling layer (6), the second three-dimensional pooling layer (7), the third three-dimensional pooling layer (8), and the fourth three-dimensional pooling layer (9) are all three-dimensional maximum pooling layers; The activation function of the batch normalization layer (14) adopts the ReLU function; The loss function of the I3D network model adopts the cross entropy loss function.

8. The improved I3D network model according to claim 7, characterized in that: The convolution kernel size of the first three-dimensional convolution layer (1) is 7×7×7, the convolution kernel size of the second three-dimensional convolution layer (2) is 1×1×1, the convolution kernel sizes of the third three-dimensional convolution layer (3) and the fourth three-dimensional convolution layer (4) are both 3×3×3, and the convolution kernel size of the fifth three-dimensional convolution layer (5) is 1×1×1.

9. An abnormal flame recognition method based on an improved I3D network model, characterized in that: The following steps are involved: Step 1: Collect flame videos of different categories and construct training sets and test sets based on the collected flame videos; Step 2: Building the improved P3D network model according to any one of claims 1 to 8; Step 3: Input the training set constructed in step 1 into the improved I3D network model constructed in step 2 for training to obtain a trained improved I3D network model; test the trained improved I3D network model using the test set established in step 1 to obtain a tested improved I3D network model; Step 4: Input the video image of the target to be identified into the tested improved I3D network model, and realize abnormal flame recognition based on the output result of the tested improved I3D network model.

10. The abnormal flame identification method based on the improved PD network model according to claim 9 is characterized in that: In step 1, the flame videos in the training set cover four types of flame situations.

Images