Improved 3D network model construction method and abnormal flame identification method based on the model

Abstract

The application provides an improved I3D network model construction method and an abnormal flame identification method based on the model, and is used for solving the technical problem that the existing flame detection method cannot analyze and process more complex video data in real time, and the accuracy and reliability are difficult to support, because the analysis data is mainly in the form of pictures. The improved I3D network model construction method adds a space-time adaptive defogging module in front of the traditional I3D network model to defog the input video, so as to reduce the interference of fog and haze. Meanwhile, an attention mechanism model integrating a spatial attention mechanism, a temporal attention mechanism and a highlight perception channel attention mechanism is constructed near the rear end of the traditional I3D network model, so as to better capture the features in the video and enhance the sensitivity of the traditional I3D network model to the change of the flame.

Description

Technical Field

[0001] This invention relates to anomaly flame identification methods, and more particularly to an improved I3D network model construction method and anomaly flame identification method based on the model. Background Technology

[0002] In flame detection, traditional image processing algorithms typically rely on color and motion features, which struggle to handle diverse backgrounds in complex scenes. To improve the accuracy and real-time performance of flame detection, the industry has begun exploring the use of advanced technologies and deep learning models.

[0003] Li et al. used spatiotemporal feature fusion to address the problem of identifying dynamic changes in flames against complex backgrounds. See Li Z, Gavrilyuk K, Gavves E, et al., “VideoLSTM convolves, attends and flows for action recognition,” Computer Vision and Image Understanding, vol. 166, pp. 4150, 2018. Kim et al. (2019) used a Long Short-Term Memory (LSTM) network to summarize features within bounding boxes in short time series of consecutive frames to determine the presence and classification of fires. Then, most of the consecutive short-term decisions were merged together, and the final result was determined by long-term voting. See Kim B, Lee J, “A video-based fire detection using deep learning models,” Applied Sciences, vol. 9, no. 14, pp. 2862-2881, 2019. Dimitropoulos et al. proposed a method combining spatio-temporal flame modeling and dynamic texture analysis; see Dimitropoulos K, Barmpoutis P, Grammalidis N, “Spatio-temporal flame modeling and dynamic texture analysis for automatic video-based fire detection,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 2, pp. 339-351, 2015. DOI: 10.1109 / TCSVT.2014.2339592. Yuming et al. focused more on simplifying the fire and smoke detection process; they proposed a DETR-based target detection model that combines a convolutional neural network (CNN) and a transformer.To improve the detection performance of small targets and accelerate the model training process, a normalization-based attention module was added in the feature extraction stage, and multi-scale deformable attention was used in the encoder-decoder structure (see Yuming L, Wei Z, Yanyan L, et al., “An efficient fire and smoke detection algorithm based on an end-to-end structured network,” Engineering Applications of Artificial Intelligence, vol. 116, 2022). Hikmat et al. proposed a modified YOLOv5 model that integrates a Stem module, replaces a larger kernel in the SPP (neck) with a smaller kernel, and adds a P6 module to the head (see Hikmat Y, Ahmad ZK, Min UFU, et al., “A modified YOLOv5 architecture for efficient fire detection in smart cities,” Expert Systems With Applications, vol. 231, 2023). This design aims to reduce the model's complexity and size while still being able to detect both small and large fire areas in images. In addition, they provided a medium-sized fire dataset, including three categories: vehicle fires, building fires, and indoor electrical fires. Each of the methods mentioned above has its own advantages and disadvantages, but the data they analyze is primarily image-based. In reality, more complex video data needs to be analyzed and processed in real time, which increases the complexity and difficulty of processing, making it difficult to support the accuracy and reliability of these methods. Summary of the Invention

[0004] The purpose of this invention is to solve the technical problem that existing flame detection methods mainly analyze images, which cannot analyze and process more complex video data in real time, making it difficult to support their accuracy and reliability. The invention provides an improved I3D network model construction method and an abnormal flame recognition method based on the model.

[0005] To achieve the above objectives, the technical solution provided by the present invention is as follows:

[0006] An improved method for constructing an I3D network model is characterized by constructing an improved I3D network model, which includes a spatiotemporal adaptive dehazing module, an I3D network model, and an attention mechanism model.

[0007] The spatiotemporal adaptive dehazing module serves as the input to the improved I3D network model. It receives externally input video and combines the video's temporal information with a traditional atmospheric scattering model to obtain a dynamic atmospheric scattering model for dehazing the input video.

[0008] The input end of the I3D network model is connected to the output end of the spatiotemporal adaptive defogging module;

[0009] The attention mechanism model is set in the middle of the I3D network model, including a spatial attention mechanism module, a temporal attention mechanism module, a highlight perception channel attention mechanism module, and a fusion module. The spatial attention mechanism module is used to adaptively assign weights to each spatial location. The temporal attention mechanism module is used to adaptively assign weights to different time frames. The highlight perception channel attention mechanism module is used to fuse channel features with brightness features to dynamically adjust channel weights. The input of the fusion module is connected to the output of the spatial attention mechanism module, the temporal attention mechanism module, and the highlight perception channel attention mechanism module, respectively, and is used to perform feature fusion on the output features of the spatial attention mechanism module, the temporal attention mechanism module, and the highlight perception channel attention mechanism module.

[0010] The output of the I3D network model serves as the output of the improved I3D network model, used to output feature information.

[0011] Furthermore, the I3D network model includes a first three-dimensional convolutional layer, a second three-dimensional convolutional layer, a third three-dimensional convolutional layer, a fourth three-dimensional convolutional layer, a fifth three-dimensional convolutional layer, a first three-dimensional pooling layer, a second three-dimensional pooling layer, a third three-dimensional pooling layer, a fourth three-dimensional pooling layer, a first hybrid pooling layer, a second hybrid pooling layer, a third hybrid pooling layer, an average pooling layer, a batch normalization layer, and an output layer;

[0012] The first 3D convolutional layer, the first 3D pooling layer, the second 3D convolutional layer, the third 3D convolutional layer, the second 3D pooling layer, the first hybrid pooling layer, the third 3D pooling layer, the second hybrid pooling layer, the fourth 3D pooling layer, and the third hybrid pooling layer are connected sequentially in the order of input and output; the input end of the first 3D convolutional layer is connected to the output end of the spatiotemporal adaptive dehazing module, and the output end of the third hybrid pooling layer is connected to the input ends of the spatial attention mechanism module, the temporal attention mechanism module, and the highlight perception channel attention mechanism module, respectively;

[0013] The fourth 3D convolutional layer, batch normalization layer, average pooling layer, fifth 3D convolutional layer, and output layer are connected sequentially in the order of input and output; the input end of the fourth 3D convolutional layer is connected to the output end of the fusion module; the output layer serves as the output end of the I3D network model.

[0014] Furthermore, the spatiotemporal adaptive dehazing module includes an input layer, a spatiotemporal pooling layer, a long short-term memory neural network layer, and a model generation and output layer connected sequentially in the input-output order;

[0015] The input layer serves as the input terminal of the spatiotemporal adaptive dehazing module, used to receive externally input video.

[0016] The spatiotemporal pooling layer is used to perform average pooling on the spatial dimension of the input video, preserving channel and temporal information;

[0017] The long short-term memory neural network layer is used to capture temporal dynamic features;

[0018] The model generation and output layer is used to predict atmospheric scattering parameters at each time step, and generate a dynamic atmospheric scattering model based on the atmospheric scattering parameters at each time step, thereby outputting the dehazed video. Its output end is connected to the input end of the first three-dimensional convolutional layer.

[0019] Furthermore, the expression for the dynamic atmospheric scattering model is as follows:

[0020]

[0021] Where J(x,t) is the dehazed video, X(x,t) is the sequence of hazy images of the input video, A(t) is the global atmospheric light value, β(t) is the scattering coefficient, d(x,t) is the depth estimate, x is the two-dimensional spatial coordinate, t is the time step index, and ε is the numerical guard, ε=1×10 -6 .

[0022] Furthermore, the high-brightness perception channel attention mechanism module includes an attention input layer, a global average pooling layer, a global max pooling layer, a brightness feature convolutional layer, an equalization layer, a splicing layer, a first fully connected layer, a second fully connected layer, and an extension layer.

[0023] The input of the attention input layer is connected to the output of the third hybrid pooling layer to receive the feature data output by the third hybrid pooling layer. The inputs of the global average pooling layer, the global max pooling layer, and the brightness feature convolution layer are respectively connected to the output of the attention input layer to perform global average pooling, global max pooling, and brightness feature convolution on the feature data output by the third hybrid pooling layer. The input of the equalization layer is connected to the output of the brightness feature convolution layer to calculate the average value of the features after brightness feature convolution along the time, height, and width dimensions to generate global brightness features.

[0024] The input of the splicing layer is connected to the output of the global average pooling layer, the global max pooling layer, and the equalization layer, respectively, and is used to splice the features after global average pooling, the features after global max pooling, and the global brightness features to form a comprehensive feature.

[0025] The input of the first fully connected layer is connected to the output of the splicing layer, and is used to compress and nonlinearly fuse the comprehensive features. The input of the second fully connected layer is connected to the output of the first fully connected layer, and is used to generate channel weights based on the compressed and nonlinearly fused comprehensive features. The input of the extension layer is connected to the output of the second fully connected layer, and is used to expand the dimension of the generated channel weights. Its output is connected to the input of the fusion module.

[0026] Furthermore, the activation function of the first fully connected layer is the ReLU function;

[0027] The activation function of the second fully connected layer is the Sigmoid function.

[0028] Furthermore, the first three-dimensional pooling layer, the second three-dimensional pooling layer, the third three-dimensional pooling layer, and the fourth three-dimensional pooling layer are all three-dimensional max pooling layers;

[0029] The activation function of the batch normalization layer is the ReLU function;

[0030] The loss function of the I3D network model is the cross-entropy loss function.

[0031] Furthermore, the kernel size of the first three-dimensional convolutional layer is 7×7×7, the kernel size of the second three-dimensional convolutional layer is 1×1×1, the kernel size of the third and fourth three-dimensional convolutional layers is 3×3×3, and the kernel size of the fifth three-dimensional convolutional layer is 1×1×1.

[0032] In addition, this invention also provides an abnormal flame identification method based on an improved I3D network model, which is characterized by including the following steps:

[0033] Step 1: Collect different types of flame videos, and construct training and testing sets based on the collected flame videos;

[0034] Step 2: Construct the improved I3D network model using the improved I3D network model construction method described above;

[0035] Step 3: Input the training set constructed in Step 1 into the improved I3D network model constructed in Step 2 for training to obtain the trained improved I3D network model; test the trained improved I3D network model using the test set established in Step 1 to obtain the tested improved I3D network model.

[0036] Step 4: Input the video image of the target to be identified into the tested improved I3D network model, and realize the abnormal flame identification based on the output results of the tested improved I3D network model.

[0037] Furthermore, in step 1, the flame videos in the training set cover four types of flame scenarios.

[0038] The advantages of this invention compared to the prior art are as follows:

[0039] 1. The present invention provides an improved I3D network model construction method, which adds a spatiotemporal adaptive dehazing module to the front end of the traditional I3D network model to dehaze the input video to reduce haze interference; at the same time, an attention mechanism model integrating spatial attention mechanism, temporal attention mechanism and highlight perception channel attention mechanism is built near the back end of the traditional I3D network model to better capture the features in the video and enhance the sensitivity of the traditional I3D network model to flame changes.

[0040] 2. The spatiotemporal adaptive defogging module in this invention combines traditional atmospheric scattering models with temporal information, which greatly enhances the effectiveness of subsequent attention mechanism models.

[0041] 3. The high-brightness perception channel attention mechanism module in this invention is a high-brightness perception-based attention mechanism designed for flame characteristics. The brightness features extracted by the module enhance the high-brightness areas that the channel focuses on, which helps improve the accuracy of network recognition.

[0042] 4. Compared with conventional abnormal flame recognition methods based on image information, the abnormal flame recognition method based on the improved I3D network model provided by this invention innovatively uses video information directly for recognition from a computer vision perspective. At the same time, based on the above-mentioned improved I3D network model, it can meet the needs of abnormal fire recognition in complex backgrounds, achieving a recognition accuracy of 97.78%, and can be used for real-time monitoring in forests and urban areas.

[0043] 5. When constructing the training set of the improved I3D network model, this invention collected 8410 videos of flames under different combustion states based on four types of flame scenarios, which can accurately train the improved I3D network model and improve the accuracy of fire early warning. Attached Figure Description

[0044] Figure 1 This is a schematic block diagram of the improved I3D network model in step 2 of an embodiment of the abnormal flame recognition method based on the improved I3D network model of the present invention.

[0045] Figure 2A schematic block diagram of the spatiotemporal adaptive defogging module in step 2 of this embodiment of the invention;

[0046] Figure 3 This is a schematic block diagram of the highlight perception channel attention mechanism module in step 2 of the present invention.

[0047] The specific reference numerals in the attached figures are as follows:

[0048] 1. First 3D convolutional layer; 2. Second 3D convolutional layer; 3. Third 3D convolutional layer; 4. Fourth 3D convolutional layer; 5. Fifth 3D convolutional layer; 6. First 3D pooling layer; 7. Second 3D pooling layer; 8. Third 3D pooling layer; 9. Fourth 3D pooling layer; 10. First hybrid pooling layer; 11. Second hybrid pooling layer; 12. Third hybrid pooling layer; 13. Average pooling layer; 14. Batch normalization layer; 15. Output layer; 16. Attention mechanism model; 161. Spatial attention mechanism module; 162. Temporal attention mechanism module; 163. Highlight perception channel attention mechanism module; 1630. Attention input layer; 1631. Global average pooling layer; 1632. Global max pooling layer; 1633. Brightness feature convolutional layer; 1634. Equalization layer; 1635. Concatenation layer; 1636. First fully connected layer; 1637. Second fully connected layer; 1638. Extension layer; 164. Fusion module; 17. Spatiotemporal adaptive dehazing module; 171. Input layer; 172. Spatiotemporal pooling layer; 173. Long short-term memory neural network layer; 174. Model generation and output layer. Detailed Implementation

[0049] To make the advantages and features of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] An abnormal flame identification method based on an improved I3D network model specifically includes the following steps:

[0051] Step 1: Construct the training set and the test set.

[0052] In this embodiment, 8410 video clips of flames under different combustion conditions were collected. These videos covered four types of flame scenarios: abnormal fire / smoke in the wild, abnormal fire / smoke in residential areas, normal fire / smoke, and no fire / smoke. Training and testing sets for the improved I3D network model were constructed based on these 8410 video clips, allowing for precise training of the improved I3D network model and improving the accuracy of fire early warning.

[0053] Step 2: Construct an improved I3D network model.

[0054] The I3D network model excels in video analysis tasks due to its ability to effectively capture temporal and spatial features from videos. Its compact structure and powerful functionality maintain high efficiency and accuracy even when processing large video datasets. The design of the I3D network model enhances its generalization and non-linear representation capabilities. Its 3D convolutional layers, based on 2D convolutional networks (such as Inception-v1), expand the 2D convolutional kernels into 3D kernels through "inflation" to process consecutive frames of video data. It contains multiple 3D Inception modules, each with multiple 3D convolutional kernels of different sizes to capture multi-scale spatiotemporal features. Inception is a classic architecture in convolutional neural networks (CNNs), and the 3D Inception modules are an extension of the Inception architecture into 3D convolutional neural networks (3D CNNs). It is worth noting that the I3D network model has a relatively high number of parameters and computational complexity; therefore, model compression or optimization may be necessary when deploying it in resource-constrained environments. However, given sufficient computing resources, the I3D network model can provide powerful video analysis capabilities and exhibits strong robustness in flame recognition in complex and noisy scenes. Given that this invention focuses on video analysis, particularly the identification of dynamic flame behavior in fire detection, and considering that the video database contains thousands of video clips, this invention employs a CNN-based I3D network model as the foundational model.

[0055] To further improve the accuracy of flame recognition, this invention improves the I3D network model, referred to as the improved I3D network model. First, to reduce haze interference, a spatiotemporal adaptive dehazing module (ST-Dehazed) 17 is added to the front end of the I3D network model to dehaze the input video. Simultaneously, this invention also constructs an attention mechanism model 16 near the back end of the I3D network model, integrating spatial attention, temporal attention, and highlight perception channel attention mechanisms, to better capture features in the video and enhance the I3D network model's sensitivity to flame changes. The spatiotemporal adaptive dehazing module 17 combines a traditional atmospheric scattering model with temporal information, thereby enhancing the effectiveness of the subsequent attention mechanism model 16.

[0056] like Figure 1As shown, the improved I3D network model includes a spatiotemporal adaptive dehazing module 17, an I3D network model, and an attention mechanism model 16. The I3D network model comprises a first 3D convolutional layer 1, a second 3D convolutional layer 2, a third 3D convolutional layer 3, a fourth 3D convolutional layer 4, a fifth 3D convolutional layer 5, a first 3D pooling layer 6, a second 3D pooling layer 7, a third 3D pooling layer 8, a fourth 3D pooling layer 9, a first hybrid pooling layer 10, a second hybrid pooling layer 11, a third hybrid pooling layer 12, an average pooling layer 13, a batch normalization layer 14, and an output layer 15. The first 3D convolutional layer 1, the first 3D pooling layer 6, the second 3D convolutional layer 2, the third 3D convolutional layer 3, the second 3D pooling layer 7, the first hybrid pooling layer 10, the third 3D pooling layer 8, the second hybrid pooling layer 11, the fourth 3D pooling layer 9, and the third hybrid pooling layer 12 are connected sequentially in the order of input and output. The input of the first 3D convolutional layer 1 is connected to the output of the spatiotemporal adaptive dehazing module 17, and the output of the third hybrid pooling layer 12 is connected to the input of the attention mechanism model 16. The fourth 3D convolutional layer 4, the batch normalization layer 14, the average pooling layer 13, the fifth 3D convolutional layer 5, and the output layer 15 are connected in the order of input and output. The input of the fourth 3D convolutional layer 4 is connected to the output of the attention mechanism model 16. The output layer 15 serves as the output of the I3D network model and is used to output feature information.

[0057] The first three-dimensional convolutional layer 1, the second three-dimensional convolutional layer 2, the third three-dimensional convolutional layer 3, the fourth three-dimensional convolutional layer 4, and the fifth three-dimensional convolutional layer 5 are used to perform three-dimensional convolution on the input video. In this embodiment, the kernel size of the first three-dimensional convolutional layer 1 is 7×7×7, the kernel size of the second three-dimensional convolutional layer 2 is 1×1×1, the kernel size of the third three-dimensional convolutional layer 3 and the fourth three-dimensional convolutional layer 4 are both 3×3×3, and the kernel size of the fifth three-dimensional convolutional layer 5 is 1×1×1. The first three-dimensional pooling layer 6, the second three-dimensional pooling layer 7, the third three-dimensional pooling layer 8, and the fourth three-dimensional pooling layer 9 are all three-dimensional max pooling layers, used to cooperate with global max pooling to reduce the dimensionality of the feature map; the first hybrid pooling layer 10, the second hybrid pooling layer 11, the third hybrid pooling layer 12, and the average pooling layer 13 are used to cooperate with global average pooling; in this embodiment, the window size of the first three-dimensional pooling layer 6, the second three-dimensional pooling layer 7, the third three-dimensional pooling layer 8, the fourth three-dimensional pooling layer 9, the first hybrid pooling layer 10, the second hybrid pooling layer 11, the third hybrid pooling layer 12, and the average pooling layer 13 is 3×3×3. The batch normalization layer 14 is used to accelerate the training process of the improved I3D network model and improve its stability; the activation function of the batch normalization layer 14 is the ReLU function. Furthermore, the choice of loss function is also crucial. Based on experimental results, this invention selects the cross-entropy loss function to provide better video recognition robustness for the improved I3D network model.

[0058] The spatiotemporal adaptive dehazing module 17 receives externally input video and combines the video's temporal information with a traditional atmospheric scattering model through deep learning to obtain a dynamic atmospheric scattering model for dehazing the input video. Figure 2 As shown, the spatiotemporal adaptive dehazing module 17 includes an input layer 171, a spatiotemporal pooling layer 172, a long short-term memory neural network (LSTM) layer 173, and a model generation and output layer 174, which are connected in the order of input and output.

[0059] Input layer 171 serves as the input terminal of the spatiotemporal adaptive defogging module 17, which is also the input terminal of the improved I3D network model, and is used to receive externally input video. When the improved I3D network model is being trained, input layer 171 receives the video input from the training set constructed in step 1. When the improved I3D network model is used to identify abnormal flames, input layer 171 receives the video of the target to be identified.

[0060] The spatiotemporal pooling layer 172 is used to perform average pooling on the spatial dimension of the input video, preserving channel and temporal information, as shown in the formula.

[0061]

[0062] Among them, X pool The feature sequence is obtained after spatial dimension average pooling, with size R. B×C×T H is the spatial height of the input feature map, W is the spatial width of the input feature map, and X is the spatial height of the input feature map. :C:Thw Let R be the feature value of the original input video at the C-th channel, T-th frame, h-th row, and w-th column, where R is the real number field, B is the batch size (i.e., the number of samples), C is the number of channels, and T is the time dimension.

[0063] Long Short-Term Memory (LSTM) neural network layer 173 is used to capture temporal dynamic features. First, X... pool After transposing the data, the transposed data is input into layer 173 of the Long Short-Term Memory neural network, resulting in:

[0064]

[0065] Among them, H lstm For the Long Short-Term Memory neural network layer 173, feature sequences X are used. pool The output feature sequence at each time step has a size of R. B×T×64 h T Let c be the hidden state vector corresponding to the T-th time step (the last frame). T For the memory unit (cell state) vector corresponding to the Tth time step, the hidden layer dimension of the Long Short-Term Memory Neural Network layer 173 is 64.

[0066] The model generation and output layer 174 is used to predict atmospheric scattering parameters at each time step and generate a dynamic atmospheric scattering model based on these parameters, ultimately outputting the dehazed video. The atmospheric scattering parameters include the scattering coefficient β(t), depth estimate d(x,t), and global atmospheric light value A(t); the scattering coefficient β(t) is defined as β(t) = σ(W β H lstm )∈[0,1] T Where σ is the Sigmoid function, W β is the weight matrix for the scattering coefficients. The depth estimate d(x,t) and the global atmospheric light value A(t) simulate the normalized atmospheric light intensity, where A(t)=σ(W A H lstm )∈[0,1] T W d W is the weight matrix for depth estimation. A Let be the weight matrix of the global atmospheric light value. Then, the transmittance t(x,t) = exp(-β(t)·d(x,t)). The dehazed video J(x,t) output after model generation and output layer 174 is:

[0067]

[0068] Where X(x,t) is the sequence of foggy images from the input video, x is the two-dimensional spatial coordinate, t is the time step index, and ε is a numeric guard used to prevent division by zero errors and ensure computational stability; typically ε = 1 × 10⁻⁶. -6 .

[0069] The output video tensor was dehazed and restored for each sample, each channel, each frame, and each pixel.

[0070] The spatiotemporal adaptive dehazing module 17 performs dehazing processing on the input video. The dehazed video features reduce the haze interference in the subsequent attention mechanism model 16, enhance key features, and make the spatiotemporal key regions more prominent, providing high signal-to-noise ratio input for subsequent processes. This allows the attention mechanism model 16 to capture spatiotemporal key regions more intuitively, thereby improving the accuracy of video target recognition.

[0071] The attention mechanism model 16 in this invention includes a spatial attention mechanism module 161, a temporal attention mechanism module 162, a brightness perception channel attention mechanism module 163, and a fusion module 164. The spatial attention mechanism module 161 adaptively assigns weights to each spatial location; the temporal attention mechanism module 162 adaptively assigns weights to different time frames; and the brightness perception channel attention mechanism module (CHAM) 163 fuses channel features with brightness features to dynamically adjust channel weights. The spatial attention mechanism module 161 and the temporal attention mechanism module 162 both employ existing attention mechanisms, while the brightness perception channel attention mechanism module 163 is a brightness perception-based attention mechanism designed for flame characteristics in this invention. The inputs of the spatial attention mechanism module 161, the temporal attention mechanism module 162, and the brightness perception channel attention mechanism module 163 are all connected to the output of the third hybrid pooling layer 12, used to respectively implement feature selection in the spatial, temporal, and brightness aspects of the video after dehazing. The video input to the attention mechanism model 16 is defined as X, where X∈R. B×C×T×H×W The expressions for feature extraction by the spatial attention mechanism module 161, the temporal attention mechanism module 162, and the highlight perception channel attention mechanism module 163 are as follows:

[0072] F avg =GAP(X), F max =GMP(X), F light =C 1×1×1 (X) (4)

[0073] Among them, F avg The features output by the spatial attention mechanism module 161 are represented by GAP(X), which represents global average pooling of video X in the spatial dimension. max GMP(X) represents the features output by the temporal attention mechanism module 162, where GMP(X) represents global max pooling of video X in the temporal dimension, and F... light C represents the feature output by the highlight perception channel attention mechanism module 163. 1×1×1 (X) represents the global average pooling of the brightness features of video X in the spatial dimension, i.e., a 1×1×1 convolution operation.

[0074] The brightness features extracted by the brightness perception channel attention mechanism module 163 enhance the brightness of the channel's attention areas, thus improving the accuracy of network recognition. The following focuses on a detailed description of the brightness perception channel attention mechanism module 163 in this invention:

[0075] like Figure 3As shown, the high brightness perception channel attention mechanism module 163 includes an attention input layer 1630, a global average pooling layer 1631, a global max pooling layer 1632, a brightness feature convolutional layer 1633, an equalization layer 1634, a splicing layer 1635, a first fully connected layer 1636, a second fully connected layer 1637, and an extension layer 1638.

[0076] The attention input layer 1630 serves as the input to the high-brightness perception channel attention mechanism module 163. Its input is connected to the output of the third hybrid pooling layer 12, receiving the feature data output by the third hybrid pooling layer 12. The inputs of the global average pooling layer 1631, the global max pooling layer 1632, and the brightness feature convolution layer 1633 are connected to the output of the attention input layer 1630, respectively, performing global average pooling, global max pooling, and brightness feature convolution on the feature data output by the third hybrid pooling layer 12. The input of the equalization layer 1634 is connected to the output of the brightness feature convolution layer 1633, averaging the convolved brightness features along the time, height, and width dimensions to generate global brightness features. This means performing global average pooling on the brightness features of video X in the spatial dimension to obtain F. light ∈R B×1 The input of the stitching layer 1635 is connected to the outputs of the global average pooling layer 1631, the global max pooling layer 1632, and the equalization layer 1634, respectively. It is used to stitch together the compressed features from the global average pooling, global max pooling, and global brightness features to generate a comprehensive feature. In this embodiment, the compression factor for the global average pooling, global max pooling, and global brightness features before stitching is three times, i.e., a squeeze×3 operation is performed. The input of the first fully connected layer 1636 is connected to the output of the stitching layer 1635. It is used to compress and nonlinearly fuse the comprehensive feature, and its activation function is the ReLU function. The input of the second fully connected layer 1637 is connected to the output of the first fully connected layer 1636. It is used to generate channel weights based on the compressed and nonlinearly fused comprehensive feature, and its activation function is the Sigmoi function. The input of the extension layer 1638 is connected to the output of the second fully connected layer 1637. It is used to expand the dimension of the generated channel weights, i.e., an unsqueeze×3 operation is performed.

[0077] The input of the fusion module 164 is connected to the output of the extended layer 1638 in the spatial attention mechanism module 161, the temporal attention mechanism module 162, and the highlight perception channel attention mechanism module 163, respectively. Its output is connected to the input of the fourth 3D convolutional layer 4, and it is used to perform feature fusion on the output features of the spatial attention mechanism module 161, the temporal attention mechanism module 162, and the highlight perception channel attention mechanism module 163.

[0078] Fcombined =Concat(F avg ,F max ,F light )∈R B×(2C+1) (5)

[0079] Among them, F combined This represents the joint feature vector obtained by fusing the features along the channel dimension. Concat means concatenating multiple feature vectors sequentially along the channel dimension.

[0080] W c =σ(W2·ReLU(W1·F) combined (6)

[0081] Among them, W c W represents the channel weights output by the attention mechanism model 16. c ∈R B×C×1×1×1 W1 and W2 are the weight matrix parameters of the first fully connected layer 1636 and the second fully connected layer 1637, respectively, where W1∈R (2C+1)×(C / r) W2∈R (C / r)×C σ is the Sigmoid function.

[0082] Step 3: Input the training set constructed in Step 1 into the improved I3D network model constructed in Step 2 for training, and obtain the trained improved I3D network model. Then, input the test set constructed in Step 1 into the trained improved I3D network model for testing, and obtain the tested improved I3D network model.

[0083] The specific working process is as follows: the training set and test set are organized and formed into a directory Txt file for reading. At the same time, the parameters of the improved I3D network model are set. In this embodiment, the number of training iterations is set to 30, and the number of samples used in each training iteration is 4, that is, the batch size is 4. The test set data volume is too large, so forward computation is performed in batches. The batch size of the test set is 10, and the step size of the model parameter update during training is 0.01.

[0084] Step 4: Input the video image of the target to be identified into the tested improved I3D network model. The improved I3D network model reads the video from the video according to the batch size of the training iteration. After reading the video, it is dehazed by the spatiotemporal adaptive dehazing module 17, and then a series of convolution and pooling operations are performed. The calculation results are input into the attention mechanism model 16, where spatial attention, temporal attention, and highlight perception channel attention are calculated respectively. After feature fusion of the three attention calculation results, three-dimensional convolution, average pooling, elimination of redundant dimensions, and alignment of feature shapes are performed to output classification features and complete the abnormal flame recognition.

[0085] The above description is only used to illustrate the technical solutions of the present invention, and is not intended to limit them. For those skilled in the art, modifications can be made to the specific technical solutions described in the above embodiments, or equivalent substitutions can be made to some of the technical features. However, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions protected by the present invention.

Claims

1. An improved method for constructing I3D network models, characterized in that: An improved I3D network model was constructed, including a spatiotemporal adaptive dehazing module, an I3D network model, and an attention mechanism model; The spatiotemporal adaptive defogging module includes a long short-term memory neural network layer and a model generation and output layer; The long short-term memory neural network layer is used to capture temporal dynamic features; the model generation and output layer is used to predict the atmospheric scattering parameters at each time step, and generate a dynamic atmospheric scattering model based on the atmospheric scattering parameters at each time step, and then output the dehazed video. The expression for the dynamic atmospheric scattering model is as follows: Where J(x,t) is the dehazed video, X(x,t) is the sequence of hazy images of the input video, A(t) is the global atmospheric light value, β(t) is the scattering coefficient, d(x,t) is the depth estimate, x is the two-dimensional spatial coordinate, t is the time step index, and ε is the numerical guard, ε=1×10 -6 ; The input end of the I3D network model is connected to the output end of the spatiotemporal adaptive defogging module; The attention mechanism model is set in the middle of the I3D network model, including a spatial attention mechanism module, a temporal attention mechanism module, a highlight perception channel attention mechanism module, and a fusion module; The high-brightness perception channel attention mechanism module is used to fuse channel features and brightness features to dynamically adjust channel weights. It includes an attention input layer, a global average pooling layer, a global max pooling layer, a brightness feature convolutional layer, an equalization layer, a splicing layer, a first fully connected layer, a second fully connected layer, and an extension layer. The attention input layer receives the feature data after hybrid pooling. The inputs of the global average pooling layer, the global max pooling layer, and the brightness feature convolution layer are respectively connected to the output of the attention input layer, and are used to perform global average pooling, global max pooling, and brightness feature convolution on the feature data after hybrid pooling, respectively. The input of the equalization layer is connected to the output of the brightness feature convolution layer, and is used to calculate the average value of the features after brightness feature convolution along the time, height, and width dimensions to generate global brightness features. The input of the splicing layer is connected to the output of the global average pooling layer, the global max pooling layer, and the equalization layer, respectively, and is used to splice the features after global average pooling, the features after global max pooling, and the global brightness features to form a comprehensive feature. The input of the first fully connected layer is connected to the output of the splicing layer, and is used to compress and nonlinearly fuse the comprehensive features; the input of the second fully connected layer is connected to the output of the first fully connected layer, and is used to generate channel weights based on the compressed and nonlinearly fused comprehensive features; the input of the extension layer is connected to the output of the second fully connected layer, and is used to expand the dimension of the generated channel weights. The fusion module is used to perform feature fusion on the output features of the spatial attention mechanism module, the temporal attention mechanism module, and the highlight perception channel attention mechanism module. The output of the I3D network model serves as the output of the improved I3D network model, used to output feature information.

2. The improved I3D network model construction method according to claim 1, characterized in that: The I3D network model includes a first three-dimensional convolutional layer, a second three-dimensional convolutional layer, a third three-dimensional convolutional layer, a fourth three-dimensional convolutional layer, a fifth three-dimensional convolutional layer, a first three-dimensional pooling layer, a second three-dimensional pooling layer, a third three-dimensional pooling layer, a fourth three-dimensional pooling layer, a first hybrid pooling layer, a second hybrid pooling layer, a third hybrid pooling layer, an average pooling layer, a batch normalization layer, and an output layer. The first 3D convolutional layer, the first 3D pooling layer, the second 3D convolutional layer, the third 3D convolutional layer, the second 3D pooling layer, the first hybrid pooling layer, the third 3D pooling layer, the second hybrid pooling layer, the fourth 3D pooling layer, and the third hybrid pooling layer are connected sequentially in the order of input and output; the input end of the first 3D convolutional layer is connected to the output end of the spatiotemporal adaptive dehazing module, and the output end of the third hybrid pooling layer is connected to the input ends of the spatial attention mechanism module, the temporal attention mechanism module, and the highlight perception channel attention mechanism module, respectively; The fourth 3D convolutional layer, batch normalization layer, average pooling layer, fifth 3D convolutional layer, and output layer are connected sequentially in the order of input and output; the input of the fourth 3D convolutional layer is connected to the output of the fusion module. The output layer serves as the output of the I3D network model.

3. The improved I3D network model construction method according to claim 2, characterized in that: The spatiotemporal adaptive defogging module also includes an input layer and a spatiotemporal pooling layer; The input layer serves as the input terminal of the spatiotemporal adaptive dehazing module, used to receive externally input video; the spatiotemporal pooling layer is used to perform average pooling operation on the spatial dimension of the input video, preserving channel and temporal information.

4. The improved I3D network model construction method according to claim 3, characterized in that: The activation function of the first fully connected layer is the ReLU function; The activation function of the second fully connected layer is the Sigmoid function.

5. The improved I3D network model construction method according to claim 2, characterized in that: The first three-dimensional pooling layer, the second three-dimensional pooling layer, the third three-dimensional pooling layer, and the fourth three-dimensional pooling layer are all three-dimensional max pooling layers; The activation function of the batch normalization layer is the ReLU function; The loss function of the I3D network model is the cross-entropy loss function.

6. The improved I3D network model construction method according to claim 5, characterized in that: The kernel size of the first three-dimensional convolutional layer is 7×7×7, the kernel size of the second three-dimensional convolutional layer is 1×1×1, the kernel size of the third and fourth three-dimensional convolutional layers is 3×3×3, and the kernel size of the fifth three-dimensional convolutional layer is 1×1×1.

7. An abnormal flame recognition method based on an improved I3D network model, characterized in that, Includes the following steps: Step 1: Collect different types of flame videos, and construct training and testing sets based on the collected flame videos; Step 2: Construct an improved I3D network model using any of the improved I3D network model construction methods described in claims 1-6; Step 3: Input the training set constructed in Step 1 into the improved I3D network model constructed in Step 2 for training to obtain the trained improved I3D network model; test the trained improved I3D network model using the test set established in Step 1 to obtain the tested improved I3D network model. Step 4: Input the video image of the target to be identified into the tested improved I3D network model, and realize the abnormal flame identification based on the output results of the tested improved I3D network model.

8. The abnormal flame recognition method based on the improved I3D network model according to claim 7, characterized in that: In step 1, the flame videos in the training set cover four types of flame scenarios.

Images