Outdoor Fire Smoke Image Detection Method Based on Recursive BIFPN Network

Abstract

An outdoor fire smoke image detection method based on the Recursive BIFPN network. First, fire smoke pictures are collected, and then a fire smoke detection network is used for detection. The detection steps include: using the backbone network of YOLOV5 to extract multi-scale feature maps with three different resolutions; constructing a Recursive BIFPN attention model, and using Recursive BIFPN to perform two feature fusions and enhancements on the three feature maps with different resolutions extracted by the backbone network; replacing the original detection head of YOLOV5 with a Swin Transformer detection head for classifying smoke targets of different sizes in fire images. The Recursive BIFPN of the present invention is beneficial to improving the accuracy of fire smoke detection under complex backgrounds and distinguishing interference objects such as clouds and fog; the detection head based on Swin Transformer is beneficial to detecting different smoke targets with large differences in the smoke area in the image, especially improving the detection accuracy of small smoke caused by early fires or long-distance shooting.

Description

Technical Field

[0001] This technical solution belongs to the field of image processing, and specifically is a fire detection method using a smoke detection neural network for image processing. Background Art

[0002] Frequently occurring wildfires not only seriously damage vegetation, the ecosystem and the environment, but also may cause local species to be on the verge of extinction. Therefore, early wildfire detection technologies and models based on machine vision are widely used. Due to the wide outdoor scene and the occlusion of vegetation, smoke is more easily detected than flames. However, the shape of smoke is not fixed and the color is also different. Therefore, researching a reliable fire smoke detection system is of great significance for the safety of people's lives and property and the protection of the ecosystem.

[0003] Early traditional sensor-based methods are easily interfered by influencing factors such as the size of the space and the size of the air flow in a wide outdoor area, making it impossible to give early warnings in time. And traditional fire detection methods based on computer vision rely on features defined artificially by researchers, such as color, texture features, etc. These artificially set features are usually only suitable for specific scenes and videos, with very poor generalization ability, and the accuracy of flame detection is also low. In recent years, convolutional neural networks (CNNs) have been widely applied to the field of flame detection. Although these detection networks can currently be directly applied to fire smoke detection, when facing the outdoor environment, these methods will face the following problems: The thick smoke, especially white smoke, generated during the early stage of wildfire occurrence has a certain similarity to clouds and fog, and these methods will result in a high false alarm rate. Moreover, the detection accuracy for smoke images affected by wind and with low resolution caused by fog, etc. is also low.

[0004] In the prior art:

[0005] In the "Flame Target Detection Method Based on Digital Images and Convolutional Features" with the publication number CN110751089A, first, a dataset containing video dynamic features is created; then, the standard convolution of VGG16 in the classic Faster R-CNN is replaced with depthwise separable convolution, and the number of convolutional layers is reduced; then, 256 image patches are cropped from the original image according to the candidate boxes generated by the RPN, and LBP features are extracted for each image patch; then, the size of the feature map output by the convolutional reduced ROI pooling layer and the number of neurons in the fully connected layer are adjusted; finally, the extracted LBP features, the dynamic features in the dataset, and the flattened feature vectors after pooling are merged and sent to the fully connected layer for classification and regression. This method is similar to existing object detection methods. When applied to wildfire smoke detection, the main limitation is that it is difficult to distinguish clouds in the sky from wildfire smoke because the color and texture of clouds in the air are very similar to those of wildfire smoke. If the suspected smoke images detected by Faster CNN contain clouds in the air, it is difficult to distinguish them relying solely on LBP texture features. In addition, cameras for wildfires usually monitor a range of several kilometers, and the smoke targets in the captured images may be very small, such as the images in the first row, first column, and third column of Figure 6 in this specification. For such wildfire smoke targets, Faster CNN may produce false negatives. The experimental results in Tables 1 and 2 in the specific embodiments of this specification also prove the deficiencies of Faster R-CNN when applied to wildfire smoke detection.

[0006] In the "Fire Image Detection Method Based on Lightweight Long-Short Distance Attention Transformer Network" with the publication number CN115171047A, first, flame images are collected; then, a flame detection network is used for detection, and the steps include: 1) using the designed lightweight feature extraction backbone network to process the input flame image to be detected and output three multi-scale flame features with different resolutions; 2) constructing a feature fusion network based on BiFPN to perform feature fusion processing on the aforementioned multi-scale flame features and output the fusion features that fuse three different resolution layers; 3) the classification layer of the network performs classification prediction on the aforementioned fusion features to determine the presence of the flame and its position in the image. Since this method is designed for flame detection, and the color and texture of the flame are more distinct from the objects in the background, when applied to wildfire smoke detection, the limitation of this method is that because clouds in the sky are very similar to wildfire smoke, the feature fusion network based on BiFPN cannot enhance the smoke features well. Therefore, it is unable to distinguish clouds in the sky from wildfire smoke well. The experimental results in the 5th and 6th rows of Table 1 in the specific embodiments of this specification also illustrate the limitations of the BiFPN feature fusion network. Summary of the Invention

[0007] To solve the above problems existing in the prior art and break through the above limitations of the prior art, the present invention proposes a method for fire detection using a fire and smoke target detection YOLO network improved by Recursive BIFPN and Swin Transformer.

[0008] Based on multi-scale feature learning, Recursive BIFPN feature enhancement, and Swin Transformer classification and detection strategies, the present invention designs a network structure that combines the multi-scale feature enhancement of Recursive BIFPN and the global modeling advantages of Swin Transformer, forming a new fire and smoke detection framework.

[0009] The network proposed by the present invention uses Recursive BIFPN to enhance and fuse multi-scale features of smoke; the introduction of Swin Transformer enhances the global modeling ability of the network and improves the network's ability to distinguish smoke from interfering objects; at the same time, it can better predict different sizes, especially small-sized smoke in the early stage.

[0010] The specific steps of the present invention are as follows:

[0011] Step 1. Input the fire and smoke picture to be detected, extract features using the YOLOV5 backbone network, and output three multi-scale smoke feature maps with different resolutions extracted.

[0012] Step 2. Construct a feature fusion network based on Recursive BiFPN to perform feature fusion processing on the multi-scale smoke features obtained in Step 1, and perform a recursive operation on the fused features, fuse them again with the original features of the same resolution, and output the features after the recursive and secondary fusion operations.

[0013] Step 2.1 Send the three multi-scale fire and smoke feature maps with different resolutions in Step 1 into the feature fusion network.

[0014] Step 2.2 Construct a Recursive BIFPN feature fusion network.

[0015] Step 2.2.1 Use BIFPN to perform the first fusion on the three multi-scale fire and smoke feature maps with different resolutions in Step 1). First, perform transposed convolution upsampling and pooling downsampling from low to high and then from high to low in sequence, and combine the input and output levels to further fuse the smoke feature information of different convolutional layers.

[0016] Step 2.2.2 Perform a recursive operation on the three feature maps with different resolutions after the first fusion by BIFPN in Step 2.2.1), that is, return to the original backbone network.

[0017] Step 2.2.3 Use BIFPN to fuse the three features with different resolutions returned by Step 2.2.2 with the backbone network features of the corresponding resolutions for the second time, obtaining multi-scale feature maps with three different resolutions again, and assigning corresponding weights to the feature maps with different resolutions according to different importance levels;

[0018] Step 3. Introduce the Swin Transformer Encoder module into YOLOV5 to replace the original detection head of YOLOV5, construct a classification detection module with global modeling ability, and perform classification prediction on the fused features obtained in Step 2 based on the detection head of Swin Transformer to judge the existence of fire smoke and its position in the image;

[0019] Step 3.1 Introduce the Swin Transformer Encoder module into YOLOV5 to replace the original detection head of YOLOV5;

[0020] Step 3.2 Calculate self-attention in each local window through W-MSA (Window Multi-Head Self Attention) to extract local features of the smoke image;

[0021] Step 3.3 Enable interaction between two adjacent windows through SW-MSA (Shifted Window Multi-Head Self Attention) to achieve the ability of global modeling, and then extract global information of the fire smoke image;

[0022] Step 3.4 Analyze the features in different windows, and finally classify and determine the position where the fire smoke exists. The Swin Transformer Encoder module adopts a hierarchical structure, enabling different receptive fields of the network to extract local features and global information of the fire smoke image on the premise of being unaffected by the shooting distance.

[0023] The beneficial effects of the present invention are as follows:

[0024] (1) The present invention proposes a neural network improved based on Recursive BIFPN and Swin Transformer. This network uses YOLOV5 as the framework and applies Recursive BIFPN and Swin Transformer to improve the feature fusion and global modeling capabilities of the network respectively. When applied to fire smoke detection, this network can detect fire smoke of different colors, such as white smoke, black smoke, brown smoke, etc., and is beneficial to detecting smoke with different areas, especially small smoke, thereby improving the accuracy of smoke detection.

[0025] (2) During the feature fusion process of the neural network of the present invention, a Recursive BIFPN feature fusion network is constructed to fuse the feature map processed by BIFPN with the feature map generated by the backbone network and then undergo another iterative process. This design enables the model to pay more attention to the less noticeable smoke and the important features of interfering clouds and fog. At the same time, through the Recursive operation, multi-scale features are more effectively enhanced and fused, which is beneficial to improving the accuracy of fire smoke in complex outdoor backgrounds, distinguishing interfering objects such as clouds and fog, and reducing the false alarm rate of smoke recognition.

[0026] (3) In the present invention, a Swin Transformer Encoder module is introduced into the prediction network. The hierarchical structure results in different receptive fields of the network. The non-overlapping local window and overlapping cross-window operations enhance the local feature and global modeling capabilities, which is beneficial to detecting different smoke targets with large differences in the smoke area in the image, especially small smoke; since self-attention is calculated in the moving window, while improving the detection accuracy of small smoke, the computational complexity is greatly reduced. Description of the Drawings

[0027] Figure 1 It is a flowchart of the smoke detection method of the present invention.

[0028] Figure 2 It is a structural diagram of the smoke detection network of the present invention.

[0029] Figure 3 [[ID=I8]]It is a schematic diagram of the Shifted window for the Swin Transformer attention calculation.

[0030] Figure 4 It is the structure of the Swin Transformer used in the present invention.

[0031] Figure 5(a) - Figure 5(e) It is a graph of the experimental results of the anti-interference capabilities of Recursive BiFPN and BiFPN.

[0032] Among them, Fig. 5(a) is the original image, Fig. 5(b) is the detection result of BiFPN, Fig. 5(c) is the feature map obtained by BiFPN, Fig. 5(d) is the detection result of Recursive BiFPN, and Fig. 5(e) is the feature map obtained by Recursive BiFPN. Since clouds and fog are not recognized as smoke, their feature maps and results are not shown.

[0033] Figure 6 It is a picture of the detection result of the fire smoke detection method of the present invention for smoke pictures that are difficult to identify.

[0034] The first column shows the results of less smoke due to the shooting distance, the second column shows the detection results of complex smoke such as black and gray under complex backgrounds, and the third column shows the detection results in the case of clouds similar to fire smoke in the background. Detailed implementation

[0035] Aiming at the problem that the existing object detection network is directly used in the machine vision-based fire smoke detection method, resulting in low detection accuracy and difficulty in distinguishing similar objects, the present invention proposes a fire smoke image detection network based on Recursive BIFPN and Swin Transformer. The present invention uses the constructed Recursive BIFPN feature fusion module to process the multi-scale features extracted by the backbone, more effectively enhancing and fusing the multi-scale features to better distinguish the subtle differences between clouds and smoke; the Swin Transformer mechanism is introduced into the prediction network, and its non-overlapping window and overlapping window operations enhance the local feature and global modeling capabilities, which is beneficial to detecting different smoke targets with large differences in the smoke area in the image, especially small smoke.

[0036] Compared with the existing deep learning fire smoke detection, the present invention designs a feature fusion module of Recursive BIFPN, enhancing the ability of multi-scale feature fusion and improving the ability to distinguish objects similar to smoke such as clouds; the present invention integrates the Swin Transformer Encoder module into the prediction network, improving the ability to detect early small fire smoke. Therefore, the present invention not only helps to improve the accuracy of fire smoke in outdoor complex backgrounds, but also can better distinguish interference objects such as clouds and fog, reducing the false alarm rate of smoke recognition.

[0037] The following further illustrates this case in conjunction with the drawings and specific implementation.

[0038] The present invention uses a convolutional neural network constructed based on Recursive BIFPN and Swin Transformer to detect the frame images of the monitoring video at the fire scene.

[0039] As Figure 1 shown, in the smoke detection network, multi-scale smoke features are extracted, the multi-scale features are enhanced and fused, and the smoke position is predicted. The structural diagram of the overall network of the present invention is as Figure 2 shown. Specifically, the implementation steps are as follows:

[0040] Step 1. Input the fire smoke picture to be detected, use the YOLOV5 backbone network to extract features, and output three multi-scale smoke feature maps with different resolutions extracted.

[0041] After the input fire smoke image enters the backbone network, it first passes through multiple rounds of 3×3 convolutions to generate three feature maps p0, p1, and p2 with the number of channels being 256, 512, and 1024 respectively. These three feature maps are the features input into the BIFPN;

[0042] Step 2. Construct a feature fusion network based on Recursive BiFPN to perform feature fusion processing on the multi-scale smoke features obtained in Step 1, and perform a recursive operation on the fused features, fuse them again with the original features of the same resolution, and output the features after the recursive and secondary fusion operations;

[0043] Step 2.1 Send the three multi-scale fire smoke feature maps with different resolutions in Step 1 into the feature fusion network;

[0044] Step 2.2 Construct a Recursive BIFPN feature fusion network

[0045] Step 2.2.1 In the feature fusion network, first perform deconvolution upsampling and pooling downsampling on the three smoke feature maps with different resolutions obtained in Step 1 from low to high and then from high to low, and cascade the input and output, thereby fusing the smoke feature information of different convolutional layers.

[0046] For the three smoke feature maps p0, p1, and p2 output by the backbone network: first upsample p2 to the size of p1 and cascade it with p1 to obtain p′1; then upsample p′1 to the size of p0 and cascade it with p′0 to obtain p′0; then downsample p′0 to the size of p1 and cascade it with the original p′1 to obtain a new p′1; then downsample the new p′1 to the size of p2 and cascade it with p2 to obtain Upsample once and cascade it with the new p′1 and cascade it with the original input p1 to obtain Upsample once and cascade it with p′0 to obtain Final output As the new input;

[0047] Step 2.2.2 As Figure 2 shown, perform a Recursive operation on the three feature maps with different resolutions after the first fusion in BIFPN in Step 2.2.1, that is, return to the original backbone network;

[0048] Step 2.2.3 Use BIFPN to perform secondary fusion on the three features with different resolutions returned in Step 2.2.2 and the backbone network features of the corresponding resolutions to obtain three multi-scale feature maps with different resolutions again;

[0049] That is, complete the construction of Recursive BIFPN and obtain three feature maps again P1 * , And according to different importance levels, corresponding weights are assigned to feature maps of different resolutions;

[0050] Step 3. The Swin Transformer Encoder module is introduced into YOLOV5 to replace the original detection head of YOLOV5, and a classification detection module with global modeling ability is constructed. The structure of the Swin Transformer is as Figure 3 shown. The detection head based on the Swin Transformer classifies and predicts the fused features obtained in Step 2. The prediction network performs regression of the position information of the smoke and confidence prediction on three output smoke feature maps ;

[0051] Step 3.1 The Swin Transformer Encoder module is introduced into YOLOV5 to replace the original detection head of YOLOV5. A Swin Transformer Encoder module is added to the prediction head of each feature map, and the picture is divided into several local windows through it;

[0052] Step 3.2 Self-attention calculation is performed in each local window through W-MSA (Window Multi-Head Self Attention) to extract local features of the smoke image;

[0053] As Figure 3 shown, assume that a local window is divided into small windows of size M×M, then the calculation formula of self-attention is as shown in Equation (1):

[0054]

[0055] Among them, are the matrices of query, key, and value of the Transformer; the dimension of d is query / key, the dimension of d is query / key, and B is the relevant position bias of the Transformer.

[0056] As Figure 4 shown, the Swin Transformer changes the Multi-Head Self Attention (MSA) of the Transformer to Window Multi-Head Self Attention (W-MSA). Let and z l be the output features of the l-th small window of the W-MSA module and the MLP module respectively, then and zl The recurrence formulas are shown in Equations (2) and (3) respectively:

[0057]

[0058]

[0059] where LN is the lateral normalization operation of the Transformer.

[0060] In Step 3.3, SW-MSA (Shifted Window Multi-Head Self Attention) is used to enable interaction between two adjacent windows, achieving the ability of global modeling, and then extracting the global information of the fire smoke image. As Figure 4 shown, the output feature calculation formulas of SW-MSA are as in (4) and Equation (5):

[0061]

[0062]

[0063] where LN is the lateral normalization operation of the Transformer.

[0064] In Step 3.4, by analyzing the features in different windows, the location where the fire smoke exists is finally classified and determined. The Swin Transformer Encoder module adopts a hierarchical structure, enabling different receptive fields of the network to extract local features and global information of the fire smoke image under the premise of being unaffected by the shooting distance. The steps include:

[0065] In Step 3.4.1, the prediction head of each feature map contains a Swin Transformer encoder block, and finally five prediction values are output for each suspected smoke area: representing the four position coordinates and a confidence level of the predicted smoke respectively;

[0066] In Step 3.4.2, the scores of the results predicted by all prediction heads are sorted, and the boxes with scores greater than the set threshold in each suspected smoke determination area are selected and determined to have smoke here;

[0067] In Step 3.4.3, when using the Swin Transformer prediction network to classify non-smoke interference areas such as clouds and fog, the interference objects judged as non-smoke are no longer output with position information and confidence levels.

[0068] In Step 3.4.4, the boxes with high overlap are recognized as the same smoke area and non-maximum suppression is performed, and finally the smoke area with the highest confidence level is obtained as the final prediction result.

[0069] Experiment:

[0070] The Recursive BIFPN feature fusion network proposed by the present invention can well distinguish interference objects from fire smoke. Images containing clouds and fog were selected for experiments and compared with the existing BIFPN. The experimental results are as Figure 5(a) - Figure 5(e) shown. It can be seen from the figure that compared with BiFPN, the feature map after Recursive BiFPN not only pays more attention to the smoke area, but also can better distinguish clouds from smoke.

[0071] In addition, as Figure 6 , the present invention can not only well detect white smoke, but also can better identify fire smoke that is difficult to detect, such as small smoke, black smoke, brown smoke, etc.

[0072] To verify the advancement of the proposed method, a comparative experiment was conducted between the proposed smoke detection method and the existing advanced smoke detection network. The comparison includes prediction accuracy and recall rate. The experimental results are shown in Table 1 Comparison of Experimental Results of Fire Smoke Recognition. It can be seen from the comparison of the experimental results in Table 1 that the smoke detection method proposed by the present invention has the highest values in the three measures of accuracy, recall rate and Map in smoke detection, and can efficiently and accurately identify wildfire smoke images.

[0073] Table 1

[0074]

[0075]

[0076] Note: The accuracy rate is calculated as the average accuracy rate when the intersection over union threshold is 0.5, and the recall rate is calculated as the average of 10 recall rates obtained at intervals of 0.05 from the intersection over union threshold of 0.5 to 0.95.

[0077] To verify the detection effect of the fire smoke network proposed by the present invention on fire smoke that is difficult to identify, such as small smoke and black smoke, 100 small smoke images and 50 black smoke images were selected for testing, and compared with mainstream object detection convolutional networks such as Faster RCNN, Efficientdet, and SSD. It can be seen from the experimental results in Table 2 that the Map value of the fire smoke method proposed by the present invention is the highest, and the accuracy rate and recall rate also reach relatively high values, and a good balance can be achieved between the accuracy rate and the recall rate. Although Faster RCNN has a relatively high recall rate, its accuracy rate is only 0.378.

[0078] Table 2

[0079] Method Accuracy Recall Map Faster RCNN 0.378 0.701 0.584 Efficientdet 0.728 0.406 0.497 SSD 0.780 0.131 0.411 YOLOV5 0.845 0.631 0.632 The method of the present invention 0.847 0.674 0.674

[0080] Summary:

[0081] The smoke detection method of the present invention is a fire smoke target detection method based on Recursive BIFPN and Swin Transformer. The fire smoke detection network based on this feature fusion network shows remarkable performance on the test data set. The present invention designs Recursive BIFPN, which can enhance the extracted multi-scale features of fire smoke. Compared with ordinary feature fusion networks, the feature fusion network of the present invention achieves better feature enhancement and fusion performance on image data sets, and can distinguish objects such as clouds similar to smoke existing in the background. At the same time, the present invention introduces Swin Transformer, which is good at extracting global information, into the prediction network, improving the detection accuracy for different scales of fire smoke, especially small-scale fire smoke.

[0082] Experiments show that this detection method has obvious advantages in terms of the accuracy of detecting various scales of smoke.

Claims

1. An outdoor fire smoke image detection method based on the Recursive BIFPN network, the steps include: First, collect fire smoke pictures; Then, use a fire smoke detection network for detection; it is characterized in that the steps of using the fire smoke detection network to detect smoke targets include: Step 1) Input the fire smoke picture to be detected, extract features using the YOLOV5 backbone network, and output three multi-scale fire smoke feature maps with different resolutions extracted; Step 2) Construct a feature fusion network of Recursive BiFPN, and perform feature fusion processing on the multi-scale smoke features obtained in Step 1) using the feature fusion network based on Recursive BiFPN; Step 3) Introduce the Swin Transformer Encoder module into YOLOV5, replace the bottleneck blocks in the original CSPDarknet53 of YOLOV5, construct a classification and detection module with global modeling ability, and use the detection head based on Swin Transformer to perform classification prediction on the fused features obtained in Step 2), and judge the presence of fire smoke and its position in the image; In the said Step 2): 2.1) Send the three multi-scale fire smoke feature maps with different resolutions in Step 1) into the feature fusion network; 2.2) Construct a Recursive BIFPN feature fusion network, fuse the multi-scale fire smoke features obtained in Step 2.1), perform a Recursive operation on the fused features, and fuse them with the original features of the same resolution again, 2.3) Output the features after the Recursive operation and the secondary fusion operation; In the said Step 3): 3.1) Introduce the Swin Transformer Encoder module into YOLOV5 and replace the original detection head of YOLOV5; 3.2) Calculate self-attention in each local window through W-MSA to extract local features of the smoke picture; W-MSA is the abbreviation of Window Multi-Head Self Attention; 3.3) Enable interaction between two adjacent windows through SW-MSA to achieve the ability of global modeling, and then extract the global information of the fire smoke picture; SW-MSA is the abbreviation of Shifted Window Multi-Head Self Attention; 3.4) According to the features in different windows, finally classify and determine the position where the fire smoke exists; The Swin Transformer Encoder module adopts a hierarchical structure, so that the receptive fields of the network are different, and is used to extract local features and global information of the fire smoke image.

2. The outdoor fire smoke image detection method based on the Recursive BIFPN network according to claim 1, characterized in that In Step 2.2), when constructing the Recursive BIFPN feature fusion network, the steps include: 2.2.1) Use BIFPN to perform the first fusion on the multi-scale fire smoke feature maps of different resolutions in step 1): first perform transposed convolution upsampling and pooling downsampling from low to high and then from high to low in sequence, and cascade the input and output, thereby fusing the multi-scale fire smoke feature information maps of different convolutional layers; 2.2.2) Perform a recursive operation on the three feature maps of different resolutions after the first fusion by BIFPN in step 2.2.1), that is, return to the original backbone network; 2.2.3) Use BIFPN to perform a secondary fusion of the multi-scale fire smoke feature maps of three different resolutions returned in step 2.2.2) with the backbone network of the corresponding resolution to obtain multi-scale fire smoke feature maps again; then, according to different importance, assign corresponding weights to the multi-scale fire smoke feature maps of different resolutions.

3. The outdoor fire smoke image detection method based on the Recursive BIFPN network according to claim 1, wherein In step 3), use the Swin Transformer detection head to perform regression of the position information of the smoke and confidence prediction on three smoke feature maps , , respectively, classify and locate the fire smoke, sort the scores of the output results, and finally obtain the final prediction result through the non-maximum suppression layer, where: Each prediction head contains a Swin Transformer encoder block, and five prediction values are output corresponding to each suspected smoke area. The five prediction values respectively represent the four position coordinates and a confidence level of the predicted smoke; Sort the scores of the results predicted by all prediction heads, select the prediction boxes with scores greater than the set threshold in each suspected smoke determination area, and determine that there is smoke here; When using the Swin Transformer detection head to classify non-smoke interference areas, the interference objects determined as non-smoke are no longer output with position information and confidence level; For the prediction boxes with high overlap, they are determined to be the same smoke area and non-maximum suppression is performed. Finally, the smoke area with the highest confidence level is obtained as the final prediction result.

4. The outdoor fire smoke image detection method based on the Recursive BIFPN network according to claim 3, characterized in that In step 3.2), the method of calculating self-attention in each local window through W-MSA is as follows: Suppose a local window is divided into small windows of size The calculation formula of self-attention is as shown in Equation (1): （1） Among them, is the matrix of the query, key, and value of the Transformer; the dimension of d is query / key, and B is the relative position bias of the Transformer; Swin Transformer changes the MSA of the Transformer to W-MSA, so that and are the output features of the th small window of the W-MSA module and the MLP module respectively. Then and The recurrence formulas of are as shown in Equations (2) and (3) respectively: （2） （3） Among them, LN is the lateral normalization operation of the Transformer.

5. A method for detecting outdoor fire smoke images based on the Recursive BIFPN network according to claim 1, characterized in that In step 3.3), the output feature calculation formula of SW-MSA is as shown in equations (4) and (5): （4） （5） Among them, LN is the lateral normalization operation of the Transformer, and are the output features of the th small window of the W-MSA module and the MLP module, respectively; Step 3.3) enables interaction between two adjacent windows through SW-MSA to achieve the ability of global modeling, thereby extracting the global information of the fire smoke picture.

Images