Forest fire detection algorithm based on spatial frequency interaction and imitation learning decoupling head

Abstract

The invention provides a forest fire detection algorithm based on a spatial frequency interaction and imitation learning decoupling head, a forest image is sent to a backbone network to extract multi-scale features of the image, a path aggregation feature pyramid network fuses the multi-scale features to obtain further features, and the further features are sent to a detection head to obtain a detection result. A spatial frequency interaction module SFIM is introduced, Fourier transform is utilized to extract color information and semantic information of an image in a frequency domain, and the color semantic information acquired by the frequency domain is complemented to a spatial domain to help a network to learn discriminative features of flames shielded by smoke. A decoupling head with an imitation branch is proposed to decouple a bounding box regression task into a bounding box position regression task and a bounding box size regression task, and a positive sample lacking flame space information and a positive sample containing the flame space information are made to learn each other, so that flame features in the positive samples are supplemented, and the positioning precision of flames of all scales is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a forest fire detection algorithm based on spatial frequency interaction and imitation learning decoupling head. Background Art

[0002] Forest fires not only pose a serious threat to the safety of life and property of surrounding residents, but also damage forest ecosystems, leading to loss of biodiversity and intensified climate change. Manual patrols are a traditional way of monitoring forest fires, requiring forest rangers and other professionals to patrol the forest regularly or irregularly to promptly discover and report fires, but this method is relatively time-consuming and laborious. Designing an efficient and real-time forest fire detection algorithm can not only reduce the burden on forest rangers, but also help to accurately and quickly detect fires, thereby greatly reducing property losses.

[0003] With the advancement of remote sensing image technology and computer vision, the combination of remote sensing images and target detection algorithms to detect forest fires has attracted the attention of more and more researchers. At present, some researchers have proposed a new embedded graph forest fire detection framework, which uses the contextual information of targets of different scales to improve the performance of small-scale forest fire detection. And some researchers have developed a multi-scale fusion network to solve the problem of low detection accuracy caused by different target scales. And some researchers have proposed a new feature pyramid network, which effectively improves the effect of different scale flame target detection by enhancing the information of high-level feature maps, channels and spatial information in the process of high-level feature extraction. These works combine multi-scale fusion strategies with target detection technology, fuse the edge texture information of low-level features with the rich semantic information of high-level features, effectively make up for the limitations of single-scale feature extraction, and thus learn discriminative features. This fusion mechanism significantly enhances the model's ability to distinguish background noise from flame targets in complex scenes, especially in the detection of small-scale flame targets. However, these methods still have the following problems:

[0004] (1) Existing networks have limited ability to extract flame targets obscured by smoke. This is mainly due to the serious interference of smoke on the visual features of flames. Under its occlusion, the outline and features of the flames become blurred. The network cannot learn these features, resulting in missing feature information, which directly affects the accuracy and performance of the target detection algorithm. Specifically, smoke not only reduces the overall clarity of the image, causing image quality degradation, but also may change the color, shape and texture features of the flame, making it impossible for the algorithm to effectively separate the flame target from the smoke background, further exacerbating the difficulty of target detection. In extreme cases, when the smoke concentration reaches a certain level, the flame target may be completely obscured, causing the target detection algorithm to completely fail. In response to this challenge, researchers are committed to developing flame detection networks that can effectively deal with smoke occlusion.

[0005] (2) When dealing with flame targets in forest fire scenes, current network models often fail to accurately locate flame targets because they cannot fully capture the flame’s spatial information. This is mainly because flame targets at fire scenes have multiple scales and significant shape differences, and the network matching positive samples cannot well cover the flame target’s spatial information, limiting the model’s ability to learn flame spatial information. Summary of the invention

[0006] The present invention provides a forest fire detection algorithm based on spatial-frequency interaction and imitation learning decoupling head, which combines the target detection network of spatial-frequency mutual learning and imitation learning decoupling head, aiming to solve the problems of smoke occlusion and difficulty in locating flame targets of different scales in forest fire detection.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A forest fire detection algorithm based on space-frequency interaction and imitation learning decoupling head, which sends the forest image to the backbone network Backbone, which extracts the multi-scale features of the image. The multi-scale features of the image are processed by the space-frequency interaction module SFIM and sent to the path aggregation feature pyramid network. The path aggregation feature pyramid network fuses the multi-scale features to obtain further features, which are then sent to the detection head to obtain the detection results, i.e., whether there is a fire point in the image and the location and range of the fire point.

[0009] The above-mentioned space-frequency interaction module SFIM consists of two branches: space DWConv and frequency Fre. First, the information in the spatial domain is converted to the frequency domain through fast Fourier transform FFT; in the frequency domain, the structure and semantic information of the flame are learned through amplitude and phase respectively, and then the integrated features in the frequency domain are converted back to the spatial domain through inverse Fourier transform IFFT.

[0010] The above-mentioned space-frequency interaction module SFIM also interacts with the AIM interaction module to guide the network to combine spatial and frequency domain features through spatial attention SA and frequency channel attention FA.

[0011] The functions of the space, frequency and interaction modules in the above-mentioned space-frequency interaction module are as follows:

[0012] First, f i ∈R C×H×W Represented as the input feature of the space-frequency interaction module, the spatial branch uses a 3×3 depth-separable convolution to process the information in the spatial domain and obtain fs1:

[0013] f s1 =GELU(Conv2d(f i )); (1)

[0014] The frequency branch first uses Fourier transform to convert fi into Fourier space to obtain F f0 , the specific operation formula is as follows:

[0015] F f0 =F(f i ); (2)

[0016] In the frequency domain, F f0 After processing, as shown in formula (3), the amplitude components AF are obtained respectively. f0 and phase component PF f0 :

[0017]

[0018] Then let each branch pass through two layers of convolutional layers with a convolution kernel of 3×3 and then be combined with F f0 Reconstruction is used to enhance phase and frequency information. The specific operation is expressed as:

[0019]

[0020] Finally, the feature map in the frequency domain is converted back to the spatial domain through inverse Fourier transform:

[0021] f f1 =F -1 (F A1 , F P1 ); (7)

[0022] Next, in order to transform the spatial branch f s1 and frequency branch f f1 The frequency interaction FA and the spatial interaction SA are designed to work between the two branches respectively. According to the characteristics of each branch, they adaptively reweight the features of the two branches from the spatial or frequency dimension; SA calculates the output f of the spatial branch. s1 ∈R C×H×W The spatial attention map is denoted as S-MAP and has a size of R 1×H×W , which is expressed as:

[0023] S-MAP(f s1 )=f(W2σ(W1B)); (8)

[0024] Where f(·) is the sigmoid function, σ(·) represents the GELU function, and W(·) represents the weight of the point-by-point convolution, which is used to reduce or enlarge the channel dimension;

[0025] The output of the frequency branch f f1 , feature f f1It is divided into multiple parts along the channel dimension; denoted as Among them A i ∈R Cj×H×W ,i∈{0,1,...,n-1},C j =C / n, and C should be divisible by n. For each part, the corresponding 2D DCT frequency component is assigned, and the 2D DCT result can be used as the compression result of channel attention, so that:

[0026]

[0027] where [ui,vi] is f f1 i The corresponding frequency component two-dimensional index, DCT is discrete cosine transform, Freqi∈R Cj The compressed C j dimensional vector, the entire compressed vector can be obtained by cascading:

[0028] Freq=compress(f f1 )=cat([Freq0, Freq1,..., Freq n-1 ]); (10)

[0029] where Freq∈R C×1×1 For the obtained multispectral vector, the frequency attention map of the entire multispectral channel attention can be written as:

[0030] F-MAP(f f1 )=sigmoid(fc(Freq)); (11)

[0031] The attention map is then applied to another input, here f f1 , thereby achieving interaction; the process is expressed as:

[0032] F f =SA(f s1 , f f1 ) = f f1 ☉S-MAP(f s1 ); (12)

[0033] F s =FA(f f1 , f f1 ) = f s1 ☉F-MAP(f f1 ); (13)

[0034] Where ⊙ represents element-by-element multiplication;

[0035] In order to effectively mix the interactive feature information, we extract the frequency feature F in the channel dimension f and spatial feature F s concatenate them and feed them into a convolutional layer to learn two main indexes w f and w s , which can be expressed as:

[0036] w f ,w s =softmax(Conv(cat[F f , F s ])); (14)

[0037] Where Conv(·) is two convolutional layers with a kernel of 3×3. Each pixel is assigned a corresponding weight, which is multiplied element-wise by the corresponding feature to reconstruct the output; the process can be defined as follows:

[0038]

[0039] in represents element-wise multiplication, Represents element-by-element addition.

[0040] The above-mentioned detection head is a decoupled detection head with an imitation branch. The imitation branch allows the decoupled prediction box features to learn from each other to improve the prediction accuracy of the object frame size and position.

[0041] The above-mentioned decoupling detection head detection process is:

[0042] The real box GTbbox of an object m , m∈{0, 1, 2, ..., M-1}, M represents the number of objects, there will be 9 prediction boxes Pbbox mlij , l∈{3,4,5}, represents the level of the feature pyramid; i∈{0,1,2}, represents different grids: the grid where the center of the object falls, that is, when i is 0 and the two grids closest to the center of the object, that is, when i is 1 or 2; j∈{0,1,2} represents different anchor boxes; the detection head generates a prediction box Pbbox mlij The prediction box offset (tx, ty, tw, th) needs to be converted to the prediction box (bx, by, bw, bh) using the following formula:

[0043]

[0044] Where cx, cy is the prediction box Pbbox mlij The upper left corner position of the corresponding grid, p w , p his the width and height of the anchor box;

[0045] Size feature wh mlij After passing through the multi-layer perceptron, embed_wh, which encodes the characteristic size information of the flame, is obtained mlij , in the embedding space, the size embed of i taking 1, 2 is brought closer to the corresponding size embed of i taking 0. The specific expression is as follows:

[0046]

[0047] In order to increase the positioning accuracy, the position feature xy mlij After passing through the multi-layer perceptron, the embedding of the encoded flame feature position information is obtained In the embedding space, embeds with different j are brought closer to each other. The specific expression is as follows:

[0048]

[0049] Where j,k∈{0, 1, 2} represents different anchor boxes.

[0050] The present invention provides a forest fire detection algorithm based on spatial frequency interaction and imitation learning decoupling head, and the present invention has the following technical effects:

[0051] (1) The present invention provides a forest fire detection algorithm, which introduces a space-frequency interaction module to use Fourier transform to extract the color information and semantic information of the image in the frequency domain, and then completes the color semantic information obtained in the frequency domain to the spatial domain to help the network learn the discriminative features of the flame obscured by smoke, and further distinguish the flame from the background.

[0052] (2) In addition, a decoupling head with an imitation branch is proposed to first decouple the bounding box regression task into the bounding box position regression task and the bounding box size regression task. Then, through the idea of ​​comparative learning, the positive samples lacking flame spatial information and the positive samples containing flame spatial information learn from each other, thereby supplementing the flame features in the positive samples and improving the positioning accuracy of flames at all scales. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0054] Figure 1 This is the overall network structure diagram of the present invention;

[0055] Figure 2 Schematic diagram of the structure of the spatial-frequency interaction module. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present invention clearer, the following content will systematically and completely describe the specific technical solutions of the present invention in combination with the drawings provided according to the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] like Figure 1 As shown, a forest fire detection algorithm based on spatial frequency interaction and imitation learning decoupling head includes the following steps:

[0058] The forest image is sent to the backbone network, which extracts the multi-scale features of the image. The multi-scale features of the image are processed by the spatial frequency interaction module and sent to the path aggregation feature pyramid network. The path aggregation feature pyramid network fuses the multi-scale features to obtain further features, which are then sent to the detection head to obtain the detection results, i.e., whether there are fire points in the image and the location and range of the fire points.

[0059] The above-mentioned spatial frequency interaction module is as follows Figure 1 As shown in the upper left corner, it consists of two branches, space and frequency. The spatial module is good at learning color and texture features in the spatial domain. The frequency branch is mainly composed of Fourier transform. First, the information in the spatial domain is converted to the frequency domain through fast Fourier transform. In the frequency domain, the structure and semantic information of the flame are learned respectively through amplitude and phase, and then the integrated features in the frequency domain are converted back to the spatial domain through inverse Fourier transform.

[0060] The above-mentioned spatial-frequency interaction module also interacts with the adaptive interaction module to guide the network to more flexibly combine spatial and frequency domain features through spatial attention and frequency channel attention.

[0061] The functions of the space, frequency and adaptive interaction modules in the above-mentioned space-frequency interaction module are as follows:

[0062] like Figure 2 As shown, first f i ∈R C×H×W Represented as the input feature of the space-frequency interaction module, the spatial branch uses a 3×3 depth-separable convolution to process the information in the spatial domain and obtain f s1 :

[0063] f s1 =GELU(Conv2d(f i )); (1)

[0064] The frequency branch first uses Fourier transform to convert fi into Fourier space to obtain Ff0. The specific operation is shown as follows:

[0065] F f0 =F(f i ); (2)

[0066] In the frequency domain, F f0 After processing, as shown in formula (3), the amplitude components are obtained respectively and phase component

[0067]

[0068] Then let each branch pass through two layers of convolutional layers with a convolution kernel of 3×3 and then be combined with F f0 Reconstruction is used to enhance phase and frequency information. The specific operation is expressed as:

[0069]

[0070] Finally, the feature map in the frequency domain is converted back to the spatial domain through inverse Fourier transform.

[0071] f f1 =F -1 (F A1 , F P1 ); (7)

[0072] Next, in order to transform the spatial branch f s1 and frequency branch f f1 The frequency interaction and spatial interaction are designed respectively, which act between the two branches respectively. According to the characteristics of each branch, the features of the two branches are adaptively reweighted from the spatial or frequency dimension; SA calculates the output f of the spatial branch s1 ∈R C×H×W The spatial attention map is denoted as S-MAP and has a size of R 1×H×W , which is expressed as:

[0073] S-MAP(f s1 )=f(W2σ(W1B)); (8)

[0074] Where f(·) is the sigmoid function, σ(·) represents the GELU function, and W (·) Represents the weight of point-by-point convolution, which is used to reduce or enlarge the channel dimension;

[0075] The output of the frequency branch f f1 , feature f f1 It is divided into multiple parts along the channel dimension; denoted as [f f1 0 ,f f1 1 ,…,f f1n-1 ], where A i ∈R Cj×H×W ,i∈{0,1,…,n-1},C j =C / n, and C should be divisible by n. For each part, the corresponding 2D DCT frequency component is assigned, and the 2D DCT result can be used as the compression result of channel attention, so that:

[0076] Freq i =2D DCT Gi,Hi (f f1 i ); (9)

[0077] where [ui,vi] is f f1 i The corresponding two-dimensional index of the frequency component, DCT is discrete cosine transform, Freq i ∈R Cj The compressed C j dimensional vector, the entire compressed vector can be obtained by cascading:

[0078] Freq=compress(f f1 )=cat([Freq0,Freq1,…,Freq O21 ]); (10)

[0079] where Freq∈R C×1×1 For the obtained multispectral vector, the frequency attention map of the entire multispectral channel attention can be written as:

[0080] F-MAP(f f1 )=sigmoid(fc(Freq)); (11)

[0081] The attention map is then applied to another input, here f f1 , thereby achieving interaction; the process is expressed as:

[0082] F f =SA(f s1 ,f f1 ) = f f1 ⊙S-MAP(f s1 ); (12)

[0083] F s =FA(f s1 ,f f1 ) = f s1 ⊙F-MAP(f f1 ); (13)

[0084] Where ⊙ represents element-by-element multiplication;

[0085] In order to effectively mix the interactive feature information, we extract the frequency feature f from the channel dimension f and spatial characteristics f s Concatenate them and input them into the convolutional layer to learn two main indexes wf and ws, which can be expressed as:

[0086] w f ,w s =softmax(Conv(cat[F f ,F s U)); (14)

[0087] Where Conv(·) is two convolutional layers with a kernel of 3×3. Each pixel is assigned a corresponding weight, which is multiplied element-wise by the corresponding feature to reconstruct the output; the process can be defined as follows:

[0088]

[0089] in represents element-wise multiplication, Represents element-by-element addition.

[0090] The above-mentioned detection head is a decoupled detection head with an imitation branch. The imitation branch allows the decoupled prediction box features to learn from each other to improve the prediction accuracy of the object frame size and position.

[0091] The above-mentioned decoupling detection head detection process is:

[0092] The real box GTbbox of an object m , m∈{0, 1, 2, ..., M-1}, M represents the number of objects, there will be 9 prediction boxes Pbbox mlij , l∈{3,4,5}, represents the level of the feature pyramid; i∈{0,1,2}, represents different grids: the grid where the center of the object falls, that is, when i is 0 and the two grids closest to the center of the object, that is, when i is 1 or 2; j∈{0,1,2} represents different anchor boxes; the detection head generates a prediction box Pbbox mlij The predicted box offset (t x ,t y ,t w ,t h ), prediction box offset (t x ,t y ,t w ,t h ) needs to be converted into a prediction box (b x ,b y ,b w ,b h):

[0093]

[0094] Where c x ,c y Pbbox is the prediction box mlij The upper left corner position of the corresponding grid, p w ,p h is the width and height of the anchor box;

[0095] From the above formula, we know that if we want to predict the box Pbbox mlij With real box GTbbox m As close as possible, the position offset (t x ,t y ) must be different, the predicted box size (t w ,t h ) must be the same. In addition, the position offset (t x ,t y ) requirements are the same. The nine predicted boxes corresponding to a real box have some parts that need to be consistent and some parts that are inconsistent. In order to solve the overall inconsistency problem caused by this partial inconsistency, this section decouples each feature representation of the object into 4 parts, and also decouples the features corresponding to each anchor box: semantic feature cls mlij 、Target feature conf mlij , position feature xy mlij and size characteristics wh mlij . Semantic feature cls mlij Used for classification tasks to obtain the category of the object, the target feature conf mlij Used for binary classification tasks to distinguish objects from backgrounds, position features xy mlij Used for regression tasks to determine the center point position of the prediction box and the size feature wh mlijUsed to determine the size of the prediction box. The semantic feature part in the feature representation of the same object must be consistent, because the same object must be the same type of object. However, because this study only needs to detect one type of object: flames, there is no need to perform a classification task in the detection network, and it can be completed through the target judgment task, because as long as a sample is determined to be an object, then this object must be the object to be detected. The loss of the classification part in the original detection network is ignored, and accordingly, the semantic features after feature decoupling in this section are also ignored. That is, the semantic features in this section are neither sent to the classification head to complete the classification of the target, nor are they used to constrain the features of each positive sample. The size features and position features obtained after decoupling are used as features for mutual learning, thereby increasing the scale information and position information contained in the flame features.

[0096] like Figure 1 As shown in the upper right corner, the size feature wh mlij After passing through the multi-layer perceptron, we get the embedding enbed_wh that encodes the characteristic size information of the flame mlij , in the embedding space, the size embed of i taking 1, 2 is brought closer to the corresponding size embed of i taking 0. The specific expression is as follows:

[0097]

[0098] In order to increase the positioning accuracy, the position feature xy mlij After passing through the multi-layer perceptron, the embedding of the encoded flame feature position information is obtained In the embedding space, embeds with different j are brought closer to each other. The specific expression is as follows:

[0099]

[0100] Where j,k∈{0, 1, 2} represents different anchor boxes.

[0101] Through this operation of pulling them closer together, the size and position information of the flame features are increased, thereby improving the accuracy of the prediction box.

[0102] Embodiment 1:

[0103] 1) Parameter settings

[0104] In the present invention, YOLOv5s is used as a baseline to construct a proposed forest fire detection algorithm based on spatial frequency interaction and imitation learning decoupling head. In the present invention, PyTorch framework (version 1.13.0) and Python (version 3.10.12) and CUDA (version 11.7) are used for model development, and training is performed on NVIDIA 3090GPU under Linux Ubuntu 18.04.6 operating system. Ablation experiments and comparative experiments are performed using forest fire datasets to verify the effectiveness of our invented algorithm. The image size of the input model in the training and inference stages is 640×640. The optimizer uses Stochastic Gradient Descent (SGD), the momentum is 0.937, the weight decay is 0.0005, the batch-size is set to 16, the initial value of the learning rate is 0.01, and the number of training rounds is 300.

[0105] 2) Experimental indicators

[0106] In the present invention, the present invention uses evaluation indicators such as mAP50 (mean accuracy of 50% IOU), precision, recall, FPS, etc., and uses mAP50 to compare and measure the detection performance of the model. It comprehensively considers the precision (Precision) and recall rate (Recall) of the model, and mAP50 is calculated by Precision and Recall. The higher the mAP50 value, the better the model performance. In forest fire detection, high precision indicates that the model can more accurately identify fire instances and reduce the probability of misclassifying non-fire instances as fires. Recall rate (Recall), also known as sensitivity or true positive rate, evaluates the ability of the model to identify positive instances relative to actual positive instances.

[0107] 3) Comparative test

[0108] In order to verify the effectiveness of the forest fire detection algorithm based on spatial frequency interaction and imitation learning decoupling head proposed in this invention, we trained and tested it on a self-built forest fire image dataset. In this invention, we compared the algorithm proposed in this invention with advanced detectors such as Faster RCNN, ATSS, Mask-RCNN, FCOS, etc. In order to compare these twelve networks in detail, we used the same equipment and processing rules for all networks. Among these advanced networks, Cascade RPN performed best on the forest fire dataset. Compared with Cascade RPN, the algorithm proposed in this invention improved the detection accuracy by 3.5%. It is worth noting that the number of parameters of Cascade RPN is 5.2 times that of the algorithm proposed in this invention, and the computational complexity is 7.5 times that of the algorithm proposed in this invention. In addition, the FPS value, which is 5.8 times higher than that of Cascade RPN, shows its excellent real-time performance, which proves that the algorithm of this invention has significant advantages in model size, computational complexity, real-time performance, and detection accuracy. Compared with other mainstream models in Table 1, the algorithm proposed in the present invention has the best overall parameter quantity, computational complexity, detection accuracy and FPS. This comprehensive performance improvement has laid a solid foundation for the wide application of the present invention in practical scenarios.

[0109] Table 1 Comparison with other detection models

[0110]

[0111] 4) Ablation experiment

[0112] In order to evaluate the performance of each module and verify the effectiveness of the proposed forest fire detection algorithm based on spatial frequency interaction SFIM and imitation learning decoupling head FIDH, the present invention conducted an ablation experiment to compare the detection performance of the model and the number of parameters of the model to illustrate the impact of specific factors on the model performance, so as to deeply understand and optimize the model.

[0113] As shown in Table 2, in the first experiment, we used YOLOv5s, the network most used in the industry for forest fires, as the baseline. Its parameter volume is only 7.0M, but the detection performance reaches 79.7%, making it an ideal choice for drone deployment and real-time detection. In the second experiment, we introduced the spatial frequency interaction module SFIM into the network, which enables the effective interaction between the spatial domain and the frequency domain of the network and further integrates them, so that the network learns the discriminative features to distinguish between flames and backgrounds, and makes the overall features of the unclear flames originally obscured by smoke clearer under its action. The experimental results show that the detection performance has been significantly improved, from 79.7% to 81.2%. This shows that the SFIM module successfully encourages the model to capture target features more effectively, especially when dealing with forest fire scenes with smoke. In the third experiment, in order to explore whether the frequency domain features play a key role in improving the network detection effect in SFIM, or whether the performance improvement is only due to the increase in the number of parameters accumulated in the operation. We removed the Fourier transform and inverse Fourier transform operations and found that the number of parameters of the module after removal was 7.59M. The number of parameters did not change compared with before the removal operation, but its detection performance decreased by 0.6% compared with the baseline. This result shows that the increase in the number of parameters is not the reason for the improvement in network detection effect. In the fourth experiment, we introduced the imitation learning decoupling head FIDH to specifically solve the problem of significant differences in flame shape and scale in forest fire scenes. This design aims to make up for the inaccurate positioning defects caused by insufficient spatial information of the prediction box in traditional algorithms. Through the synergy of the decoupling head and the imitation branch, the features lacking flame spatial information and the features rich in flame spatial information are mutually imitated and learned, achieving effective supplementation of spatial information, and improving its overall detection accuracy by 1.4% compared with the baseline, improving the model's ability to locate flames in space. In the fifth experiment, the SFIM module and FIDH were fully integrated into the network, and special optimization was performed for flame targets of various scales that were severely obscured by smoke. The detection accuracy was further improved by comprehensively utilizing the interactive learning of the spatial and frequency domains and the mutual learning mechanism of the prediction box spatial information, reaching a detection accuracy of 82.4%, an increase of 2.7% compared to the baseline model. Surprisingly, the number of parameters at this time was still 7.59M.

[0114] These experimental results prove that the forest fire detection algorithm based on spatial frequency interaction and imitation learning decoupling head proposed in the present invention has significant superiority and efficiency in detecting targets of various scales obscured by smoke in forest fire scenes. From the perspective of practical application, the algorithm proposed in the present invention can be effectively deployed on mobile devices.

[0115] Table 2 Effects of spatial-frequency interaction module and imitation learning decoupling head

[0116]

Claims

1. A forest fire detection algorithm based on spatial-frequency interaction and imitation learning decoupling head, characterized in that: The forest image is sent to the backbone network Backbone, which extracts the multi-scale features of the image. The multi-scale features of the image are processed by the spatial frequency interaction module SFIM and sent to the path aggregation feature pyramid network. The path aggregation feature pyramid network fuses the multi-scale features to obtain further features, and then the further features are sent to the detection head to obtain the detection results, that is, whether there is a fire point in the image and the location and range of the fire point.

2. According to the forest fire detection algorithm based on spatial frequency interaction and imitation learning decoupling head described in claim 1, it is characterized in that: The space-frequency interaction module consists of two branches, space and frequency. First, the information in the spatial domain is converted to the frequency domain through fast Fourier transform; in the frequency domain, the structure and semantic information of the flame are learned through amplitude and phase respectively, and then the integrated features in the frequency domain are converted back to the spatial domain through inverse Fourier transform.

3. According to the forest fire detection algorithm based on spatial frequency interaction and imitation learning decoupling head described in claim 2, it is characterized in that: The space-frequency interaction module also interacts with the interaction module to guide the network to combine spatial and frequency domain features through spatial attention and frequency channel attention.

4. According to the forest fire detection algorithm based on spatial frequency interaction and imitation learning decoupling head described in claim 3, it is characterized in that: The functions of the space, frequency and interaction modules in the space-frequency interaction module are specifically as follows: First, f i ∈R C×H×W Represented as the input feature of the space-frequency interaction module, the spatial branch uses a 3×3 depth-separable convolution to process the information in the spatial domain and obtain f s1 : f s1 =GELU(Conv2d(f i )); (1) The frequency branch first uses Fourier transform to convert f i Transform to Fourier space to get F f0 , the specific operation formula is as follows: F f0 =F(f i ); (2) In the frequency domain, F f0 After processing, as shown in formula (3), the amplitude components are obtained respectively and phase component Then let each branch pass through two layers of convolutional layers with a convolution kernel of 3×3 and then be combined with F f0 Reconstruction is used to enhance phase and frequency information. The specific operation is expressed as: Finally, the feature map in the frequency domain is converted back to the spatial domain through inverse Fourier transform: f f1 =F -1 (F A1 ,F P1 ); (7) Next, in order to separate the spatial branch f s1 and frequency branch f f1 The frequency interaction FA and the spatial interaction SA are designed to work between the two branches respectively. According to the characteristics of each branch, they adaptively reweight the features of the two branches from the spatial or frequency dimension; SA calculates the output f of the spatial branch. s1 ∈R C×H×W The spatial attention map of is denoted as S-MAP and has a size of R 1×H×w , which is expressed as: S-MAP(f s1 )=f(W2σ(W1B)); (8) Where f(·) is the sigmoid function, σ(·) represents the GELU function, and W(·) represents the weight of the point-by-point convolution, which is used to reduce or enlarge the channel dimension; The output of the frequency branch f f1 , feature f f1 It is divided into multiple parts along the channel dimension; denoted as [f f1 0 , f f1 1 , ..., f f1 n -1 ], where A i ∈R Cj×H×W ,i∈{0,1,...,n-1},C j =C / n, and C should be divisible by n; for each part, the corresponding 2D DCT frequency component is assigned, and the 2D DCT result can be used as the compression result of channel attention, so that: Freq i =2D DCT ui，vi (f f1 i ); (9) where [ui,vi] is f f1 i The corresponding two-dimensional index of the frequency component, DCT is discrete cosine transform, Freq i ∈R Cj The compressed C j dimensional vector, the entire compressed vector can be obtained by cascading: Freq=compress(f f1 )=cat([Freq0,Freq1,...,Freq n-1 ]); (10) where Freq∈R C×1×1 For the obtained multispectral vector, the frequency attention map of the entire multispectral channel attention can be written as: F-MAP(f f1 )=sigmoid(fc(Freq)); (11) The attention map is then applied to another input, here f f1 , thereby achieving interaction; the process is expressed as: F f =S-A(f s1 ,f f1 )=f f1 ⊙S-MAP(f s1 ); (12) F s =F-A(f s1 ,f f1 )=f s1 ⊙F-MAP(f f1 ); (13) Where ⊙ represents element-by-element multiplication; In order to effectively mix the interactive feature information, we extract the frequency feature F in the channel dimension f and spatial feature F s concatenate them and feed them into a convolutional layer to learn two main indexes w f and w s , which can be expressed as: w f ,w s =softmax(Conv(cat[F f ,F s ])); (14) Where Conv(·) is two convolutional layers with a kernel of 3×3. Each pixel is assigned a corresponding weight, which is multiplied element-wise by the corresponding feature to reconstruct the output; the process can be defined as follows: in represents element-wise multiplication, Represents element-by-element addition.

5. A forest fire detection algorithm based on spatial frequency interaction and imitation learning decoupling head according to claim 4, characterized in that: The detection head is a decoupled detection head with an imitation branch, which allows the decoupled prediction box features to learn from each other through the imitation branch to improve the prediction accuracy of the object frame size and position.

6. A forest fire detection algorithm based on spatial frequency interaction and imitation learning decoupling head according to claim 5, characterized in that: The detection process of the decoupling detection head is as follows: The real box GTbbox of an object m , m∈{0, 1, 2, ..., M-1}, M represents the number of objects, there will be 9 prediction boxes PTbbox mlij , l∈{3, 4, 5}, represents the level of the feature pyramid; i∈{0, 1, 2}, represents different grids: the grid where the center of the object falls, that is, when i is 0, and the two grids closest to the center of the object, that is, when i is 1 or 2; j∈{0, 1, 2}, represents different anchor boxes; the detection head generates a prediction box Pbbox mlij The prediction box offset (tx, ty, tw, th) needs to be converted to the prediction box (bx, by, bw, bh) using the following formula: Where cx, cy is the prediction box Pbbox mlij The upper left corner position of the corresponding grid, p w ,p h is the width and height of the anchor box; Size feature wh mlij After passing through the multi-layer perceptron, embed_wh, which encodes the characteristic size information of the flame, is obtained mlij , in the embedding space, the size embed of i taking 1, 2 is brought closer to the corresponding size embed of i taking 0. The specific expression is as follows: In order to increase the positioning accuracy, the position feature xy mlij After passing through the multi-layer perceptron, the embedding of the encoded flame feature position information is obtained In the embedding space, embeds with different j are brought closer to each other. The specific expression is as follows: Where j,k∈{0, 1, 2} represents different anchor boxes.

Images