Forest fire detection method based on GS-YOLO v5

Abstract

The application discloses a forest fire detection method based on GS-YOLO v5. First, the data set is expanded by using methods such as flipping and artificially adding noise, and data sets such as clouds, rain, fog and sunset are added, so that the data amount is increased and the robustness of the algorithm is improved to a certain extent; secondly, a new Super-SPPF module is proposed in the paper, which learns from the CSP structure to avoid image processing distortion while retaining more semantic information, thereby reducing the false positive rate of the detection method. Then, through the serial SoftPooling and GhostConv structure, the detection speed is improved and the parameter amount of the model is reduced. Finally, the position attention (CA) module is introduced into the Backbone of YOLOv5 to obtain the relationship between the channel and the space, so that the network can more accurately obtain the position information of interest, further improving the detection accuracy of early fire. In order to verify the performance of the proposed method, the application establishes a DLFire data set, collects a large number of interference samples to improve the robustness of the model and effectively reduce the false positive rate.

Description

Technical Field

[0001] This invention relates to the field of target detection, and mainly to a forest fire detection method based on GS-YOLO v5. Background Technology

[0002] Due to the effects of global warming, forest fires are occurring frequently around the world. Each forest fire is a severe blow to the local ecosystem and endangers people's lives and property. Given the rapid spread of forest fires, early detection is crucial. Flame detection can be achieved using sensors and infrared or ultraviolet detectors. However, the detection environment for forest fires is vast and complex. Traditional flame detection technologies are more susceptible to interference and are more expensive, and they cannot provide spatial information about the fire's location. Satellite remote sensing can provide better results for detecting large-scale fires, but it is less effective for early detection of forest fires.

[0003] With the continuous development of deep learning, deep learning-based forest fire detection methods have been proposed. Unsupervised feature learning in deep learning algorithms not only solves the problem of manual feature extraction but also far surpasses traditional algorithms in both accuracy and speed. As a single-stage detector, each generation of the YOLO series has maintained a good balance between accuracy and speed. Redmond et al. proposed the first version (YOLO) in 2016, which is considered a breakthrough in real-time object detection / tracking. Previous object detectors (such as R-CNN and Fast R-CNN) relied on region proposal to suggest potential object bounding boxes in a given image, followed by bounding box classification to classify the objects within them, and finally bounding box refinement and deduplication. Despite the great success of these frameworks, their architectures were quite complex. For example, each of the preceding components required separate training, and inference remained slow. To address these bottlenecks, YOLO tackled the object detection problem from a regression perspective, with the previous components executed jointly in a single network. This significantly reduced inference overhead and false alarms. Furthermore, YOLO exhibited high generalization ability.

[0004] However, compared to other target detection methods, the images in the forest fire dataset are more complex and contain more interfering factors. Sunsets, maple leaves, fog, and other elements in the images can all interfere with detection, leading to a high false positive rate. The aforementioned algorithm, in its pursuit of speed, sacrifices the accuracy of the feature extraction layer, thus failing to achieve the desired detection precision. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a forest fire detection method based on GS-YOLO v5, which addresses the issues of excessive interference in the dataset leading to a high false positive rate in forest fire detection and insufficient feature extraction resulting in accuracy falling short of expectations.

[0006] The technical solution adopted by this invention to solve its technical problem mainly includes the following steps:

[0007] In this invention, virtual samples are first constructed by processing the dataset images using data augmentation methods. These virtual samples are then mixed with the original data to form a new dataset, which has 1.5 times the number of samples as the original dataset. The new dataset is then fed into the network for training. However, as the number of samples increases, the detection accuracy becomes insufficient.

[0008] Secondly, this paper improves the YOLO v5 network structure by adding a coordinate attention block (CA) after the C3 block at the end of the backbone network. This CA can be considered a computational unit that enhances the algorithm's ability to learn feature representations, accurately locating the position of objects of interest in the image without significantly increasing the number of parameters. A novel Super-SPPF module is used, which borrows from the CSP structure to avoid image processing distortion while preserving more semantic information, thereby reducing the false positive rate of the detection method. Through a sequential SoftPooling and GhostConv structure, the detection speed is improved and the number of model parameters is reduced.

[0009] The beneficial effects of this invention are as follows: By applying data augmentation algorithms to introduce artificial noise into the algorithm, the high false positive rate during training is effectively alleviated; by adding a positional attention mechanism after the last C3 layer in the backbone part of the YOLO v5 network, weights can be assigned to different spatial and channel positions of the feature map after training, thereby distinguishing discriminative and non-discriminative features, improving the overall feature extraction capability; and by assigning weights according to the importance of features, the integrity of the extracted feature information is ensured, ultimately improving the accuracy of small targets in forest detection. Furthermore, the application of the Super-SPPF structure retains more semantic information from the feature map, thus improving the computational speed. By combining the idea of ​​Ghost Modules in the backbone network and the Super-SPPF block, the number of model parameters is reduced. Attached Figure Description

[0010] Appendix Figure 1 This is a flowchart of a method for forest fire detection based on a GS-YOLO v5 network disclosed in this invention.

[0011] Appendix Figure 2 This is a diagram of the Super-SPPF architecture.

[0012] Appendix Figure 3 This is a schematic diagram of the Ghost Module.

[0013] Appendix Figure 4 This is a flowchart illustrating the overall implementation process of the present invention.

[0014] Appendix Figure 5 This is a network structure diagram of GS-YOLOv5. Detailed Implementation

[0015] The present invention is not limited to the following embodiments, and the specific implementation can be determined according to the technical solution of the present invention and the actual situation.

[0016] Combined with appendix Figure 1 This is a flowchart of a method for forest fire detection based on a GS-YOLO v5 network disclosed in this invention, which specifically includes the following steps:

[0017] A1. Construct a data-augmented sample dataset, apply the data to expand the sample dataset, obtain a new expanded dataset, and then feed the new dataset into the subsequent network for training, wherein:

[0018] The specific steps for constructing a sample dataset based on data augmentation include:

[0019] B1. Obtaining the forest fire dataset:

[0020] In this invention, the original forest fire dataset is used directly as input data without any dimensionality reduction processing. This preserves the data structure and yields complete image information.

[0021] B2. Data augmentation of the original dataset yields:

[0022] In this step, the original dataset is transformed into an expanded dataset by changing its saturation, adding mosaic, changing its color tone, and flipping the images.

[0023] B3. Combine the augmented dataset with the original dataset to obtain the data augmentation dataset:

[0024] In this step, the augmented dataset is merged with the original dataset to obtain a data augmentation dataset, and then randomly divided into training, validation, and test sets using a script.

[0025] B4. Label the training set, validation set, and test set:

[0026] The three datasets were labeled using labelImg to obtain labeled datasets.

[0027] A2. Construct a forest fire detection network based on YOLO v5. Using the YOLO v5 network as the base model, add an attention mechanism to the backbone part of the network and apply Super-SPPF to form an improved YOLO v5 network. The specific steps include:

[0028] B5. Add a CABlock after the last C3 layer in the internal Backbone structure:

[0029] CA decomposes channel attention into two 1D feature encoding processes, aggregating features along two spatial directions respectively. This allows for the capture of long-range dependencies along one spatial direction while preserving precise location information along the other. The resulting feature maps are then encoded into a pair of direction-aware and location-sensitive attention maps, which can be complementaryly applied to the input feature map to enhance the representation of the object of interest.

[0030] For a given input feature tensor X = [c1, x2, x3, ..., xc2], ... C The global information embedding of the c-th channel is shown in formula (1).

[0031]

[0032] Where z c This is the output associated with the c-th channel. The input feature tensor X comes directly from a convolutional layer with a fixed kernel size. The CA attention mechanism uses two pooling kernels, (H,1) and (1,W), to transform Equation (1) into two one-dimensional feature encoding operations. Therefore, the output of the c-th channel at height h is as shown in Equation (2):

[0033]

[0034] Similarly, the output of the c-th channel at width w can be expressed as:

[0035]

[0036] The two transformations described above aggregate features along two spatial directions, generating a pair of direction-aware feature maps. These two transformations allow the CA attention block to capture long-range dependencies along one spatial direction while preserving precise location information along the other. This helps the network more accurately locate the position of objects of interest.

[0037] After obtaining a pair of orientation-aware feature maps, coordinate attention is generated. First, it is fed into a transformation function F1 that shares a 1×1 convolution, as shown in Equation (4).

[0038] f=δ(F1([zh ,z w ])) (4)

[0039] Where [·,·] represents the cascade operation along the spatial dimension, and δ is a nonlinear activation function. This is an intermediate feature map that encodes spatial information in the horizontal and vertical directions. r is a scaling factor used to control the block size. Then, f is partitioned into two independent tensors along the spatial dimensions. and The other two 1×1 convolution transformations are used to transform F h and F w The transformation is to a tensor with the same channel number as the input X, as shown in Equations (5) and (6).

[0040] g h =σ(F h (f h (5)

[0041] g w =σ(F w (f w (6)

[0042] Where σ is the sigmoid function, which affects the output g. h and g w The values ​​are expanded and used as attention weights. The output of the coordinate attention block Y is shown in Equation (7).

[0043]

[0044] Attention along both the horizontal and vertical axes is applied simultaneously to the input tensor. Each element in both attention maps reflects the presence of the object of interest in the corresponding row and column. This encoding process allows coordinate attention to more accurately locate the precise position of the object of interest, thereby improving the model's detection accuracy.

[0045] In typical CNN network structures, the final classification layer is usually composed of fully connected layers. A key characteristic of fully connected layers is their fixed number of features. This means that the image input to the network must be of a fixed size. However, in reality, image sizes vary widely. If the image size doesn't meet the network's input requirements, it cannot be used for forward computation. Therefore, to obtain a fixed-size image, cropping or stretching is necessary, which can lead to image distortion and affect the final accuracy. Ideally, the network should maintain a consistent input image size to achieve maximum accuracy. Spatial Pyramid Pooling (SPP) addresses two main issues: effectively avoiding image distortion caused by cropping and scaling; and resolving the problem of repetitive feature extraction in convolutional neural networks, significantly improving the speed of candidate box generation and saving computational costs. The SPP module combines local and global features, which is why the largest pooling kernel size in the SPP module should be as close as possible to or equal to the size of the feature map to be pooled. After the feature map is fused with local and global features, its expressive power is enriched, which is beneficial for situations where the target size in the image to be detected varies greatly, especially for complex multi-target detection like YOLOv3, thus greatly improving the detection accuracy.

[0046] SPPF is a novel SPP structure proposed by the authors of YOLOv5. It uses a cascaded max-pooling structure with three 5×5 kernels. It fully utilizes the output of each pooling layer to obtain pooling outputs equivalent to three different kernel sizes. This approach reduces computational cost and increases computational speed.

[0047] B6. Combined with appendix Figure 2 The SPPF block is modified using the CSP block structure. A residual network structure is used, with Ghost Conv blocks of 1×1, 3×3, and 1×1 kernels sequentially added to the input of the SPPF structure, and a residual network branch at the very beginning of the input. After three cascaded soft pooling operations, the output passes through Ghost Conv blocks of 1×1 and 3×3 kernels, and is then fused with the branch to obtain the output. This structure reduces information loss during pooling of the SPPF block, thus preserving more semantic information and facilitating more accurate feature maps through multi-scale fusion in the Neck layer.

[0048] B7, combined with appendix Figure 3 This combines the C3 module in the backbone network with the Cov block in the Super-SPPF module into a GhostModule structure. Figure 3This illustrates the basic idea behind the GhostModule structure. It transforms convolution operations into half convolution and half linear operations. Finally, the feature maps generated by both are concatenated to obtain a complete feature map. This method reduces the number of model parameters by minimizing the use of convolution kernels, and the generation of redundant feature maps through linear transformations ensures the accuracy of the feature maps.

[0049] SoftPooling obtains activation weights through exponential weighting within a region, allowing larger activation values ​​to contribute more to the output. Furthermore, SoftPooling is differentiable, allowing for continuous gradient updates during backpropagation. Therefore, using SoftPooling for feature scaling effectively reduces information loss. SoftPooling assigns differentiable values ​​to each activation α within region R. i weight ω i The calculation formula is as follows:

[0050]

[0051] The final output of SoftPooling The transformation formula is obtained by performing a nonlinear operation on all activations and their corresponding weights within region R and then summing them.

[0052]

[0053] A3. Input the test set into the forest fire detection network and use the obtained pre-trained weights to obtain the detection results.

[0054] Combined with appendix Figure 4 This is the overall implementation flowchart. Combining the aforementioned target detection methods, the overall implementation process can be divided into the following steps:

[0055] First, input the augmented training set;

[0056] Secondly, join the CA Backbone network;

[0057] Secondly, replacing the SPPF block with the Super-SPPF block yields an improved YOLO v5 forest fire detection network;

[0058] Secondly, the Head layer performs predictions;

[0059] Secondly, the pre-trained model is obtained;

[0060] Finally, the network is evaluated using the test set.

[0061] Combined with appendix Figure 5 This is the overall network architecture diagram proposed in this invention.

[0062] Dataset

[0063] Research resources in the field of early forest fire detection are limited, and there is currently no complete public dataset of forest fire characteristics available for research. Therefore, this paper uses web crawlers to collect images of forest fires and manually selects them to obtain a self-built early forest fire dataset, L-SF. Based on this, this paper selects Pedro ViníciusAlmeida Borges de The team created the fire and smoke dataset D-Fire and further supplemented it with the hybrid dataset DL-Fire. The images in the D-Fire dataset are mainly collected from surveillance cameras and fire test videos of different resolutions. Since most of the images come from real surveillance footage, they can better describe various scenarios of real forest fires. The fire and smoke information in the D-Fire dataset mostly occurs at a distance and the targets are small; there are not many close-up fire and smoke images. The L-Fire dataset mainly consists of real close-up fire scenes collected from the internet, which can serve as a good complement to the D-Fire dataset. The images in the DL-Fire dataset have different resolutions, which can better adapt to cameras of different resolutions. The composition of the datasets used in this experiment is shown in Table 1. This paper uses the LabelImg software to label the datasets. Since the algorithms for the comparative experiments require VOC format datasets for training and validation, this paper uniformly obtains VOC format datasets and then converts the VOC format datasets to YOLO format through code. This paper divides the DL-Fire dataset into training, validation, and test sets in a 7:2:1 ratio for related experiments.

[0064] Table 1 Number of data sets

[0065]

[0066] Evaluation indicators

[0067] This invention uses mAP, the most commonly used metric in object detection, as its evaluation standard. Regarding its calculation method, precision and recall are defined as follows:

[0068]

[0069]

[0070] TP, FP, and FN represent true positives, false positives, and false negatives, respectively. AP (Average Precision) is the area under the Precision-Recall curve. mAP is the average of all classes. 0.5 This represents the mAP value when the IOU threshold is 0.5.0.5:0.95 This represents the average of 10 mAP values ​​obtained by setting the IOU threshold from 0.5 to 0.95 with a step size of 0.05.

[0071] Experimental Results and Analysis

[0072] To demonstrate the performance of the proposed GS-YOLOv5 lightweight forest fire detection algorithm, this paper compares the proposed method with mainstream object detection algorithms in the current field. The experimental results are shown in Table 2. In this experiment, the dataset without the training portion of the DL-Fire dataset was used as the test set to verify the basic performance of the model. Since using the test set for verification cannot reflect the performance of the detection algorithm in a real-world detection environment, this paper uses a 10-minute video to simulate a real-world application scenario to verify the real-time performance and accuracy of the algorithm. Experimental results show that the proposed detection method outperforms other algorithms in both detection accuracy and speed. Faster-RCNN, a two-stage object detection algorithm, achieved the worst mAP. 0.5 The result is that, due to the large proportion of distant fire images in the self-built dataset, even the smallest anchor is much larger than the small targets in the image, leading to missed or false detections. SDD, as the simplest single-stage object detection algorithm in terms of structure, has the second fastest detection speed, but its accuracy is inferior to other models. Among algorithms exceeding 60 FPS, the algorithm proposed in this paper outperforms existing algorithms in both model size and accuracy.

[0073] To illustrate the performance of the proposed GS-YOLOv5 lightweight forest fire detection algorithm, this paper compares the proposed method with mainstream detection algorithms in the current object detection field. The experimental results are shown in Table 2. In this experiment, the dataset without the training portion of the DL-Fire dataset was used as the test set to verify the basic performance of the model. Since using the test set for verification cannot reflect the performance of the detection algorithm in a real detection environment, this paper uses a 10-minute video to simulate a real-world application scenario to verify the real-time performance and accuracy of the algorithm. The experimental results show that the detection accuracy and speed of the proposed detection method are superior to other algorithms. Faster-RCNN, as a two-stage object detection algorithm, achieved the worst mAP50 result. This is because the self-built dataset contains a large proportion of distant fire images, causing even the smallest anchor to be much larger than small targets in the image, leading to missed or false detections. SDD, as the single-stage object detection algorithm with the simplest structure, has the second fastest detection speed, but its accuracy is inferior to other models. Among algorithms with over 60 FPS, the proposed algorithm outperforms existing algorithms in both model size and accuracy.

[0074] Table 2 compares the detection performance of GS-YOLOv5 with other models on the forest fire detection dataset.

[0075]

[0076] Based on the above experimental results, the Super-SPPF module in GS-YOLOv5 improves detection speed, retains more semantic information, and reduces the false positive rate. CABlock, by considering spatial and locational information, provides more accurate weights to the feature map, resulting in a higher detection accuracy. C3Ghost, thanks to its Ghost bottleneck structure, reduces the overall model's parameter count. In conclusion, GS-YOLOv5 can meet the current needs for early detection of forest fires.

[0077] ablation experiment

[0078] To verify the performance improvement effects of Super-SPPF, coordinate attention, and C3Ghost on YOLOv5, specific ablation experiments were conducted using a self-built DL-Fire dataset, and the results are shown in Table 3.

[0079] Table 3. Ablation test results.

[0080]

[0081]

[0082] As shown in Table 3, after adding the CA attention mechanism module, the model's accuracy improved by 0.3% compared to YOLOv5. (mAP50 and mAP...) 0.5:0.95 All improved by 0.6%. After replacing the C3Ghost block, accuracy did not decrease, but the number of parameters decreased significantly. Introducing Super-SPPF improved accuracy by 0.3%, and mAP... 0.5 With mAP 0.5:0.95 These improvements were 0.4% and 2.1% respectively. Further combining these three improvements in pairs leads to the following conclusions: The CA attention mechanism enhances expressive power and improves object detection accuracy by better capturing positional information and channel relationships. C3Ghost, benefiting from the Ghost bottleneck, reduces the number of parameters required by the network with almost no impact on accuracy. Super-SPPF better reduces feature loss during pooling, preserving more feature details, and its detection speed is faster than SPP structures. The use of group convolutions and GhostConv offsets the increase in parameters brought about by the CSP structure.

[0083] Table 4. Speed ​​Analysis of Super-SPPF

[0084]

[0085] The experiment compared the running speed and detection performance of Super-SPPF and the SPPCSPC structure proposed by YOLOv7. The experimental results are shown in Table 4. This experiment used the improved network structure as the standard structure, replacing SPPCSPC for comparative testing. A test set was used to test both network structures and verify their detection performance. 100 tensors were randomly generated and directly input into the SPPCSPC and Super-SPPF modules to observe their processing speed. The experimental results show that the Super-SPPF module is close to the SPPCSPC module in terms of detection accuracy improvement, and is much faster than the SPPCSPC block in processing speed. Moreover, Super-SPPF has fewer parameters. Therefore, the Super-SPPF block is more suitable for performing forest fire detection tasks.

[0086] This invention discloses a method for forest fire detection based on GS-YOLO v5. It addresses the issue that the limited learning and perceptual abilities of individual learners in complex forest image environments are insufficient for performing well on complex tasks, leading to high false positive rates. It also addresses the shortcomings of YOLO v5 in feature extraction and the inadequate information fusion capabilities of its internal PANet during training. The method reduces the false positive rate by training on a data-augmented dataset. CA is embedded in the original backbone network to obtain the relationship between space and channels. Through continuous network training, discriminative and non-discriminative features can be effectively distinguished, thereby improving the network's feature extraction capabilities. Features trained with an attention mechanism are then fed into Super-SPPF, allowing the network to retain more semantic information after soft pooling, enhancing the algorithm's ability to detect small targets and reducing the false positive rate. Furthermore, GhostConv and C3Ghost are used in GS-YOLOv5 to reduce the number of network parameters, thus meeting the computing power requirements of edge devices. Experimental results show that this algorithm effectively improves image detection accuracy and is suitable for long-term promotion and application.

[0087] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make more forms under the guidance of the present invention without departing from the spirit and scope of the claims, and these forms are all within the protection scope of the present invention.

Claims

1. A forest fire detection algorithm based on GS-YOLO v5, characterized in that, The main steps include: S1) Augment the obtained dataset and label it using the labelImg software; S2) Divide the total dataset into training set, validation set and test set in a ratio of 8:1:

1. Then input the images from all datasets into the GS-YOLOv5 network. S3) Replace the SPPF in the network with a Super-SPPF block, introduce a position attention mechanism in the third-to-last layer of the backbone network, and combine the C3 block with the GhostModule to obtain the C3Ghost block and replace the C3 block in the network. The Super-SPPF module is used to improve the network as follows: A branch is added to the original SPPF structure to form a residual network; Ghost Conv blocks with kernel sizes of 1×1, 3×3, and 1×1 are added sequentially at the input; Ghost Conv blocks with kernel sizes of 1×1 and 3×3 are added sequentially at the output of the serial pooling layer; Ghost Conv blocks with kernel sizes of 1×1 are added to the newly added branch; and the original parallel MaxPooling blocks are replaced with serial SoftPooling blocks. S4) The normalized training and test set images are input into the GS-YOLO v5 network for training.

2. The forest fire detection algorithm according to claim 1, characterized in that, In step S1, OpenCV is used to perform data augmentation on the image, simulating the shooting scene of the camera in the wild, thereby improving the robustness of the algorithm.

3. The forest fire detection algorithm according to claim 1, characterized in that... In step S3, a position attention mechanism is used to simultaneously acquire inter-channel relationships and long-distance position information, thereby improving detection accuracy while slightly increasing the number of parameters.

4. The forest fire detection algorithm according to claim 1, characterized in that... In S3, a Ghost Module is introduced into the convolutional layer of the C3 module, which increases the nonlinearity and generalization ability of the model, while greatly reducing the number of model parameters.

Images