Lightweight forest fire real-time detection method based on YOLOv7

Abstract

The invention discloses a lightweight forest fire real-time detection method based on YOLOv7, and belongs to the technical field of forest fire real-time detection based on deep learning. Comprising the following steps: S1, constructing an initial forest fire image data set, and carrying out data set segmentation and image data enhancement on the data set; s2, based on a traditional image processing method, constructing and extracting a binary image of a region where a fire may occur in the forest fire image; s3, constructing a GhostSE module and introducing a CA attention module to improve a YOLOv7 feature extraction network; and S4, training and evaluating the forest fire detection model by using the forest fire image data set, and storing the model training weight after the expected precision is achieved. According to the invention, the calculation amount and parameter amount of the model are significantly reduced through the GhostSE module for real-time detection, and the algorithm precision is improved by means of the CA attention module, so that the requirements of the precision and real-time performance of the forest fire detection model are balanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of real-time forest fire detection based on deep learning, and specifically relates to a lightweight real-time forest fire detection method based on YOLOv7. Background Art

[0002] Forest fires are natural disasters that have attracted much attention from the society. They bring huge damage and losses to forest ecosystems and humans. Flame detection technology is the key to forest fire prevention. Currently, there are three main types of flame detection technology: traditional detectors, traditional image processing technology, and target detection technology based on deep learning.

[0003] Traditional detectors detect flames through physical sensors, mainly light sensors, smoke sensors and temperature sensors, which detect infrared rays, smoke concentration and ambient temperature respectively to identify flames. However, in outdoor scenes, the false alarm rate is high and manual assistance is required. With the development of traditional image processing technology, it has gradually been applied to flame detection. It identifies flames by using the color, texture and dynamic features of the image, and color is the main feature of flame detection.

[0004] The target detection technology based on deep learning is currently an important one in the field of forest fire detection technology, which aims to identify specific flame targets in images. Forest fire detection algorithms based on deep learning are generally divided into one-stage algorithms and two-stage algorithms. The two-stage algorithm needs to generate candidate areas first, and then classify the candidate areas in two stages. Commonly used algorithms include R-CNN, Fast R-CNN and Faster R-CNN. Since two stages are required, the real-time performance of the two-stage algorithm is limited. In contrast, the one-stage algorithm has higher real-time performance. Among them, the YOLO series of algorithms occupy an important position in the field of forest fire detection technology. The YOLOv7 algorithm supports multi-scale detection and has improved detection speed compared to the two-stage algorithm, but the YOLOv7 algorithm still has the following shortcomings:

[0005] 1. Due to the large amount of convolution calculation in the ELAN module, the YOLOv7 model has many calculations and parameters, which makes it difficult to meet higher real-time performance requirements; 2. It is difficult to maintain a high accuracy rate in complex scenes, and there are problems of false detection and missed detection of small forest fire targets. Therefore, the present invention attempts to propose a new detection model and method from the aspects of balancing model accuracy and real-time performance. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the present invention proposes a lightweight real-time forest fire detection method based on YOLOv7, which can achieve the requirements of balancing the accuracy and real-time performance of the forest fire detection model.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A lightweight forest fire real-time detection method based on YOLOv7, comprising the following steps:

[0009] S1: Build the initial forest fire image dataset, segment the dataset and enhance the image data;

[0010] S2: Based on traditional image processing methods, a binary image of the area where fire may occur in forest fire images is constructed and extracted;

[0011] S3: Build the GhostSE module and introduce the CA attention module to improve the YOLOv7 feature extraction network to obtain the forest fire detection model;

[0012] S4: Use the forest fire image dataset to train and evaluate the forest fire detection model. After achieving the expected accuracy, save the model training weights.

[0013] S5: Apply the trained forest fire detection model to high-point fixed monitoring of forest fire detection.

[0014] As a preferred technical solution of the present invention, in S2, the steps of constructing a binary image of the area where fire may occur in the forest fire image based on the traditional image processing method include:

[0015] (1) obtaining a digital image to be processed;

[0016] (2) Convert the digital image to be processed from RGB to HSV color space;

[0017] (3) Using a specific threshold to create a mask and obtain a binary image, white indicates that there may be flames in the current area, that is, pixels within the threshold range, and other areas are filled with black; the specific thresholds include hue between 0 and 30, saturation between 120 and 255, and brightness between 120 and 255;

[0018] (4) For the binary image obtained in step (3), change the size to 640*640 and output it.

[0019] As a preferred technical solution of the present invention, in S3, the YOLOv7 network is used as the basic model, and the ELAN module is replaced by the GhosSE module based on GhostConv and SE attention mechanism. The GhostSE module can effectively and significantly reduce the model parameters and calculation amount through GhostConv to improve the detection speed; the SE attention mechanism effectively compensates for the accuracy loss caused by the simplification of the network structure; the GhostSE module can be regarded as a unit with low parameters and strong feature extraction ability, which can effectively balance the detection accuracy and real-time performance. The GhostSE module includes CBM structure, GhostConv structure, feature channel merging and SE attention mechanism. The CBM specific convolutional network structure includes 3*3 convolution, batch normalization and Mish activation function; the SE attention mechanism is introduced to improve the feature representation ability without increasing the amount of calculation. The CA (Coordinate Attention) attention module is introduced at the tail of the YOLOv7 feature extraction backbone network to focus on local position information, while improving the performance of the small target forest fire detection model without consuming too many resources.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0021] The present invention enhances the forest fire data set by adding noise and improving the generalization ability of the network model. Secondly, the traditional image processing method brings more feature information input to the deep learning network model, significantly improving the detection performance. GhosSE is used to reduce the number of parameters and calculations of the overall network for real-time detection, while the CA attention module is used to improve the accuracy of the algorithm, thereby balancing the accuracy and real-time requirements of the forest fire detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the workflow of the present invention.

[0023] Figure 2 A structural diagram of the improved YOLOv7 network model constructed for the present invention.

[0024] Figure 3 This is the structure diagram of the GhostSE module.

[0025] Figure 4 This is the structure diagram of the Ghost Module.

[0026] Figure 5 This is the SE attention structure diagram.

[0027] Figure 6 This is the CA attention structure diagram. DETAILED DESCRIPTION

[0028] In order to further clearly illustrate the implementation mode of the present application, the present invention is further described below in conjunction with the drawings in the present application.

[0029] See also Figure 1 As shown, the present invention proposes a lightweight forest fire real-time detection method based on YOLOv7, comprising the following steps:

[0030] S1: Build the initial forest fire image dataset, segment the dataset and enhance the image data;

[0031] S2: Based on traditional image processing methods, a binary image of the area where fire may occur in forest fire images is constructed and extracted;

[0032] S3: Build the GhostSE module and introduce the CA attention module to improve the YOLOv7 feature extraction network to obtain the forest fire detection model;

[0033] S4: Use the forest fire image dataset to train and evaluate the forest fire detection model. After achieving the expected accuracy, save the model training weights.

[0034] S5: Apply the trained forest fire detection model to high-point fixed monitoring of forest fire detection.

[0035] First, in step S1, the initial forest fire image dataset is labeled using software, and then divided into a training set and a test set at a ratio of 8:2. The initial forest fire training set is then processed using the data enhancement provided by YOLOv7: mosaic data enhancement (mosic), random cropping, and color jittering.

[0036] In step S2, a binary image of the possible fire area is obtained based on the traditional image processing method. The obtained output is merged with the original input forest fire image to form feature information with a size of 640*640 and 4 channels as the input of the improved YOLOv7 algorithm model. The binary image plays a similar role to the attention mechanism in the feature extraction process, and can focus on the area where the fire may occur. Assuming the initial RGB image X, the input size is R H×W×3 First, convert it into HSV color mode, and use hue, saturation and brightness control for each pixel; then, obtain the binary image by means of a specific threshold mask. Finally, resize the feature size to the feature scale of R 640×640×4 .

[0037] The specific steps are as follows: first, convert the processed digital image from RGB to HSV color space, and obtain binary feature information by creating a mask using a specific threshold. White indicates that there may be flames in the current area, that is, pixels within the threshold range, and other areas are filled with black. The specific thresholds include hue between 0 and 30, saturation between 120 and 255, and brightness between 120 and 255. The obtained binary feature information is resized to 640*640 size and merged with the original input RGB image into 640*640*3 feature information. It passes through the convolution downsampling module once to extract the feature information.

[0038] In step S3, the YOLOv7 network is used as the basic model, and the ELAN module is replaced by the GhosSE module based on GhostConv and SE attention mechanism. The GhostSE module can effectively and significantly reduce the model parameters and calculation amount through GhostConv, thereby improving the model detection speed. The SE attention mechanism can effectively compensate for the accuracy loss caused by the simplification of the network structure. The GhostSE module can be regarded as a unit with low parameters and strong feature extraction capabilities, which can effectively balance detection accuracy and real-time performance.

[0039] Feature extraction passes through the GhostSE module, which takes the output of the previous layer as input, such as Figure 3 As shown in the figure, the GhostSE module performs average channel splitting on the feature information of the previous layer, and enters two branches for feature extraction, one through a CBM-specific convolution structure and two GhostConv modules, and the other through a CBM-specific convolution structure. Finally, the feature information of the two branches is merged and output after passing through the SE attention module and the CBM-specific convolution structure. Among them, CBM is 3*3 convolution, batch normalization and Mish activation function.

[0040] Specifically, in the GhostSE module, if Figure 4 As shown in the figure, the first convolution in the GhostConv structure is a 1*1 convolution. Here, the convolution kernel size of 1*1 is used to reduce the number of feature information channels to half, reducing the amount of subsequent calculations. The feature map obtained in the previous step is then subjected to a DWConv with a convolution kernel size of 5*5. DWConv performs convolution on each feature channel separately, and finally stacks the obtained feature map to restore the number of channels to the same number as before entering GhostConv, and then outputs it.

[0041] Furthermore, in the GhostConv structure, assuming X∈R H×W×C , where C, H and W represent the input data channel, height and width respectively. First, use the conventional 1*1 convolution to get the feature map Y 1It can be expressed as: 1 =X*f, then Y 1 The number of channels is m; then Y 1 The feature map of each channel is represented by Φ ij Operation generates feature map Y i，j , the formula is:

[0042]

[0043] Where C = m + S, Φ ij It is the DWConv described above, which convolves the feature maps one by one to obtain Y i,j ; Finally, we will get Y i,j Features and Y 1 Perform channel stacking and then output.

[0044] Furthermore, the DWConv is composed of depthwise convolution and pointwise convolution. Assuming that the input feature scale is H×W and the number of channels is C in , use K×K convolution kernel for depth convolution, and the output channel size is C out The number of parameters using standard convolution is K×K×C in , the computational effort is H×W×K×K×C in ×C out , for DW convolution parameters, the amount is K×K×C in +C in ×C out , the amount of calculation is H×W(K×K×C in +C in ×C out ), compared with standard convolution, DWConv can significantly reduce the amount of calculation and parameters, especially when C in and C out When it is larger, the effect is more obvious, which effectively improves the model reasoning speed, but this sacrifices the detection accuracy.

[0045] Furthermore, in order to compensate for the loss of accuracy, the SE attention mechanism is introduced in the GhostSE module. Common attention mechanisms include ECA, CBAM and CA, but these methods are more complex in structure than the SE attention module, and the introduction of these attention modules will lead to a decrease in the algorithm reasoning speed. The present invention extracts location semantic information through the CA module, uses the location semantic information to focus on the location of interest, and enhances the network feature information representation capability. The small target forest fire detection capability is improved while consuming very little resources.

[0046] The SE attention module is as follows Figure 5 As shown in Figure 2, the main operations are squeezing and excitation. Assume a feature input χ∈R H×W×C, H, W and C represent the height, width and number of channels of feature information respectively. First, the feature map is globally averaged pooled to generate a 1×1×C vector Z c , the formula is:

[0047]

[0048] The second step is to generate a weight vector S through two layers of full connection, which is used to measure the degree of attention paid to each channel feature information in the χ feature information. The formula is:

[0049] S=σ(W2δ(W1Z c )) (3)

[0050] The vector Z obtained in the previous step is processed by two fully connected parameters W. c The third step is to use the weight vector S generated in the second step to assign weights to the feature χ and obtain the output χ out , the formula is:

[0051] χ out =Sχ (4)

[0052] That is, the feature information of each channel in the feature map χ is multiplied by the weight of the corresponding channel.

[0053] Specifically in this embodiment, in step S3, a CA (Coordinate Attention) attention module is introduced at the end of the feature extraction backbone network based on the YOLV7 network. The difference between forest fire target detection and other target detection tasks is that it has the characteristics of complex scenes and inconsistent scales of target objects, which brings great difficulty to the target detection task. Therefore, it is particularly important for the network model to pay attention to the location information of possible targets when extracting features. Figure 6 As shown in the figure, the CA (Coordinate attention) attention module takes into account the channel information and also pays attention to the directional space information. Assuming that the input feature map χ∈R C*H*W , c represents the number of feature channels, H represents the feature height, and W represents the feature width. For χ, pooling is performed in the W direction and the H direction respectively, and the formula is as follows:

[0054]

[0055] Secondly, the generated C*1*W feature map is converted to C*W*1 and the W-direction pooled feature map for concat operation, and then the 1*1 convolution is used to reduce the dimension to generate the feature map χ 1 ∈R C / r×(H+W)×1 . Then χ 1 Perform Split operation to generate f h ∈RC / r×H×1 and f w ∈R C / r×W×1 Use 1×1 convolution to increase the dimension and combine it with Sigmoid activation to get g h ∈R C×H×1 and g w ∈R C×1×W Finally, the output of the CA attention module is obtained by multiplying the original input feature vector. The formula is as follows:

[0056]

[0057] In the specific implementation process, Figure 2 As shown, the improved YOLOv7 algorithm is used to complete the detection of forest fires. The input feature map size of the improved YOLOv7 network model is 640*640*4, which includes the binary features obtained based on the traditional image processing method and the RGB image features obtained after data enhancement; then the backbone network feature extraction is performed, which successively passes through two CBM network modules, GhostSE module, maximum pooling, GhostSE module, maximum pooling, GhostSE module, maximum pooling and GhostSE module, and the output of each layer is used as the input of the next layer, and finally the CA attention module is introduced at the tail. The neck network selects the output of the C5, C7 and C10 layers in the backbone network as input. First, C10 is first subjected to CSPSPP to obtain the feature map F1, and then F1 is upsampled and concat operation is performed with C7 to obtain the feature map F2, and F2 is further upsampled and concat with C5 to obtain F3; similarly, F3 is downsampled and spliced ​​with F2 to obtain F4, and F4 is further upsampled and spliced ​​with F1 to obtain F5. Use F3, F4 and F5 as input for detecting the head.

[0058] After detecting the head, the input F3, F4 and F5 features are input into the CBM specific convolutional network module to obtain classification prediction, coordinate regression prediction and target confidence prediction. The target confidence loss and classification loss use BCEWithLogitsLoss (binary cross entropy loss with log), and the coordinate loss uses CIoU loss. The total loss is as follows:

[0059] Loss = L cls +L obj +L Ciou (8)

[0060] Start training with parameters of initial learning rate 0.01, cyclic learning rate 0.01, momentum 0.973, and weight decay coefficient 0.0005. Start training without pre-training, set the number of training iterations to 300, the batch size to 64, and the input image size to 640px*640px.

[0061] In order to verify the effectiveness of the method of the present invention, a comparative experiment was conducted on the same data set with the current mainstream algorithm. The evaluation indicators used accuracy P and mAP@0.5%, model weight size, and frames per second FPS as evaluation criteria. The results are shown in Table 1.

[0062] Table 1 Test results of different models

[0063]

[0064] The results in Table 1 show that the method of the present invention has a high mAP0.5 and a high real-time performance. The high mAP0.5 reflects the detection performance and model generalization ability of the method of the present invention, and further illustrates that the method of the present invention effectively balances forest fire detection accuracy and real-time performance.

[0065] The above contents are merely examples and explanations of the concept of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A lightweight real-time forest fire detection method based on YOLOv7, characterized in that: The following steps are involved: S1: Build the initial forest fire image dataset, segment the dataset and enhance the image data; S2: Based on traditional image processing methods, a binary image of the area where fire may occur in forest fire images is constructed and extracted; S3: Build the GhostSE module and introduce the CA attention module to improve the YOLOv7 feature extraction network to obtain the forest fire detection model; S4: Use the forest fire image dataset to train and evaluate the forest fire detection model. After achieving the expected accuracy, save the model training weights. S5: Apply the trained forest fire detection model to high-point fixed monitoring of forest fire detection.

2. The real-time forest fire detection method according to claim 1, characterized in that: In S2, the steps of constructing a binary image of the area where fire may occur in the forest fire image based on the traditional image processing method include: (1) obtaining a digital image to be processed; (2) Convert the digital image to be processed from RGB to HSV color space; (3) Using a specific threshold to create a mask and obtain a binary image, white indicates that there may be flames in the current area, that is, pixels within the threshold range, and other areas are filled with black; the specific thresholds include hue between 0 and 30, saturation between 120 and 255, and brightness between 120 and 255; (4) For the binary image obtained in step (3), change the size to 640*640 and output it.

3. The real-time forest fire detection method according to claim 1, characterized in that: In S3, the YOLOv7 network is used as the basic model, and the ELAN module is replaced by the GhosSE module based on GhostConv and SE attention mechanism. The GhostSE module can effectively and significantly reduce the model parameters and calculation amount through GhostConv to improve the detection speed; the SE attention mechanism effectively compensates for the accuracy loss caused by the simplification of the network structure; the GhostSE module can be regarded as a unit with low parameters and strong feature extraction capability, which can effectively balance the detection accuracy and real-time performance.

4. The real-time forest fire detection method according to claim 3, characterized in that: The GhostSE module described in S3 includes a CBM structure, a GhostConv structure, feature channel merging, and a SE attention mechanism.

5. The real-time forest fire detection method according to claim 3, characterized in that: In S3, the CBM-specific convolutional network structure includes 3*3 convolution, batch normalization and Mish activation function; the SE attention mechanism is introduced to improve the feature representation capability without increasing the amount of computation.

6. The real-time forest fire detection method according to claim 3, characterized in that: In S3, the CA (Coordinate Attention) attention module is introduced at the end of the YOLOv7 feature extraction backbone network to focus on local location information, which improves the performance of the small target forest fire detection model without consuming too many resources.

Images