An Improved Forest Fire Detection Method for YOLOv8

Abstract

This invention discloses an improved YOLOv8 forest fire detection method, belonging to the field of image detection technology. The method includes: preprocessing raw image data; extracting feature information of strip-shaped flames using a proposed serpentine Transformer module in the backbone network; capturing fine-grained scale information of the target using a hybrid deep convolutional network designed in the neck network; locating smoke boundaries using angle penalty in the SIoU loss of the head network; constructing an improved YOLOv8 forest fire detection model; and performing fire detection using the improved YOLOv8 forest fire detection model to complete the forest fire detection task. The improved YOLOv8 forest fire detection method proposed in this invention shows good performance in detection accuracy and localization effect, and has a certain degree of effectiveness.

Description

Technical Field

[0001] This invention belongs to the field of image detection technology, and in particular relates to an improved YOLOv8 method for forest fire detection. Background Technology

[0002] Forest fire detection is a significant research direction and technological challenge in the field of computer vision. Frequent forest fires constantly threaten human lives and property, and have become one of the world's eight major natural disasters. While forest rangers can reduce the probability of forest fires to some extent, the vastness of forests, rugged terrain, and overgrown vegetation make movement difficult for rangers. In the event of an emergency, evacuation is challenging, posing significant safety risks. Therefore, relying solely on manual forest fire patrols is unsuitable. With the rise of drones and mountain robots, target detection algorithms are gradually emerging as a new direction in forest fire detection. Drones and mountain robots typically use smoke sensors, flame sensors, and temperature sensors for forest fire detection. However, sensor detection is easily affected by environmental factors, resulting in low accuracy. Machine learning-based forest fire detection methods require powerful hardware, making them unsuitable for small devices like drones. However, using an improved YOLOv8-based forest fire detection method can greatly help in accurately locating the fire's position and intensity, significantly improving the performance of forest fire detection.

[0003] In recent years, scholars at home and abroad have conducted in-depth analysis and research on forest fire detection. Among them, the most famous and effective methods in the existing literature mainly include: 1. Multi-directional detection method based on value transformation attention mechanism module: In 2022, Hu Y, Zhan J, Zhou G, et al. Fast forest fire smoke detection using MVMNet[J]. Knowledge-Based Systems, 2022, 241:108219-108238. This method is based on the joint weighting strategy of horizontal and vertical directions, and constructs a value transformation attention mechanism module to extract the color and texture of smoke, thereby improving the accuracy of smoke detection. 2. A deep learning method for real-time smoke detection using spatiotemporal information of camera images: In 2022, Dewangan A, Pande Y, Braun HW, et al. FigLib & SmokeyNet: Dataset and deep learning model for real-time wildland fire smoke detection[J]. Remote Sensing, 2022, 14(4): 1007-1021. This method extracts the original image information using ResNet34, combines the temporal information in the current frame with the corresponding temporal information in the previous frame using LSTM, and finally performs image prediction using ViT, which reduces the response time and enhances the accuracy of smoke detection. 3. A multi-scale feature extraction method adapted to small target forest fire detection: In 2023, Zhang L, Wang M, Ding Y, et al. MS-FRCNN: A Multi-Scale Faster RCNN Model for Small Target Forest Fire Detection[J]. Forests, 2023, 14(3): 616-633. This method uses ResNet50 as the backbone network of Faster RCNN. The advantage of feature pyramid multi-scale feature extraction helps the model to obtain detailed feature information. In the region proposal network, the PAM attention module is used. Through parallel operation of channel attention and spatial attention, the influence of complex background in the image is reduced, and the semantic and location information of small target forest fires are focused. The detection accuracy of this method is improved by 5.7%.4. A forest fire detection model based on YOLOv5: In 2023, Lin J, Lin H, Wang FA semi-supervised method for real-time forest fire detection algorithm based on adaptive spatial feature fusion[J]. Forests, 2023, 14(2): 361-381. First, the Transformer encoder is combined with the convolutional neural network as a feature extraction network to enhance the extraction of global forest fire information. Second, the coordinate attention mechanism is integrated to not only obtain inter-channel information but also consider the orientation-related positional information, helping the model to better locate and identify forest fire targets. Finally, semi-supervised learning is used to save manual annotation work. This method can more accurately locate targets. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes an improved YOLOv8 forest fire detection method to resolve the issues present in the prior art.

[0005] To achieve the above objectives, this invention provides an improved forest fire detection method for YOLOv8, comprising:

[0006] Acquire raw image data, preprocess the raw image data, and obtain preprocessed image data;

[0007] An improved YOLOv8 forest fire detection model is constructed by using a snake-shaped Transformer module as the backbone network, a hybrid deep convolutional network as the neck network, and SIoU loss as the head network.

[0008] The preprocessed image is input into the improved YOLOv8 forest fire detection model for calculation to obtain the forest fire detection results.

[0009] Preferably, the process of obtaining the preprocessed image data includes:

[0010] Four original image data are randomly read, and a region is randomly selected as the center region for stitching.

[0011] The four original image data are randomly cropped, randomly scaled, and randomly arranged. Then, the four original image data are stitched together based on the central region to obtain the enhanced image data.

[0012] The enhanced image data is then subjected to Letterbox image data enhancement to obtain the preprocessed image data.

[0013] Preferably, the feature information of the ribbon flame is extracted based on the snake-shaped Transformer module as the backbone network;

[0014] The process of extracting the feature information of the ribbon-shaped flame includes:

[0015] Local features of the flame are extracted using the snake-shaped Transformer module and convolutional structure, while global features are extracted using the Transformer structure to generate a flame attention score.

[0016] The feature vector after serpentine convolution is obtained based on the flame attention score and the serpentine convolution module;

[0017] The feature vector after the serpentine convolution process is transformed into the feature information of a ribbon-like flame.

[0018] Preferably, the expression for generating the flame attention score is:

[0019] Score(Q,K) = Score tr (Q,K)+Score sc (Score tr (Q,K));

[0020] Score tr Score represents the flame attention score of the Transformer structure. sc Let Score(Q,K) represent the flame attention score of the serpentine convolution, where Q represents the query vector, K represents the key vector, and Score(Q,K) represents the flame attention score.

[0021] The expression for the flame attention score of the Transformer structure is:

[0022]

[0023] Where T denotes matrix transpose. This represents the square root of the dimension of the key vector.

[0024] Preferably, the hybrid deep convolutional network is used as the neck network to capture fine-grained scale information of the target flame;

[0025] The process of capturing fine-grained information about the target flame includes:

[0026] Flame features were extracted using the CBS module to obtain an H×W×C2 feature map;

[0027] The H×W×C2 feature map is segmented along the channel dimension using the Split operation to obtain two H×W×0.5C2 feature maps.

[0028] The target fine-grained scale information of the two H×W×0.5C2 feature maps is captured by N binary deep convolution modules to obtain feature maps of size H×W×0.5×N×C2.

[0029] The feature maps of the size H×W×0.5×N×C2 are stitched together to obtain the H×W×0.5(N+2)C2 feature map;

[0030] Based on the CBS module, flame features are extracted from the H×W×0.5(N+2)C2 feature map, and the H×W×C2 feature map is obtained again, thus completing the capture of fine-grained scale information of the target flame.

[0031] Preferably, SIoU loss is composed of angle penalty loss, distance loss, shape loss and IoU loss;

[0032] The angle penalty loss is used to measure the angle difference between the predicted bounding box and the ground truth bounding box;

[0033] The distance loss is used to measure the difference in distance between the predicted bounding box and the ground truth bounding box;

[0034] The shape loss is used to measure the difference in shape between the predicted bounding box and the ground truth bounding box;

[0035] The IoU loss is used to measure the difference in overlap area between the predicted bounding box and the ground truth bounding box.

[0036] Preferably, the expression for the angle penalty loss is:

[0037]

[0038]

[0039] Among them, c h α is the height distance between the center points of the predicted bounding box and the ground truth bounding box, and σ is the distance between the center points of the predicted bounding box and the ground truth bounding box. When α is 0 or π / 2, the angle penalty loss Λ is 0.

[0040] Preferably, the expression for the height distance between the center point of the predicted bounding box and the center point of the ground truth bounding box is:

[0041]

[0042] The expression for the distance between the center point of the predicted bounding box and the center point of the ground truth bounding box is:

[0043]

[0044] in, and Represented as the coordinates of the center point of the prediction box. and This represents the coordinates of the center point of the true bounding box.

[0045] Compared with the prior art, the present invention has the following advantages and technical effects:

[0046] This invention proposes an improved YOLOv8 forest fire detection method, which enhances both the performance and generalization ability of forest fire target detection. It also proposes an improved YOLOv8 forest fire detection model. Addressing the issue that traditional feature extraction methods, limited by their inherent characteristics, cannot focus on the features of elongated, ribbon-like flames during forest fires, leading to low detection model accuracy, this invention proposes a serpentine Transformer module in the backbone network. This module combines convolutional structures to extract local features with Transformer structures to extract global features, and utilizes convolutional operations to improve erroneous global feature associations within the Transformer structure. Furthermore, serpentine convolution is employed to focus on the pixel features of the winding, elongated flames, effectively improving the model's flame feature extraction capability. This invention further improves the forest fire detection model based on the improved YOLOv8 model. In forest fire detection models, to address the challenges of unclear boundaries between flames and smoke, and the diverse scales of flames and smoke in images due to the influence of drone aerial photography altitude and perspective, making localization difficult, a hybrid deep convolutional network is designed in the neck network to capture fine-grained scale information of the target. Two proposed hybrid deep convolutional modules are cascaded to obtain multi-scale target features, refining the perception capabilities of each layer of the feature pyramid and path aggregation network for targets of different scales. The MDC module splits feature channels through deep convolution, prompting different receptive field convolutional kernels to learn scale-sensitive features within their corresponding feature channels. This increases the receptive field while mitigating the speed loss caused by parallelism. Furthermore, to improve the model's ability to localize fire targets with irregular shapes and varying orientations, a SIoU loss function is introduced in the head network, utilizing angle penalties to more accurately locate smoke boundaries. This invention proposes an improved YOLOv8 forest fire detection method that can effectively complete forest fire detection tasks and improve detection accuracy. Attached Figure Description

[0047] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0048] Figure 1 This is a flowchart of an improved YOLOv8 forest fire detection method according to an embodiment of the present invention;

[0049] Figure 2 This is a structural diagram of the snake-shaped Transformer module in an embodiment of the present invention;

[0050] Figure 3 This is a structural diagram of the hybrid depthwise convolutional network in an embodiment of the present invention;

[0051] Figure 4 This is a structural diagram of the head network in an embodiment of the present invention;

[0052] Figure 5 This is a structural diagram of the improved YOLOv8 forest fire detection model in an embodiment of the present invention;

[0053] Figure 6 Figure 1 shows a performance comparison of an improved YOLOv8 forest fire detection model trained on the D-Fire dataset according to an embodiment of the present invention. Figure 2 shows the comparison results of precision on the D-Fire dataset; Figure 3 shows the comparison results of recall on the D-Fire dataset; Figure 4 shows the comparison results of mAP@0.5 on the D-Fire dataset; and Figure 5 shows the comparison results of mAP@0.5:0.95 on the D-Fire dataset.

[0054] Figure 7 Figure (a) shows the performance comparison of different hybrid depth convolutional networks on the D-Fire dataset in this embodiment of the invention; Figure (b) shows the comparison results of precision on the D-Fire dataset.

[0055] Figure 8 This is a comparison diagram of the training process of YOLOv8 models with different losses on the D-Fire dataset in this embodiment of the invention.

[0056] Figure 9 This is a comparison chart of detection results on the D-Fire dataset in this embodiment of the invention;

[0057] Figure 10 This is a comparison chart of detection results on the fire-dataset dataset in this embodiment of the invention;

[0058] Figure 11 This is a comparison chart of detection results on the Wildfire Smoke dataset in an embodiment of the present invention. Detailed Implementation

[0059] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0060] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0061] Example 1

[0062] like Figure 1 As shown, this embodiment provides an improved forest fire detection method for YOLOv8, including:

[0063] Forest fire image data is preprocessed, and a serpentine Transformer module is proposed in the backbone network. This module combines convolutional structures to extract local features with Transformer structures to extract global features. Convolutional operations are used to improve erroneous global feature associations in the Transformer structure. Furthermore, serpentine convolution is used to focus on the pixel features of winding and slender flames, effectively improving the model's flame feature extraction capability. In the neck network, a hybrid deep convolutional network is designed to capture fine-grained scale information of the target. Two proposed hybrid deep convolutional modules are cascaded to obtain multi-scale target features, refining the perception of different scale targets at each level of the feature pyramid and path aggregation network. The MDC module, through deep convolution, splits feature channels, prompting convolutional kernels with different receptive fields to learn scale-sensitive features within their corresponding feature channels. This increases the receptive field while mitigating the speed loss caused by parallelism. Building upon this, to improve the model's ability to locate irregularly shaped and oriented fire targets, a SIoU loss function is introduced into the head network, utilizing angle penalties to more accurately locate smoke boundaries. Publicly available forest fire datasets are used as input to the improved YOLOv8 forest fire detection model. Comparative experiments are conducted to evaluate the target detection performance, and ablation experiments are performed to verify the module's effectiveness, thus completing the research on an improved YOLOv8 forest fire detection method. The proposed improved YOLOv8 forest fire detection method exhibits good target detection performance and can be effectively applied in forest fire environments and general fire scenarios, demonstrating strong generalization ability.

[0064] like Figure 1 As shown in the figure, this embodiment discloses an improved YOLOv8 forest fire detection method, including: preprocessing forest fire image data; extracting strip flame features using a snake-shaped Transformer module; extracting multi-scale features of the target using a hybrid deep convolutional network; locating the target boundary using a head network; constructing an improved YOLOv8 forest fire detection model; and inputting the forest fire image data into the improved YOLOv8 forest fire detection model for feature extraction to generate forest fire detection results.

[0065] S1. Preprocessing forest fire image data

[0066] Mosaic image data enhancement is performed on the original image data. Four original image data are randomly read, and a region is randomly selected as the center region for stitching. The four images, which have been randomly cropped, scaled, and arranged, are stitched together according to the center region to obtain the preprocessed image.

[0067] The Mosaic image data enhancement is performed using Letterbox image data enhancement. The longer side of the image data is scaled to a specified size, and the shorter side is filled with black pixels to obtain the preprocessed image. The formula for Letterbox image data enhancement is as follows:

[0068]

[0069] Where d is the length to be filled on one side, w is the width of the initial image, h is the height of the initial image, w′ is the specified width after image scaling, and h′ is the specified height after image scaling. The improved YOLOv8 forest fire detection model is downsampled a total of 5 times, scaling the image features to 1 / 32 of the input image. Therefore, the size of the input image needs to be a multiple of 32. The total filling length is obtained by mod32 operation, and the filling length on one side is obtained by dividing by 2.

[0070] S2. The snake-shaped Transformer module extracts ribbon-like flame features.

[0071] To address the issue of elongated, slender flames spreading along mountain slopes during forest fires, traditional feature extraction methods are limited by their inability to focus on the features of these flames, leading to low accuracy in detection models. This paper proposes a serpentine Transformer module in the backbone network. By combining convolutional structures to extract local features with Transformer structures to extract global features, and utilizing convolutional operations to improve erroneous global feature associations within the Transformer structure, a serpentine convolution is further employed to focus on the pixel features of the winding, elongated flames, effectively enhancing the model's flame feature extraction capabilities. Figure 2 This is a structural diagram of the snake-shaped Transformer module.

[0072] The specific methods of the snake-shaped Transformer module are as follows:

[0073] The Transformer architecture calculates the flame attention score. It acquires global flame features using a Transformer module, with the input being a feature map whose size is reduced through average pooling layers to lower computational costs. Specifically, each pixel in the feature map is mapped to a high-dimensional embedding vector, treating each pixel as a token and representing its features with the embedding vector. Token embeddings are then performed. Since the snake-like Transformer algorithm itself does not contain positional information of the sequence, additional positional encodings are added to represent the positional information of each pixel. These positional encodings are fixed vectors whose values ​​vary depending on the pixel's position. Combining positional encoding with token embeddings helps the model understand different positional information within the sequence. The token-embedded input vectors are then normalized, transforming each input vector into three different vectors: query, key, and value. These vectors are obtained by multiplying the input vectors by different weight matrices. A multi-head attention mechanism is then applied, with residual connections used to obtain the result. This is followed by further normalization and fully connected network operations to improve the vector's representational power. Finally, residual connections are used to obtain the final output vector.

[0074] The multi-head attention mechanism divides the model's attention layer into multiple independent heads to calculate the attention weights for the flame. Specifically, each head maps the query vector, key vector, and value vector through a linear transformation. Then, the query vector and key vector are multiplied by a dot product, and the result is divided by the square root of the key vector's dimension to ensure gradient stability. Finally, the flame attention score for the Transformer structure is obtained. The formula for the flame attention score in the Transformer structure is as follows:

[0075]

[0076] Where Q represents the query vector and K represents the key vector. This represents the square root of the dimension of the key vector.

[0077] The attention mechanism in the Transformer module compares the attention between adjacent pixels. The relevance of each query-key pair is calculated independently, ignoring the association between other query-key pairs, leading to erroneous relevance. Due to noise, irrelevant information, and complex background elements in forest fire images, or inaccurate and incomplete flame feature extraction, local flame information in the image is not concentrated or clear, resulting in noisy and blurred flame attention weights, limiting the model's flame detection performance. To address these issues, the Snake Transformer algorithm employs an internal optimization mechanism within the attention mechanism. It assumes that adjacent pixels should have similar attention and refines the relevance by comparing the attention between adjacent pixels and seeking consistency among all relevant vectors. Specifically, the Snake Transformer inserts an internal optimization module composed of convolutions into the traditional attention mechanism to enhance the correct association of locally relevant query-key pairs and improve the erroneous association of distant, background-irrelevant query-key pairs. The flame attention score calculation formula of the Snake Transformer algorithm is as follows:

[0078] Score(Q,K) = Score tr (Q,K)+Score sc (Score tr (Q,K))

[0079] Among them, Score sc This represents the flame attention score of the internal optimization module.

[0080] A serpentine convolutional structure is used to calculate the flame attention score. Flames in forest fire images often have long, thin, and complex shapes. Ordinary convolutions and dilated convolutions cannot adjust the region of interest based on flame features, resulting in poor optimization of local pixel associations. Although deformable convolutions can adaptively learn the region of interest based on flame features, their feature-learning ability is limited when dealing with long, thin flames. Therefore, the internal optimization module of the serpentine Transformer algorithm uses serpentine convolution. Serpentine convolution can adaptively focus on long, winding local structures, accurately capturing flame features. The 2D convolution kernel coordinates are CK, and the center coordinates are CK. i =(x i ,y i A convolutional kernel CK with an expansion ratio of 1 and a size of 3×3 is represented as follows:

[0081] CK={(x-1,y-1),(x-1,y),…,(x+1,y+1)}

[0082] The serpentine convolution extends the 2D convolution kernel along both the x-axis and y-axis. Taking the x-axis as an example, the specific position of each grid cell is denoted as CK. i±c =(x i±c ,yi±c ), where c = {0, 1, 2, 3, 4} represents the horizontal distance from the central grid. Each grid position CK in the convolution kernel... i±c The selection is a cumulative process; keeping the center position constant, the other positions swing, starting from the center position CK. i Initially, the position away from the center grid depends on the position of the previous grid, CK. i+1 Compared to CK i An offset Δ = {δ|δ∈[-1,1]} is added. By accumulating these offsets, the linear morphological structure of the convolution kernel is ensured. The formula for calculating the grid position of the convolution kernel along the x-axis is as follows:

[0083]

[0084] The formula for calculating the grid position of the convolution kernel along the y-axis is as follows:

[0085]

[0086] Since the Δ offset is in decimal form and the coordinates are in integer form, bilinear interpolation is used:

[0087] CK=∑ CK′ B(CK′,CK)·CK′

[0088] Where CK represents CK in the above formula. i±c With CK j±c In the fractional part, CK′ represents enumerating the entire integer space, and B represents the two-dimensional bilinear interpolation kernel, which is decomposed into two one-dimensional kernels, as shown in the following formula:

[0089] B(CK,CK′)=b(CK x CK x ′)·b(CK y CK y ′)

[0090] The feature map obtained by bilinear interpolation sampling along the x-axis has a size of [B,C,K*H,W]. After passing through a convolution kernel of size K×1 and the ReLU activation function, the output feature map in the x-axis direction has a size of [B,C,H,W].

[0091] The serpentine convolution will improve the flame attention score of the Transformer module. tr Taking (Q,K) as input, the three flame feature maps, after being processed by convolutional kernels along the x-axis, y-axis, and a 3×3 kernel, are concatenated. Then, a regular convolution operation is performed, and finally, a fully connected layer maps the serpentine convolutional flame feature map to a flame attention score. The formula for calculating the flame attention score using serpentine convolution is as follows:

[0092] Score sc (x) = Linear(Conv(Concat(Conv)) y (x),Conv(x),Conv x (x))))

[0093] Here, Linear(·) represents a fully connected layer, and Concat(·) represents concatenating tensors according to the channel dimension.

[0094] The attention weights in the snake-like Transformer algorithm are normalized using the softmax activation function, then multiplied by the attention weights, and finally summed using a weighted average to obtain the final output. The multi-head attention calculation formula for the snake-like Transformer algorithm is as follows:

[0095] head i =Attention(Q,K,V)=softmax(Score(Q,K))V

[0096] MultiHead(Q,K,V)=Concat(head1,…,head h )

[0097] Here, the value of h is 8, indicating the use of 8 attention heads. The snake-like Transformer algorithm normalizes the output vector layer of the Transformer module, modifies its size to [B,C,H,W], performs bilinear interpolation, and adds it to the input feature map to obtain the final result.

[0098] S3. Hybrid deep convolutional networks extract multi-scale features of targets.

[0099] Because forest fire images contain flames and smoke of varying scales, a hybrid deep convolutional network is proposed. This network utilizes a binary mixed deep convolution (BMDC) module to capture fine-grained scale information of the target. Figure 3 This is a structural diagram of a hybrid deep convolutional network.

[0100] The BMDC module employs two MDC modules connected in series to progressively extract more complex features from flames and smoke. The MDC module primarily consists of a CBS module and a depthwise convolution (DC) module. The specific process is as follows: the input feature map has dimensions [B, C1, H, W]. CBS downsampling is performed to obtain a feature map of dimensions [B, C2 / 4, H, W]. Four DC modules with different kernel sizes (3×3, 5×5, 7×7, 9×9) are connected in parallel to extract flame and smoke features in each channel. Finally, these modules are concatenated to obtain an output feature map of dimensions [B, C2, H, W]. By mixing depthwise convolution modules of different sizes, the MDC module enhances the model's ability to perceive targets at different scales. Given the speed loss caused by parallel connections, using depthwise convolution modules reduces computation and improves the model's operational efficiency.

[0101] S4. Head network locates target boundary.

[0102] The CIoU loss function, a traditional loss function for object detection, relies on bounding box regression metrics such as the overlap between the predicted and ground truth boxes, distance, and aspect ratio. It fails to consider the orientation mismatch between the predicted and ground truth boxes, leading to slow convergence during training and the predicted boxes "wandering" during training, making it difficult to locate smoke boundaries. The SIoU loss function redefines the penalty loss, addressing the issues of irregular flame and smoke shapes and variable orientations. The SIoU loss function mainly consists of four loss components: angle penalty loss, distance loss, shape loss, and IoU loss. Figure 4 This is a structural diagram of the head network.

[0103] The angle penalty loss is the core component of the SIoU loss function. By comparing the angle difference between the predicted box and the ground truth box, the predicted box can quickly regress to be on the same horizontal or vertical line as the ground truth box, accelerating the convergence process. The formula for the angle penalty loss is as follows:

[0104]

[0105]

[0106] Among them, c h α is the height distance between the center points of the predicted bounding box and the ground truth bounding box, and σ is the distance between the center points of the predicted bounding box and the ground truth bounding box. When α is 0 or π / 2, the angle penalty loss Λ is 0. h The formula for calculating σ is as follows:

[0107]

[0108]

[0109] in, and Represented as the coordinates of the center point of the prediction box. and This represents the center point coordinates of the ground truth bounding box. Distance loss measures the difference in distance between the predicted bounding box and the ground truth bounding box; the distance loss formula is as follows:

[0110]

[0111] γ=2-Λ

[0112] Specifically, when α approaches 0, the contribution of distance loss decreases significantly; when α approaches π / 4, the contribution of distance loss increases significantly. x With ρ y The formula is as follows:

[0113]

[0114]

[0115] Among them, c w With c h Let be the width and height of the minimum bounding rectangles of the predicted bounding box and the ground truth bounding box, respectively. Shape loss measures the difference in shape between the predicted and ground truth bounding boxes, converging from the length and width sides to achieve overall shape convergence. The shape loss formula is as follows:

[0116]

[0117] Where θ represents the degree of attention paid to shape loss, and the value of θ is set to 4, ω x With ω h The formula is as follows:

[0118]

[0119]

[0120] Where w and h are the width and height of the predicted bounding box, respectively. gt with h gt Here, represents the width and height of the ground truth bounding box, respectively. The IoU loss measures the difference in overlap area between the predicted and ground truth bounding boxes. The IoU loss formula is as follows:

[0121]

[0122] Among them, B and B GT Let represent the predicted bounding box and the ground truth bounding box, respectively. The SIoU loss function introduces the concept of angle loss, allowing the predicted bounding box to better align with the ground truth bounding box during regression, thus improving the speed of model training and the accuracy of localization. The SIoU loss function formula is as follows:

[0123]

[0124] The formula for the Binary Cross-Entropy Loss (BCE) function is as follows:

[0125]

[0126] Where N represents the total number of samples, y i Let y represent the category to which the i-th sample belongs. pred Let be the probability that the model predicts the i-th sample, and log(·) be the logarithmic function. The model's predicted value y pred As close as possible to the true label y i When y i When the value of is 1, the expected value of y is... pred When y approaches 1 i When the value of is 0, the expected value of y is... pred Close to 0.

[0127] Image labels are discrete points. Treating the labels as absolutely correct targets results in a Dirac distribution, with a probability density represented by a sharp vertical line. However, forest fire images suffer from occlusion, shadows, and blurring, making the boundary between flames and smoke unclear. Therefore, learning a distribution with a wider range is more reasonable. The Distribution Focal Loss (DFL) loss function allows the network to quickly focus on values ​​near the flame and smoke labels, maximizing the probability density at those labels. The cross-entropy function optimizes the probabilities at the two positions to the left and right of the label y, focusing the network distribution on the vicinity of the label value. The DFL loss function formula is as follows:

[0128] DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 ));

[0129] Among them, S i With S i+1 The network output is converted into probability values, y, through softmax. i With y i+1 These are the two points closest to y, where y is the label value. and When the global minimum value is obtained, the regression coordinate is closest to y.

[0130] S5. Construct an improved YOLOv8 forest fire detection model

[0131] The improved forest fire detection method for YOLOv8 proposed in this invention aims to accurately detect flames and smoke in forest fire scenarios. Figure 5 This is a structural diagram of the improved YOLOv8 forest fire detection model.

[0132] The CBS module combines three components to form a single operational unit of convolution, normalization, and SiLU activation, used to extract flame features and perform nonlinear transformations. The C2f module mainly consists of the CBS module and the Bottleneck module. When the shortcut in the Bottleneck module is set to True, it utilizes residual connections. Residual connections alleviate the vanishing and exploding gradient problems by directly adding the input and output flame features, and allow information transfer between layers, avoiding deep network degradation. The SPPF module mainly consists of the CBS module and max pooling. All max pooling kernels are 5×5, employing a combination of serial and parallel methods to concatenate the results before and after pooling, improving the model's computational speed.

[0133] The publicly available forest fire dataset was used as input to the improved YOLOv8 forest fire detection model. Comparative experiments were conducted to evaluate the target detection effect, and ablation experiments were conducted to verify the effectiveness of the module. This completed the research on an improved YOLOv8 forest fire detection method.

[0134] Precision, recall, average precision (AP), and mean average precision (mAP) are used as evaluation metrics.

[0135] Precision is the proportion of true positive samples out of all samples predicted as positive. It is calculated as follows:

[0136]

[0137] Where TP is the number of positive samples predicted as positive by the model, and FP is the number of negative samples predicted as positive by the model.

[0138] Recall is the proportion of all true positive samples that are predicted as positive, and the formula is as follows:

[0139]

[0140] Where TP is the number of positive samples predicted as positive by the model, and FN is the number of positive samples predicted as negative by the model.

[0141] The PR curve, also known as the precision-recall curve, has recall on the x-axis and precision on the y-axis. Mean precision is the area under the PR curve. The formula is as follows:

[0142]

[0143] In object detection, datasets typically contain multiple classes. AP (Average Precision) is often used to measure a model's detection performance for a specific class of objects. To evaluate the detection performance across multiple classes, the AP values ​​for each class are summed and averaged to obtain the mean accuracy. mAP (Maximum Average Precision) is a more comprehensive evaluation metric that reflects the overall performance of the model in multi-class object detection tasks. The formula is as follows:

[0144]

[0145] The IoU threshold affects the model's mAP value. Commonly used mAP metrics are mAP@0.5 and mAP@0.5:0.95. mAP@0.5 represents the average precision across all classes when the IoU threshold is 0.5. mAP@0.5:0.95 is the averaging result of IoU thresholds from 0.5 to 0.95 with a step size of 0.05. The formula is as follows:

[0146]

[0147] This invention uses the D-Fire dataset for model learning and performance validation. This dataset contains forest fire image data for both flames and smoke. The invention will also test the fire-dataset and Wildfire Smoke datasets to verify the model's generalization performance. The fire-dataset dataset only contains flame targets, and most images are of indoor fires and car fires; furthermore, the dataset has a limited number of images, making it unsuitable for training. The Wildfire Smoke dataset only contains smoke targets, and also has a limited number of images and a single category, making it unsuitable for training. The ratio of D-Fire training data, test data, and validation data is 8:1:1. The fire-dataset and Wildfire Smoke datasets are used only for testing, with 500 training epochs. To demonstrate the performance gain in forest fire detection brought about by the improvements in this invention, and to verify the model's robustness, this experiment will conduct ablation experiments on the D-Fire dataset to test the improved model performance. Table 1 presents the results of testing on the D-Fire, fire-dataset, and Wildfire Smoke datasets, comparing the precision, recall, mean precision, and mean mean precision before and after the improvement of the original, unmodified YOLOv8 model. In the descriptions of this experiment and the figures below, ST-YOLOv8 (Snake Transformer - You Only Look Once version 8) represents adding a snake-shaped Transformer module to the backbone network of the baseline model; MDC-YOLOv8 (Mixed Depth-wise Convolution - You Only Look Once version 8) represents adding a mixed deep convolutional network to the neck network and an SIoU loss function to the head network of the baseline model; and SMDC-YOLOv8 (Snake Transformer with Mixed Depth-wise Convolution - You Only Look Once version 8) refers to the improved YOLOv8 forest fire detection model. "-" indicates that the model does not exist.

[0148] Table 1

[0149]

[0150] As shown in Table 1, on the D-Fire dataset, compared to the previous model, the ST-YOLOv8 model achieved a 0.3% improvement in accuracy, a 1.3% improvement in mAP@0.5, and a 1% improvement in average accuracy for smoke. In particular, the average accuracy for flame improved by 1.7%, and the mAP@0.5:0.95 ratio improved by 1.7%, indicating a significant improvement in detection performance metrics. Compared to the previous model, the MDC-YOLOv8 model achieved a 1.4% improvement in accuracy, a 0.6% improvement in mAP@0.5, a 0.9% improvement in average accuracy for smoke, a 0.3% improvement in average accuracy for flame, and a 0.6% improvement in mAP@0.5:0.95 ratio, demonstrating improvements in detection performance metrics across the board. Combining the two models, the SMDC-YOLOv8 model, compared to the previous model, achieved a 1% improvement in accuracy, a 1.5% improvement in mAP@0.5, a 1.1% improvement in average accuracy for smoke, and a particularly significant 1.9% improvement in average accuracy for flames, with a 2% improvement in mAP@0.5:0.95. The performance improvement on the fire-dataset dataset was similar to that on the D-Fire dataset, with the SMDC-YOLOv8 model achieving a 10.4% improvement in accuracy compared to the previous model. Since the dataset only contains one flame class, both mAP@0.5 and average accuracy for flames improved by 3.2%, and the mAP@0.5:0.95 improved by 2.2%, demonstrating the excellent performance of the SMDC-YOLOv8 model in flame detection and its versatility in flame detection. In Wildfire... The performance improvement on the Smoke dataset is similar to that on the D-Fire dataset. Compared to the previous model, the SMDC-YOLOv8 model shows a 15.1% improvement in recall. Since the dataset only contains one smoke class, both the mAP@0.5 value and the average precision for smoke are improved by 9.8%, and the mAP@0.5:0.95 ratio is improved by 6.6%, demonstrating the excellent performance of the SMDC-YOLOv8 model in smoke detection and its general applicability to smoke detection. In summary, the performance of the SMDC-YOLOv8 model on the D-Fire, fire-dataset, and Wildfire Smoke datasets proves the effectiveness and robustness of the improved YOLOv8 forest fire detection method in this invention, demonstrating that the improved YOLOv8 forest fire detection method has better detection performance. Figure 6Figure 1 shows a performance comparison of an improved YOLOv8 forest fire detection model trained on the D-Fire dataset in an embodiment of the present invention. Figures 1(a) and 2(b) show the comparison results of precision and recall on the D-Fire dataset, respectively, and Figures 3(c) and 4(d) show the comparison results of mAP@0.5 and mAP@0.5:0.95 on the D-Fire dataset, respectively.

[0151] To verify the impact of the proposed snake-shaped Transformer module on model detection performance, this experiment will compare it with different types of modules on the D-Fire dataset, including traditional ordinary convolutional modules, mainstream Transformer modules, and snake-shaped convolutional modules with special structures. Table 2 shows the comparison results of model detection performance when different modules with the same position as the snake-shaped Transformer module are added to the backbone network.

[0152] Table 2

[0153]

[0154] As shown in Table 2, both the ordinary convolutional module and the Transformer module improved the mAP@0.5 value by 0.6%. The ordinary convolutional module excels at extracting local features of flames in images, while the Transformer module extracts global features of flames through a self-attention mechanism. Although their focus differs, both models show improved detection performance. The serpentine convolutional module improved the mAP@0.5 value by 0.8%. This module extracts local features of elongated flames, further improving detection performance. The ST-YOLOv8 model improved the mAP@0.5 value by 1.3%, and the average accuracy for both flames and smoke also improved. This is because the serpentine Transformer module uses convolutional modules to reduce erroneous global associations in the Transformer structure and utilizes serpentine convolution to extract features of elongated flames, allowing the model to learn more features and achieve better detection results.

[0155] To verify the impact of the hybrid depthwise convolutional network designed in this invention on the model's detection performance, this experiment will compare it with different types of hybrid depthwise convolutional networks on the D-Fire dataset. Table 3 shows the comparative experimental results of different hybrid depthwise convolutional networks. "+shuffle" refers to the MDC module using a shuffling structure to group data according to the number of channels before concatenating them to obtain the final result. "+Concat" means the MDC module still uses a concatenation structure. "+MDC" means replacing the downsampling CBS module in the neck network with the MDC module. Figure 7Table 3 shows a performance comparison of different hybrid depth convolutional network models on the D-Fire dataset. Analysis of the data in Table 3 reveals that MDCNet-1 improved accuracy by 1.6% and flame accuracy by 1.4%. MDCNet-2 improved the mAP@0.5:0.95 for smoke by 0.3%, and MDCNet-3 improved the mAP@0.5:0.95 for flame by 0.6%. In contrast, the MDCNet-4 module improved the mAP@0.5 by 0.2% and the mAP@0.5:0.95 by 0.1%, with average accuracy for smoke improving by 0.1% and average accuracy for flame by 0.2%. Considering all metrics, MDCNet-4 was ultimately chosen as the hybrid depth convolutional network, demonstrating that adding a hybrid depth convolutional network improves the detection performance.

[0156] Table 3

[0157]

[0158] To verify the impact of the SIoU loss function introduced in the head network of this invention on model performance, this experiment will compare it with different loss functions on the D-Fire dataset. Table 4 shows the comparative experimental results of different loss functions. Here, "EIoU" refers to the EIoU loss function introduced in the head network, and "WIoU" refers to the WIoU loss function introduced in the head network.

[0159] Table 4

[0160]

[0161] Analysis of the experimental data in Table 4 shows that the model's accuracy improved by 1.3% after using the EIoU loss function. The mAP@0.5:0.95 value for flames improved by 0.2% after using the WIoU loss function. The SIoU loss function improved accuracy by 1.5%, smoke accuracy by 1.5%, average smoke accuracy by 1.2%, and flame accuracy by 1.4%, with mAP@0.5 increasing by 0.5% and mAP@0.5:0.95 by 0.8%. This indicates that the SIoU loss function is significantly superior to other loss functions in improving detection performance, and therefore, the SIoU loss function was chosen as part of the regression loss for the SMDC-YOLOv8 model. Figure 8 This is a comparison chart of the training process of YOLOv8 models with different losses on the D-Fire dataset.

[0162] To demonstrate that the SMDC-YOLOv8 model proposed in this invention has better detection performance than other object detection methods, this paper conducts comparative experiments on the D-Fire dataset, the fire-dataset dataset, and the Wildfire Smoke dataset, respectively. Table 5 shows the comparison results of the SMDC-YOLOv8 model's detection performance with other state-of-the-art models on the D-Fire dataset; Table 6 shows the comparison results of the SMDC-YOLOv8 model's detection performance with other state-of-the-art models on the fire-dataset dataset; and Table 7 shows the comparison results of the SMDC-YOLOv8 model's detection performance with other state-of-the-art models on the Wildfire Smoke dataset.

[0163] Table 5

[0164]

[0165] Table 6

[0166]

[0167] Table 7

[0168]

[0169] Observing the data in Tables 5, 6, and 7, it can be seen that the experimental model demonstrates competitive performance on the D-Fire dataset, the fire-dataset dataset, and the Wildfire Smoke dataset. Overall, the model achieves good average accuracy for smoke and flame on the D-Fire dataset, reaching 85.4% and 73.4%, respectively. This indicates that the model can accurately predict and detect forest fire targets. Furthermore, the model also achieves good average accuracy (mAP@0.5) and average accuracy (mAP@0.5:0.95) on the D-Fire dataset, at 79.4% and 47.6%, respectively. This demonstrates the model's good performance in multi-class target detection tasks. The above analysis proves that the model can better detect forest fire targets at the pixel level and more accurately capture target feature information.

[0170] like Figure 9As shown, the YOLOv8s model's detection results show numerous overlapping boxes, indicating redundancy and uncertainty in flame and smoke detection. In this embodiment, the SMDC-YOLOv8 model's detection results do not generate redundant prediction boxes, effectively reducing duplicate detections and improving both the accuracy and reliability of flame and smoke detection. The YOLOv8s model's detection results show significant missed detections, with flame targets being overlooked and not correctly detected. The SMDC-YOLOv8 model's detection results show no missed detections, accurately distinguishing and detecting flames. While the YOLOv8s model can detect targets, the predicted positions in the image are inaccurate. In the SMDC-YOLOv8 model's detection results, the predicted box positions better match the actual target positions; the smoke prediction box is closer to the target, accurately capturing smoke features; and the flame detection box is closer to the target, precisely extracting flame features, demonstrating the SMDC-YOLOv8 model's improved performance in target location.

[0171] like Figure 10 As shown, the YOLOv8 model exhibits overlapping predicted boxes, missed detections, and false detections in its detection results. The SMDC-YOLOv8 model, however, shows no overlapping predicted boxes. It successfully detected flames and accurately identified smoke, while also avoiding the error of misidentifying gray text boxes as smoke. This demonstrates the accuracy and stability of the SMDC-YOLOv8 model, showcasing excellent detection performance on the fire-dataset dataset.

[0172] like Figure 11 As shown, the YOLOv8 model exhibits significant missed detections, failing to capture smoke. In contrast, the SMDC-YOLOv8 model shows no overlapping bounding boxes. Even when near-objects like clouds are present, the model still detects smoke, and the predicted bounding boxes are closer to the smoke outline, accurately defining the smoke's extent. This demonstrates the superiority of the SMDC-YOLOv8 model, showcasing excellent detection performance on the Wildfire Smoke dataset.

[0173] In summary, the improved YOLOv8 forest fire detection method proposed in this invention can effectively complete the forest fire detection task and has good detection accuracy and generalization.

[0174] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An improved forest fire detection method of YOLOv8, characterized in that, Includes the following steps: Acquire raw image data, preprocess the raw image data, and obtain preprocessed image data; An improved YOLOv8 forest fire detection model is constructed by using a snake-shaped Transformer module as the backbone network, a hybrid deep convolutional network as the neck network, and SIoU loss as the head network. The preprocessed image is input into the improved YOLOv8 forest fire detection model for calculation to obtain the forest fire detection results; The snake-shaped Transformer module is used as the backbone network to extract the feature information of the ribbon flame; The process of extracting the feature information of the ribbon-shaped flame includes: Local features of the flame are extracted using the snake-shaped Transformer module and convolutional structure, while global features are extracted using the Transformer structure to generate a flame attention score. The feature vector after serpentine convolution is obtained based on the flame attention score and the serpentine convolution module; The feature vector after the snake-shaped convolution is transformed into the feature information of a ribbon-like flame. The expression for generating the flame attention score is: ； The flame attention score represents the value of the Transformer structure. The flame attention score represents the serpentine convolution. Represents the query vector. Represents the key vector. Indicates the flame attention score; The expression for the flame attention score of the Transformer structure is: ； Where T denotes matrix transpose. This represents the square root of the dimension of the key vector.

2. The improved YOLOv8 forest fire detection method according to claim 1, characterized in that, The process of obtaining the preprocessed image data includes: Four original image data are randomly read, and a region is randomly selected as the center region for stitching. The four original image data are randomly cropped, randomly scaled, and randomly arranged. Then, the four original image data are stitched together based on the central region to obtain the enhanced image data. The enhanced image data is then subjected to Letterbox image data enhancement to obtain the preprocessed image data.

3. The improved YOLOv8 forest fire detection method according to claim 1, characterized in that, The method involves using a hybrid deep convolutional network as the neck network to capture fine-grained scale information of the target flame. The process of capturing fine-grained information about the target flame includes: Flame features were extracted using the CBS module to obtain Feature map; Based on the Split operation, the Segmentation is performed along the channel dimension of the feature map to obtain two... Feature map; Based on N binary depthwise convolutional modules, the two... Capture fine-grained scale information of the target from the feature map to obtain Feature diagram of dimensions; Regarding the The feature maps of the dimensions are stitched together to obtain Feature map; Based on the CBS module, the above Flame features are extracted from the feature map and re-obtained Feature maps are used to capture fine-grained scale information about the target flame.

4. The improved YOLOv8 forest fire detection method according to claim 1, characterized in that, SIoU loss is composed of angle penalty loss, distance loss, shape loss and IoU loss; The angle penalty loss is used to measure the angle difference between the predicted bounding box and the ground truth bounding box; The distance loss is used to measure the difference in distance between the predicted bounding box and the ground truth bounding box; The shape loss is used to measure the difference in shape between the predicted bounding box and the ground truth bounding box; The IoU loss is used to measure the difference in overlap area between the predicted bounding box and the ground truth bounding box.

5. The improved YOLOv8 forest fire detection method according to claim 4, characterized in that, The expression for the angle penalty loss is: ； ； in, The height distance between the center point of the predicted bounding box and the center point of the ground truth bounding box. The distance between the center point of the predicted bounding box and the center point of the ground truth bounding box, when 0 or At that time, angle penalty loss It is 0.

6. The improved YOLOv8 forest fire detection method according to claim 5, characterized in that, The expression for the height distance between the center point of the predicted bounding box and the center point of the ground truth bounding box is: ； The expression for the distance between the center point of the predicted bounding box and the center point of the ground truth bounding box is: ； in, and Represented as the coordinates of the center point of the prediction box. and This represents the coordinates of the center point of the true bounding box.

Images