Fire detection method and device based on YOLO-Unet

Abstract

The invention relates to the technical field of image processing, and discloses a fire detection method and device based on YOLO-Unet, and the method comprises the steps: obtaining a sample image set; enhancing the sample image set to obtain an enhanced image set; training an initial fire detection model based on the enhanced image set to obtain a target fire detection model, the initial fire detection model including a YOLOv11 model and a U-net model; acquiring a to-be-detected image acquired by the unmanned aerial vehicle; and detecting the to-be-detected image by using the target fire detection model to obtain a fire detection result. According to the method, the sample image set is acquired to help the model to understand the fire features, the fire detection model is composed of the YOLOv11 model and the U-net model, detection and segmentation integration is realized, the fire area can be quickly positioned, the accurate shape can be provided, the fire detection accuracy is improved, and the fire detection efficiency is improved. And an important basis is provided for subsequent fire situation analysis, emergency response and prevention and control decision.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a fire detection method and device based on YOLO-Unet. Background Art

[0002] Fire detection is crucial for ecological protection and disaster prevention. Currently, deep learning has been applied to fire detection, using models such as Faster R-CNN and YOLOv5. However, these models are poorly adapted to fire environments and struggle to accurately identify the shape of fire zones, limiting their ability to effectively predict fire spread and prevent and control fires. Summary of the Invention

[0003] In view of this, the present invention provides a fire detection method and device based on YOLO-Unet to solve the problems that existing fire detection methods have poor adaptability to fire environments and inaccurate detection.

[0004] In a first aspect, the present invention provides a fire detection method based on YOLO-Unet, the method comprising: Get a sample image set; The sample image set is enhanced to obtain an enhanced image set; The initial fire detection model is trained based on the enhanced image set to obtain the target fire detection model. The initial fire detection model includes the YOLOv11 model and the U-net model. Obtain the image of the target area to be detected collected by the drone; The target fire detection model is used to detect the image to be detected and obtain the fire detection result.

[0005] The present invention obtains a sample image set and performs data enhancement on it, and uses the enhanced images for model training, which helps the model understand the fire characteristics of the fire environment. The fire detection model is composed of the YOLOv11 model and the U-net model, realizing the integration of detection and segmentation. It can not only quickly locate the fire area, but also provide its precise shape. When conducting actual fire detection, the images to be detected collected by the drone are input into the target fire detection model, which can obtain accurate fire detection results, providing an important basis for subsequent fire situation analysis, emergency response and prevention and control decisions.

[0006] In an optional embodiment, the sample image set carries sample annotation information; The sample image set is enhanced to obtain an enhanced image set, including: For each sample image in the sample image set, the sample image is randomly cropped to obtain multiple cropped sample images; For each cropped sample image, randomly scale the cropped sample image to obtain a scaled sample image; All scaled sample images and their sample annotation information are mixed and enhanced to obtain multiple enhanced sample images and their enhanced annotation information, forming an enhanced image set.

[0007] Through random cropping, the present invention can simulate different scale and position changes, and further expand the sample diversity through random scaling. Through hybrid enhancement, multiple sample images are combined into new training samples to generate more diverse training data. Training based on these samples can significantly improve the generalization ability of the model, thereby improving the accuracy and reliability of fire detection.

[0008] In an optional embodiment, all scaled sample images and their sample annotation information are mixedly enhanced to obtain multiple enhanced sample images and their enhanced annotation information to form an enhanced image set, including: Dividing all scaled sample images into a first sample set and a second sample set according to a preset ratio; Using the MixUp interpolation method, interpolate any two scaled sample images and their sample annotation information in the first sample set to obtain corresponding enhanced sample images and their enhanced annotation information; The CutMix interpolation method is used to interpolate any two scaled sample images and their sample annotation information in the second sample set to obtain corresponding enhanced sample images and their enhanced annotation information.

[0009] The present invention divides the scaled sample images into two sample sets, which are used for MixUp and CutMix enhancement respectively, to ensure that the generated enhanced sample images have higher diversity and quality, thereby enhancing the flexibility and robustness of the training process.

[0010] In an optional embodiment, the initial fire detection model is trained based on the enhanced image set to obtain a target fire detection model, including: Using the YOLOv11 model in the initial fire detection model, each enhanced sample image in the enhanced image set is detected to obtain the corresponding first detection result; Performing confidence screening based on the first detection result of each enhanced sample image to obtain a plurality of target sample images; The U-net model in the initial fire detection model is used to detect each target sample image and obtain the corresponding second detection result; Calculating a first training loss based on the first detection result and enhanced annotation information of each enhanced sample image, and calculating a second training loss based on the second detection result and enhanced annotation information of each target sample image; Based on the first training loss and the second training loss, the model parameters of the initial fire detection model are updated to obtain the target fire detection model.

[0011] The present invention can efficiently perform preliminary detection of fire areas in enhanced sample images by using the YOLOv11 model, obtain target sample images through confidence screening, and input them into the U-net model for re-detection. The U-net model focuses on fine-grained feature extraction of the target sample images, generates bounding boxes and fire contours for each fire area, and finally calculates the training loss based on the detection results and sample annotation information, thereby updating the model parameters based on the training loss to obtain a target fire detection model. The model combines the fast global detection capability of YOLOv11 with the fine local segmentation capability of U-net, significantly improving the accuracy of fire area detection and contour extraction.

[0012] In an optional embodiment, the YOLOv11 model includes a backbone module, a neck module, and a head module; The YOLOv11 model in the initial fire detection model is used to detect each enhanced sample image in the enhanced image set to obtain the corresponding first detection result, including: The backbone module is used to extract the first down-sampled features from the enhanced sample image; The neck module is used to perform information fusion based on the first down-sampled features to obtain multi-scale features; The head module is used to determine the first detection result based on multi-scale features.

[0013] The present invention significantly improves the accuracy of fire area detection through the efficient feature extraction of the backbone module, the multi-scale feature fusion of the neck module, and the precise prediction of the head module.

[0014] In an optional embodiment, the U-net model includes an encoder and a decoder; The U-net model in the initial fire detection model is used to detect each target sample image and obtain the corresponding second detection results, including: Using the encoder, convolution and pooling are performed on the target sample image to obtain the second down-sampled features after each convolution and pooling; Using the decoder, deconvolve each second downsampled feature to obtain the corresponding deconvolution feature map; Perform skip connection and multi-level feature fusion on each second down-sampled feature in the encoder and each deconvolution feature map in the decoder to obtain multiple fused feature maps; Stack multiple fusion feature maps to obtain a comprehensive feature map, and classify each pixel in the comprehensive feature map to obtain the classification result; Based on the classification results of the comprehensive feature map, a fire area mask of the target sample image is generated; Extract the fire contour and bounding box of each fire area based on the fire area mask; The bounding box and fire outline of each fire area in the target sample image are used as the second detection result.

[0015] The present invention significantly improves the accuracy of fire area detection through the combined effects of efficient feature extraction by the encoder, high-resolution feature recovery by the decoder, and multi-level feature fusion through jump connections.

[0016] In an optional embodiment, the backbone module includes a focusing module, a cross-stage partial network and a deformable convolutional network; The backbone module is used to extract the first down-sampling features from the enhanced sample image, including: extracting first sample image features from the enhanced sample image using a focusing module; Using a cross-stage partial network to extract the second sample image features from the first sample image features; A deformable convolutional network is used to extract the first down-sampled features from the second sample image features.

[0017] Through the collaborative work of the focusing module, the cross-stage partial network and the variable convolutional network, the present invention enables the backbone module to extract high-quality feature information layer by layer, providing a foundation for subsequent detection tasks.

[0018] In an optional embodiment, the neck module includes a path aggregation network, a feature pyramid network and a dual attention mechanism; The neck module is used to perform information fusion based on the first down-sampling features to obtain multi-scale features, including: Using a path aggregation network, the first down-sampled features are subjected to multi-scale feature fusion to obtain a first fused image feature; Using a feature pyramid network, the first fused image features are subjected to multi-scale feature fusion to obtain the second fused image features; A dual attention mechanism is adopted to adjust the channel weights of the second fused image features and generate multi-scale features.

[0019] Through the collaborative work of the path aggregation network, feature pyramid network and dual attention mechanism, the present invention enables the neck module to efficiently perform multi-scale feature fusion and generate high-quality multi-scale features, significantly improving the model's adaptability to complex fire scenes and providing a high-quality feature representation basis for fire detection tasks.

[0020] In an optional embodiment, the head module includes a classification branch and a regression branch; The head module is used to determine the first detection result based on multi-scale features, including: The classification branch is used to determine the fire probability of the enhanced sample image based on multi-scale features; The regression branch is used to determine the bounding box and confidence score of the fire area in the enhanced sample image based on multi-scale features; The non-maximum suppression algorithm is used to remove redundant bounding boxes in the enhanced sample image; The fire probability of the enhanced sample image, the bounding box of each fire area, and the confidence score are used as the first detection result.

[0021] The present invention realizes accurate and efficient fire detection by combining the collaborative work of the classification branch and the regression branch with the optimization of the non-maximum suppression algorithm, so that the head module can accurately determine the location of the fire probability and the fire area.

[0022] In a second aspect, the present invention provides a fire detection device based on YOLO-Unet, the device comprising: A first acquisition module is used to acquire a sample image set; An enhancement module, used for enhancing the sample image set to obtain an enhanced image set; A training module is used to train the initial fire detection model based on the enhanced image set to obtain a target fire detection model. The initial fire detection model includes a YOLOv11 model and a U-net model. The second acquisition module is used to obtain the image to be detected of the target area collected by the drone; The detection module is used to detect the image to be detected using the target fire detection model to obtain the fire detection result.

[0023] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to thereby execute the YOLO-Unet-based fire detection method of the first aspect or any corresponding embodiment thereof.

[0024] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the YOLO-Unet-based fire detection method of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 is a flowchart of a fire detection method based on YOLO-Unet according to an embodiment of the present invention; Figure 2 is a workflow diagram of an initial fire detection model according to an embodiment of the present invention; Figure 3 2 is a structural block diagram of a fire detection device based on YOLO-Unet according to an embodiment of the present invention; Figure 4 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0028] Existing fire detection models suffer from poor adaptability to fire environments and inaccurate detection. This invention, by acquiring a sample image set, helps the model understand the fire characteristics of the fire environment. The fire detection model used is composed of the YOLOv11 model and the U-net model, achieving integrated detection and segmentation. This allows for rapid location of the fire area and provides its precise shape, resulting in accurate fire detection results.

[0029] According to an embodiment of the present invention, an embodiment of a fire detection method based on YOLO-Unet is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that shown here.

[0030] In this embodiment, a fire detection method based on YOLO-Unet is provided, which can be used in terminals such as computers. Figure 1 : is a flow chart of a fire detection method based on YOLO-Unet according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps: Step S101: Acquire a sample image set.

[0031] Specifically, the sample image set is rich in sources, covering a variety of terrains, including lakes, swamps, mudflats, and meadows. It also includes images of both fire-affected and non-fire-affected areas. By acquiring this sample image set, a fire detection model applicable to various fire environments was constructed. This sample image set was then used for training to improve the model's generalization capabilities.

[0032] Step S102: enhancing the sample image set to obtain an enhanced image set.

[0033] Specifically, sample image sets often suffer from insufficient sample size and uneven image quality. Directly using such sample image sets for model training can result in weak generalization and low detection accuracy. Therefore, enhancing the sample image set significantly increases sample diversity and improves image quality, providing richer and more representative samples for subsequent model training and effectively improving training results.

[0034] Step S103 : Training the initial fire detection model based on the enhanced image set to obtain a target fire detection model. The initial fire detection model includes a YOLOv11 model and a U-net model.

[0035] Specifically, fire detection models in related technologies generally use Faster R-CNN or YOLOv5, which have limitations when detecting fires, such as insufficient integration of detection and segmentation, poor adaptability to fire environments, difficulty identifying underground fires, delayed diffusion phenomena, and inability to accurately provide shape information of the fire area, which is not conducive to predicting the spread trend of the fire. Therefore, an embodiment of the present invention uses a combination of the YOLOv11 model and the U-net model to form an initial fire detection model. Through this combination, the global positioning provided by the YOLOv11 model and the local details of the U-net model complement each other, and can effectively process fire characteristics in complex scenes. The U-net model can help verify the continuity of the fire area and eliminate isolated bright spots that are not fires, such as reflections on the water surface, thereby reducing false alarms and improving the accuracy and reliability of fire detection.

[0036] Step S104: obtaining an image of the target area to be detected collected by the drone.

[0037] Specifically, traditional manual inspections cannot obtain fast and comprehensive images. Drones equipped with high-definition cameras or infrared sensors can achieve rapid cruising and transmit images of the target area to be inspected in real time, effectively improving the efficiency and accuracy of fire detection.

[0038] Step S105 : Using the target fire detection model to detect the image to be detected, and obtaining a fire detection result.

[0039] Specifically, a trained target fire detection model is used to detect the image to be detected. The final fire detection result not only includes whether there is a fire area in the image, but also accurately gives the bounding box information of the fire area, which is conducive to a more comprehensive understanding of the fire situation and provides a reliable basis for fire situation analysis and emergency response.

[0040] The present invention obtains a sample image set and performs data enhancement on it, and uses the enhanced images for model training, which helps the model understand the fire characteristics of the fire environment. The fire detection model is composed of the YOLOv11 model and the U-net model, realizing the integration of detection and segmentation. It can not only quickly locate the fire area, but also provide its precise shape. When conducting actual fire detection, the images to be detected collected by the drone are input into the target fire detection model, which can obtain accurate fire detection results, providing an important basis for subsequent fire situation analysis, emergency response and prevention and control decisions.

[0041] In this embodiment, a fire detection method based on YOLO-Unet is provided, which can be used in the above-mentioned terminal. The method specifically includes the following steps: Step S201: Obtain a sample image set. Figure 1Step S101 of the illustrated embodiment will not be described in detail here.

[0042] Step S202: Enhance the sample image set to obtain an enhanced image set.

[0043] Specifically, the above step S202 includes: Step S2021 : For each sample image in the sample image set, randomly crop the sample image to obtain a plurality of cropped sample images.

[0044] Specifically, considering that the fire area may appear at any position in the actual fire scene, random cropping can help simulate this phenomenon, thereby enhancing the generalization ability of the model. For each sample image, multiple starting positions of the cropping area are randomly generated to simulate the situation of observation from different perspectives and positions. Optionally, the cropping area is generally a rectangle, with its starting position at the upper left corner of the rectangle, and the size of the rectangle is determined by the preset height. and width Determine the size of the rectangle . Assume that the sample image is ,in is the height of the sample image, is the width of the sample image, is the number of channels of the sample image. For RGB images, the number of channels is 3. The cropped image can be expressed by the following formula (1).

[0045] (1) in, Represents the cropped sample image; Indicates the coordinates of the upper left corner of the cropped area. To ensure that it is located in the sample image, it must satisfy the constraints: ; Indicates the number of channels; Indicates the height of the clipping area; Indicates the width of the cropping area.

[0046] Step S2022 : For each cropped sample image, randomly scale the cropped sample image to obtain a scaled sample image.

[0047] Specifically, in real fire scenes, the size of the fire area in the image will vary depending on its distance from the observation point. For example, a fire area at a close distance typically appears larger in the image, while a fire area at a distance appears smaller. By adjusting the size of the cropped sample images, the changes in the fire area at different distances are simulated. More specifically, for each cropped sample image, a scaling factor is randomly selected within a pre-set scaling factor range (e.g., 0.8x-1.2x), and the cropped sample image is scaled to the corresponding size to obtain a scaled sample image. By increasing sample diversity through random scaling, the model's ability to recognize objects of different scales is improved.

[0048] Step S2023 : performing mixed enhancement on all scaled sample images and their sample annotation information to obtain a plurality of enhanced sample images and their enhanced annotation information to form an enhanced image set.

[0049] In some optional implementations, step S2023 includes: Step S20231: Divide all scaled sample images into a first sample set and a second sample set according to a preset ratio.

[0050] Specifically, since there are far more non-fire images than fire images in the sample image set, in order to make full use of the limited fire samples and balance the data distribution, the scaled sample images are divided into two sample sets according to a preset ratio (such as 7:3), and processed using different hybrid enhancement methods, thereby ensuring the diversity of the overall data and the robustness of the model to complex scenes.

[0051] Step S20232: Using the MixUp interpolation method, interpolate any two scaled sample images and their sample annotation information in the first sample set to obtain corresponding enhanced sample images and their enhanced annotation information.

[0052] Specifically, MixUp is a method based on linear interpolation that can not only fuse images but also interpolate their annotation information, thereby simulating mixed scenes between different target objects. Among them, each image in the sample image set carries sample annotation information, which not only clearly marks whether there is a fire area in the image, but also uses a border to accurately delineate the fire area, including key parts such as flames and smoke. For the fire area, MixUp can effectively simulate the mixture of the fire area and other non-fire areas, such as the overlap of the fire area with plants, land, sky and other areas, thereby helping the model learn the characteristics of the fire area in a complex environment. Optionally, the following formula (2) is the enhanced sample image after mixing, and the following formula (3) is the enhanced annotation information after mixing.

[0053] (2) (3) in, represents the enhanced sample image; represents the mixing coefficient, usually from the Beta distribution Random sampling is used to control the degree of mixing, and the larger will make Closer to 0 or 1, the greater the difference between image and annotation information, the smaller It will make Closer to 0.5, making the mixture of the two images and the annotation information more balanced; and Represents any two scaled sample images; Represents the enhanced annotation information after mixing; and Indicates the sample annotation information corresponding to the two scaled sample images.

[0054] Step S20233: Use the CutMix interpolation method to interpolate any two scaled sample images and their sample annotation information in the second sample set to obtain corresponding enhanced sample images and their enhanced annotation information.

[0055] Specifically, CutMix generates new samples by replacing a portion of one image with the corresponding region of another image. This is used to simulate local mixing between target objects, such as a mixture of a fire area and other areas. Optionally, the following equation (4) represents the enhanced sample image after mixing. The enhanced annotation information after mixing can be obtained by referring to equation (3).

[0056] (4) in, represents the enhanced sample image; Indicates the coordinates of the upper left corner of the clipping area; Indicates the number of channels; Indicates the height of the clipping area; Indicates the width of the cropping area; and represents any two scaled sample images.

[0057] Step S203 : Training the initial fire detection model based on the enhanced image set to obtain a target fire detection model. The initial fire detection model includes a YOLOv11 model and a U-net model.

[0058] Specifically, the above step S203 includes: In step S2031 , the YOLOv11 model in the initial fire detection model is used to detect each enhanced sample image in the enhanced image set to obtain a corresponding first detection result. The YOLOv11 model includes a backbone module, a neck module, and a head module.

[0059] In some optional implementations, the above step S2031 includes: In step S20311, a backbone module is used to extract the first down-sampling feature from the enhanced sample image. The backbone module includes a focusing module, a cross-stage partial network, and a variable convolutional network.

[0060] In some optional implementations, step S20311 includes: Step a1: using a focusing module to extract a first sample image feature from the enhanced sample image.

[0061] Specifically, the Focus Module is a key component of the YOLOv11 model's backbone. In fire detection scenarios, enhanced sample images contain a wealth of background information, while the fire area may only occupy a small portion of the image. The Focus Module accurately locates potential fire areas within complex image backgrounds and extracts the primary sample image features associated with them, reducing interference from redundant background on model judgment.

[0062] Step a2: Using a cross-stage partial network, extract the second sample image features from the first sample image features.

[0063] Specifically, the Cross-Stage Partial Network (CSP) plays a key role in optimizing feature extraction within the backbone module. The first-stage image features contain information at different levels and dimensions. The CSP further processes these features through a specific network structure and processing method. This allows for effective fusion of features from different stages, preserving the detailed information in low-level features while integrating the semantic information of higher-level features. This allows for the extraction of richer and more representative second-stage image features from the first-stage image features, helping to enhance the model's understanding and grasp of fire area characteristics and improve its ability to detect fires in complex environments.

[0064] Step a3: Using a deformable convolutional network, extract the first down-sampling feature from the second sample image feature.

[0065] Specifically, the Deformable Convolutional Network Version 4 (DCNv4) is one of the core technologies of the backbone module to cope with complex backgrounds. In a fire environment, complex background factors such as water surface reflection and interlaced vegetation can interfere with the detection of fire areas. Traditional convolution operations use fixed convolution kernel shapes and positions, which are difficult to adapt to the deformation of the target and changes in complex backgrounds. DCNv4, by introducing deformable convolution, allows the shape and position of the convolution kernel to be dynamically adjusted according to the content of the second sample image features. The deformable convolution network obtains the offset of the convolution kernel position shown in the following formula (5) through an additional offset prediction network, allowing the convolution kernel to move and deform freely on the second sample image features, thereby better capturing the local features of the target and the contextual information of the complex background. In this process, the second sample image features are downsampled to obtain the first downsampled features, which significantly improves the model's detection accuracy and robustness for fire areas in complex backgrounds, and provides high-quality feature data for subsequent models' fire detection tasks.

[0066] (5) in, and Indicates the horizontal and vertical offset of the convolution kernel position; represents each position in the second sample image feature; Represents the offset prediction network, such as the activation function; and Indicates the horizontal and vertical offset of the second sample image feature.

[0067] In step S20312, a neck module is used to perform information fusion based on the first down-sampled features to obtain multi-scale features. The neck module includes a path aggregation network, a feature pyramid network, and a dual attention mechanism.

[0068] In some optional implementations, step S20312 includes: In step b1, a path aggregation network is used to perform multi-scale feature fusion on the first down-sampled features to obtain a first fused image feature.

[0069] Specifically, the Path Aggregation Network (PANet) plays a key role in the neck module. The first downsampled features contain information at different scales, but this information is relatively dispersed. PANet's role is to effectively integrate this multi-scale information through a specific structure and algorithm. It uses a bottom-up approach to fuse low-level features rich in detail with high-level features with strong semantic information. In this process, features at different scales complement each other, enabling the model to simultaneously focus on both the details and overall characteristics of the fire area. For example, small-scale features can help identify the details of small fire sources, while large-scale features help grasp the overall situation of a large-scale fire. Through this fusion method, the resulting first fused image features combine the advantages of different scales, providing more comprehensive information for subsequent processing and improving the model's ability to detect fires of different sizes.

[0070] In step b2, a feature pyramid network is used to perform multi-scale feature fusion on the first fused image features to obtain second fused image features.

[0071] Specifically, the Feature Pyramid Network (FPN), a classic multi-scale feature fusion method, further optimizes the first fused image features. FPN constructs a multi-scale feature pyramid, starting with the high-resolution feature map at the top layer of the network and upsampling it layer by layer. The upsampled high-level feature maps are then combined with feature maps from lower layers through lateral connections, achieving a further fusion of features at different levels. This method leverages the semantic information of high-level feature maps while preserving the spatial details of low-level feature maps. In fire detection, fire areas may appear at different scales. FPN can capture this multi-scale information, enhancing the detection of low-visibility fires (such as underground fires or thin smoke). The resulting second fused image features, processed by FPN, further improve the fusion of multi-scale information, providing stronger support for accurate fire area detection.

[0072] In step b3, a dual attention mechanism is used to adjust the channel weights of the second fused image features and generate multi-scale features.

[0073] Specifically, the dual attention mechanism combines channel attention and spatial attention to further optimize the second fused image features to generate more targeted multi-scale features. In terms of channel attention, by calculating the global average pooling value of each channel, the feature representation of each channel is obtained as shown in the following equation (6). Then, the attention vector shown in the following equation (7) is generated through the activation function. The weight of each channel is dynamically adjusted to enhance the attention to the channel containing important fire information, suppress the influence of irrelevant channels, and highlight the key features of the fire area.

[0074] (6) in, Represents the feature representation of each channel; Indicates the height of the second fused image; Indicates the height of the second fused image; represents the channel of the second fused image; Indicates the location The feature map value at .

[0075] (7) in, represents the attention vector; Represents an activation function, such as sigmoid; Represents the learned weight matrix; represents the feature representation of each channel.

[0076] In terms of spatial attention, the average pooling and maximum pooling of the channel dimension of the feature map are calculated to generate channel statistics. Then, through an activation function, such as the sigmoid function, a spatial weight map is generated as shown in the following equation (8). The weight of each spatial position is dynamically adjusted to enhance the focus on the important spatial locations where the fire area is located. The dual attention mechanism comprehensively captures the global and local information of the feature map, enabling the model to focus on the fire area more accurately. The generated multi-scale features can be more effectively used for fire area detection, significantly improving the model's ability to detect fire areas in complex scenarios.

[0077] (8) in, represents a spatial weight map; Represents an activation function, such as sigmoid; Represents the learned spatial weight matrix; represents the average pooled feature representation; represents the maximum pooling feature representation; Represents a splicing operation.

[0078] Step S20313: Using a head module, based on multi-scale features, determine a first detection result. The head module includes a classification branch and a regression branch.

[0079] In some optional implementations, step S20313 includes: In step c1, the classification branch is used to determine the fire probability of the enhanced sample image based on multi-scale features.

[0080] Specifically, in the head module, the classification branch is used to determine the presence of a fire in the enhanced sample image and the fire probability. The classification branch receives multi-scale features, which fuse information from different levels and scales and contain rich clues about the image content. These multi-scale features are deeply analyzed and processed through a series of convolutional and fully connected neural network structures, learning the association patterns between different feature combinations and the presence of fire. The multi-scale features are then mapped to a probability value, which represents the likelihood of a fire in the image. For example, when the model learns that typical features of flames and smoke (such as specific colors, textures, and shapes) appear in certain combinations within the multi-scale features, the predicted fire probability increases; conversely, if these key features are not detected, the fire probability decreases.

[0081] In step c2, the regression branch is used to determine the bounding box and confidence score of the fire area in the enhanced sample image based on multi-scale features.

[0082] Specifically, the regression branch, similarly based on the multi-scale features of the neck module, processes these features through a specific network structure to predict the bounding box coordinates of the fire area. These bounding box coordinates define a rectangular area that is considered likely to contain a fire. The regression branch also generates a confidence score for each predicted bounding box. This score reflects the model's confidence that a fire actually exists within the bounding box. A higher score indicates a greater likelihood of a fire within the bounding box.

[0083] In step c3, a non-maximum suppression algorithm is used to remove redundant bounding boxes in the enhanced sample image.

[0084] Specifically, after processing by the classification and regression branches, the model may predict multiple bounding boxes, some of which may overlap or be redundant, meaning that multiple bounding boxes may point to the same fire area. The non-maximum suppression (NMS) algorithm can remove these redundant bounding boxes, retaining only the optimal one or a few, thereby improving the accuracy and efficiency of detection results. More specifically, the NMS algorithm sorts all predicted bounding boxes based on their confidence scores and selects the bounding box with the highest confidence score as the retained box. It then calculates the degree of overlap between the remaining bounding boxes and the retained box. If the degree of overlap between a bounding box and the retained box exceeds a preset threshold (e.g., 0.5), the bounding box is considered redundant and removed. This process is repeated until all bounding boxes have been processed, resulting in a filtered and de-redundant set of bounding boxes. This avoids duplicate detection of the same fire area, improving the accuracy and practicality of fire detection.

[0085] In step c4, the fire probability of the enhanced sample image, the bounding box of each fire area, and the confidence score are used as the first detection result.

[0086] Specifically, the bounding box of each fire area after removing redundant bounding boxes and its confidence score are used together with the fire probability as the first detection result to achieve preliminary detection of the fire area.

[0087] Step S2032 : Confidence screening is performed based on the first detection result of each enhanced sample image to obtain a plurality of target sample images.

[0088] Specifically, since detection results may contain misjudgments and not all detection results are accurate and reliable, they need to be screened based on confidence. An appropriate confidence threshold is set, and the confidence score of each first detection result is compared with the threshold one by one. The image corresponding to the first detection result with a confidence score that reaches the threshold is retained, i.e., the target sample image, which serves as the input for the subsequent U-net model to further accurately segment the fire area and improve the accuracy of fire detection.

[0089] In step S2033, the U-net model in the initial fire detection model is used to detect each target sample image to obtain a corresponding second detection result. The U-net model includes an encoder and a decoder.

[0090] Specifically, the above step S2033 includes: In step S20331, an encoder is used to perform convolution and pooling on the target sample image to obtain a second down-sampling feature after each convolution and pooling.

[0091] Specifically, in the U-net model, the encoder performs preliminary feature extraction and dimensionality reduction on the target sample image. Convolution operations use different convolution kernels to slide across the image, extracting various features from the image, such as texture, color, shape, and other information, capturing different local patterns in the image. Pooling operations are used to reduce resolution and data volume while retaining key features. Multiple convolutions and pooling operations are performed on the target sample image in sequence to obtain the corresponding second-level downsampled features, which contain more representative and discriminative features of the target sample image, providing an important data foundation for subsequent decoder operations.

[0092] In step S20332, a decoder is used to perform deconvolution on each second downsampled feature to obtain a corresponding deconvolution feature map.

[0093] Specifically, the decoder corresponds to the encoder and is used to restore the resolution and accurately segment the fire area. Deconvolution gradually restores the image details by increasing the size of the feature map, expanding and reconstructing the second downsampled features.

[0094] In step S20333, each second down-sampled feature in the encoder and each deconvolution feature map in the decoder are skip-connected and multi-level feature fused to obtain multiple fused feature maps.

[0095] Specifically, underground fire spread or localized low-temperature combustion may manifest as irregular shapes, while the diffusion of smoke and steam may obscure the fire source. Skip connections can help the model retain these detailed information, thereby improving segmentation accuracy and reducing the risk of misidentification when dealing with complex fire forms. Multi-level feature fusion proportionally introduces deconvolution feature maps from different encoder layers between each convolutional layer of the decoder, fusing low-level detail features with high-level semantic features, as shown in the following equation (9), resulting in a fused feature map that contains both low-level details and high-level semantics.

[0096] (9) in, After fusion, The fused feature map of the convolutional layer; Represents a hyperparameter, which is used to control the fusion ratio between low-level features and high-level features. ,The model can find the best balance between features at different levels, using low-level features to capture details and high-level features to obtain global semantic information; Indicates the first The second down-sampled features obtained by the convolutional layer; Indicates the first Deconvolution feature map corresponding to the convolution layer.

[0097] In step S20334, multiple fused feature maps are stacked to obtain a comprehensive feature map, and each pixel in the comprehensive feature map is classified to obtain a classification result.

[0098] Specifically, multiple fused feature maps are concatenated according to the channel dimension, or feature maps at different levels are aggregated into a comprehensive feature map through convolution. The comprehensive feature map is then classified using convolutional layers and activation functions to obtain a classification result for each pixel. This classification result represents the probability that the pixel belongs to a fire area.

[0099] Step S20335: Generate a fire area mask of the target sample image based on the classification result of the comprehensive feature map.

[0100] Specifically, a mask is set for the fire area and the non-fire area, for example, the fire area is 1 and the non-fire area is 0. At the same time, a preset threshold is set, and pixels whose classification results exceed the preset threshold are marked as 1 to indicate that the pixel belongs to the fire area, and pixels whose classification results are below the preset threshold are marked as 0 to indicate that the pixel belongs to the non-fire area, thereby obtaining a binary mask of the target sample image, that is, the fire area mask.

[0101] Step S20336: Extract the fire outline and bounding box of each fire area based on the fire area mask.

[0102] Specifically, an image processing algorithm, such as the Canny edge detection algorithm, is used to perform a connected domain analysis on the fire area mask, extracting the edge contours of the fire area as the fire outline. For each connected domain, a minimum enclosing rectangle is calculated as the bounding box, and its coordinates are determined. Optionally, an area threshold is set and the area enclosed by each bounding box is compared with the threshold. Bounding boxes with an area smaller than the threshold are eliminated.

[0103] Step S20337: The bounding box and fire outline of each fire area in the target sample image are used as the second detection result of the target sample image.

[0104] Specifically, the bounding box and fire outline of each fire area in the target sample image are used as the second detection result of the target sample image to achieve accurate detection of the fire area.

[0105] Step S2034: Calculate a first training loss based on the first detection result and enhanced annotation information of each enhanced sample image, and calculate a second training loss based on the second detection result and enhanced annotation information of each target sample image.

[0106] Specifically, the first detection result, generated by the YOLOv11 model, includes the fire probability, bounding box coordinates, and confidence score. This first detection result is compared with the augmented annotations, and a first training loss is calculated using an appropriate loss function (such as the cross-entropy loss function for classification tasks or the mean squared error loss function for regression tasks). The second detection result, generated by the U-net model, is compared with the augmented annotations and a second training loss is calculated using a corresponding loss function (such as the Dice loss function, commonly used in image segmentation tasks). These two training losses are used to evaluate the model's performance at different stages, providing a basis for subsequent model parameter updates, enabling the model to better fit real-world data and improving fire detection accuracy.

[0107] Step S2035 : Based on the first training loss and the second training loss, the model parameters of the initial fire detection model are updated to obtain a target fire detection model.

[0108] Specifically, the gradient of the loss function with respect to the model parameters is calculated to determine the direction and step size of the parameter update. With each iteration, the model parameters are adjusted accordingly based on the gradient information, gradually reducing the first and second training losses. As training progresses, the model's ability to detect and segment fire areas gradually improves. When the model training reaches a certain stopping condition, training concludes, resulting in the target fire detection model, which exhibits higher accuracy and robustness in fire detection tasks and can be better applied in real-world fire monitoring scenarios.

[0109] In some optional embodiments, Figure 2 is a workflow diagram of the initial fire detection model according to an embodiment of the present invention, such as Figure 2 As shown, the sample image set is first processed through random cropping, random scaling, and data augmentation to obtain an enhanced dataset. The initial fire detection model consists of a YOLOv11 model and a U-net model. The enhanced dataset is first input to the backbone module of the YOLOv11 model. The focus module extracts first sample image features, followed by a cross-stage partial network to extract second sample image features. Finally, a variable convolutional network dynamically adjusts the features to obtain first downsampled features, which are then input to the neck module. These features are first fused through a path aggregation network to obtain first fused image features, which are then further fused through a feature pyramid network to obtain second fused image features. Finally, a dual attention mechanism adjusts these features to obtain multi-scale features, which are then input to the head module. The classification branch first obtains the fire probability, and the regression branch determines the bounding box coordinates and confidence score of the fire area. Non-maximum suppression is used to address redundancy, ultimately yielding the first detection result. Confidence-based filtering is performed to obtain the target sample image, which is then input to the U-net model. The encoder first performs convolution and pooling to obtain the second downsampled features, which are then input to the decoder. The decoder performs deconvolution to obtain a deconvolution feature map. The encoder and decoder are skipped and multi-level features are fused to obtain the second detection result.

[0110] Step S204: Obtain the image of the target area to be detected collected by the drone. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0111] Step S205: Use the target fire detection model to detect the image to be detected and obtain the fire detection result. Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.

[0112] The present invention obtains a sample image set and performs data enhancement on it, and uses the enhanced images for model training, which helps the model understand the fire characteristics of the fire environment. The fire detection model is composed of the YOLOv11 model and the U-net model, realizing the integration of detection and segmentation. It can not only quickly locate the fire area, but also provide its precise shape. When conducting actual fire detection, the images to be detected collected by the drone are input into the target fire detection model, which can obtain accurate fire detection results, providing an important basis for subsequent fire situation analysis, emergency response and prevention and control decisions.

[0113] This embodiment also provides a fire detection device based on YOLO-Unet, which is used to implement the above-mentioned embodiments and preferred embodiments. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0114] This embodiment provides a fire detection device based on YOLO-Unet. Figure 3 As shown, including: The first acquisition module 301 is used to acquire a sample image set.

[0115] The enhancement module 302 is configured to enhance the sample image set to obtain an enhanced image set.

[0116] The training module 303 is used to train the initial fire detection model based on the enhanced image set to obtain a target fire detection model. The initial fire detection model includes a YOLOv11 model and a U-net model.

[0117] The second acquisition module 304 is used to acquire the image to be detected of the target area collected by the drone.

[0118] The detection module 305 is used to detect the image to be detected using the target fire detection model to obtain a fire detection result.

[0119] In some optional implementations, the sample image set carries sample annotation information; The enhancement module 302 includes: The cropping unit is configured to randomly crop each sample image in the sample image set to obtain a plurality of cropped sample images.

[0120] The scaling unit is configured to randomly scale each cropped sample image to obtain a scaled sample image.

[0121] The enhancement unit is used to perform mixed enhancement on all scaled sample images and their sample annotation information to obtain multiple enhanced sample images and their enhanced annotation information to form an enhanced image set.

[0122] In some optional embodiments, the enhancement unit includes: The division subunit is used to divide all scaled sample images into a first sample set and a second sample set according to a preset ratio.

[0123] The first enhancement subunit is configured to interpolate any two scaled sample images and their sample annotation information in the first sample set using a MixUp interpolation method to obtain corresponding enhanced sample images and their enhanced annotation information.

[0124] The second enhancement subunit is configured to interpolate any two scaled sample images and their sample annotation information in the second sample set using a CutMix interpolation method to obtain corresponding enhanced sample images and their enhanced annotation information.

[0125] In some optional implementations, the training module 303 includes: The first detection unit is configured to detect each enhanced sample image in the enhanced image set by using the YOLOv11 model in the initial fire detection model to obtain a corresponding first detection result.

[0126] The screening unit is used to perform confidence screening based on the first detection result of each enhanced sample image to obtain multiple target sample images.

[0127] The second detection unit is used to detect each target sample image using the U-net model in the initial fire detection model to obtain a corresponding second detection result.

[0128] The calculation unit is used to calculate the first training loss based on the first detection result and the enhanced annotation information of each enhanced sample image, and calculate the second training loss based on the second detection result and the enhanced annotation information of each target sample image.

[0129] The training unit is used to update the model parameters of the initial fire detection model based on the first training loss and the second training loss to obtain a target fire detection model.

[0130] In some optional embodiments, the YOLOv11 model includes a backbone module, a neck module, and a head module; The first detection unit includes: The first extraction subunit is configured to extract a first down-sampling feature from the enhanced sample image using a backbone module.

[0131] The first fusion subunit is used to use the neck module to perform information fusion based on the first down-sampling feature to obtain a multi-scale feature.

[0132] The first determination subunit is configured to determine a first detection result based on multi-scale features using a head module.

[0133] In some optional embodiments, the U-net model includes an encoder and a decoder; The second detection unit includes: The second extraction subunit is used to use an encoder to perform convolution and pooling on the target sample image to obtain a second down-sampling feature after each convolution and pooling.

[0134] The deconvolution subunit is configured to perform deconvolution on each second downsampled feature using a decoder to obtain a corresponding deconvolution feature map.

[0135] The second fusion subunit is used to perform jump connection and multi-level feature fusion on each second down-sampling feature in the encoder and each deconvolution feature map in the decoder to obtain multiple fused feature maps.

[0136] The classification subunit is used to stack multiple fusion feature maps to obtain a comprehensive feature map, and classify each pixel in the comprehensive feature map to obtain a classification result.

[0137] The generation subunit is used to generate a fire area mask of the target sample image based on the classification result of the comprehensive feature map.

[0138] The third extraction subunit is configured to extract a fire outline and a bounding box of each fire area based on the fire area mask.

[0139] The second determining subunit is configured to take the bounding box and the fire outline of each fire area in the target sample image as the second detection result.

[0140] In some optional embodiments, the backbone module includes a focusing module, a cross-stage partial network, and a deformable convolutional network; The first extraction subunit is specifically used to use a focusing module to extract a first sample image feature from the enhanced sample image; use a cross-stage partial network to extract a second sample image feature from the first sample image feature; and use a variable convolutional network to extract a first downsampling feature from the second sample image feature.

[0141] In some optional embodiments, the neck module includes a path aggregation network, a feature pyramid network, and a dual attention mechanism; The first fusion subunit is specifically used to use a path aggregation network to perform multi-scale feature fusion on the first down-sampled features to obtain a first fused image feature; use a feature pyramid network to perform multi-scale feature fusion on the first fused image features to obtain a second fused image feature; use a dual attention mechanism to adjust the channel weights of the second fused image features and generate multi-scale features.

[0142] In some optional embodiments, the head module includes a classification branch and a regression branch; The first determination subunit is specifically used to use a classification branch to determine the fire probability of the enhanced sample image based on multi-scale features; use a regression branch to determine the bounding box and confidence score of the fire area in the enhanced sample image based on multi-scale features; use a non-maximum suppression algorithm to remove redundant bounding boxes in the enhanced sample image; and use the fire probability of the enhanced sample image, the bounding box of each fire area, and the confidence score as the first detection result.

[0143] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0144] The fire detection device based on YOLO-Unet in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0145] The embodiment of the present invention also provides a computer device having the above Figure 3 The fire detection device based on YOLO-Unet is shown.

[0146] See also Figure 4 , Figure 4 is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 4As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0147] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0148] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0149] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0150] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0151] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 4 The bus connection is taken as an example.

[0152] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the computer device. Examples include a touch screen, keypad, mouse, trackpad, touchpad, pointing stick, one or more mouse buttons, trackball, joystick, etc. The output device 40 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibration motors). Such display devices include, but are not limited to, liquid crystal displays, light emitting diodes, monitors, and plasma displays. In some optional embodiments, the display device may be a touch screen.

[0153] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0154] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0155] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A fire detection method based on YOLO-Unet, characterized in that: The method comprises: Get a sample image set; enhancing the sample image set to obtain an enhanced image set; Training an initial fire detection model based on the enhanced image set to obtain a target fire detection model, wherein the initial fire detection model includes a YOLOv11 model and a U-net model; Obtain the image of the target area to be detected collected by the drone; The target fire detection model is used to detect the image to be detected to obtain a fire detection result.

2. The method according to claim 1, characterized in that The sample image set carries sample annotation information; The step of enhancing the sample image set to obtain an enhanced image set includes: For each sample image in the sample image set, randomly cropping the sample image to obtain a plurality of cropped sample images; For each cropped sample image, randomly scaling the cropped sample image to obtain a scaled sample image; All scaled sample images and their sample annotation information are mixedly enhanced to obtain a plurality of enhanced sample images and their enhanced annotation information, forming the enhanced image set.

3. The method according to claim 2, characterized in that The hybrid enhancement of all scaled sample images and their sample annotation information to obtain a plurality of enhanced sample images and their enhanced annotation information to form the enhanced image set includes: Dividing all scaled sample images into a first sample set and a second sample set according to a preset ratio; Using the MixUp interpolation method, interpolate any two scaled sample images and their sample annotation information in the first sample set to obtain corresponding enhanced sample images and their enhanced annotation information; The CutMix interpolation method is used to interpolate any two scaled sample images and their sample annotation information in the second sample set to obtain corresponding enhanced sample images and their enhanced annotation information.

4. The method according to claim 2, characterized in that The training of the initial fire detection model based on the enhanced image set to obtain a target fire detection model includes: Using the YOLOv11 model in the initial fire detection model, detect each enhanced sample image in the enhanced image set to obtain a corresponding first detection result; Performing confidence screening based on the first detection result of each enhanced sample image to obtain a plurality of target sample images; Using the U-net model in the initial fire detection model, each target sample image is detected to obtain a corresponding second detection result; Calculating a first training loss based on the first detection result and enhanced annotation information of each enhanced sample image, and calculating a second training loss based on the second detection result and enhanced annotation information of each target sample image; Based on the first training loss and the second training loss, the model parameters of the initial fire detection model are updated to obtain the target fire detection model.

5. The method according to claim 4, characterized in that The YOLOv11 model includes backbone module, neck module and head module; The YOLOv11 model in the initial fire detection model is used to detect each enhanced sample image in the enhanced image set to obtain a corresponding first detection result, including: Using the backbone module, extracting a first down-sampling feature from the enhanced sample image; Using the neck module, information fusion is performed based on the first down-sampled features to obtain multi-scale features; The head module is used to determine the first detection result based on the multi-scale features.

6. The method according to claim 4, characterized in that The U-net model includes an encoder and a decoder; The U-net model in the initial fire detection model is used to detect each target sample image to obtain a corresponding second detection result, including: Using the encoder, convolution and pooling are performed on the target sample image to obtain a second down-sampled feature after each convolution and pooling; Deconvolving each second downsampled feature using the decoder to obtain a corresponding deconvolution feature map; Performing skip connection and multi-level feature fusion on each second down-sampled feature in the encoder and each deconvolution feature map in the decoder to obtain multiple fused feature maps; Stacking the multiple fusion feature maps to obtain a comprehensive feature map, and classifying each pixel in the comprehensive feature map to obtain a classification result; generating a fire area mask of the target sample image based on the classification result of the comprehensive feature map; extracting a fire outline and a bounding box of each fire area based on the fire area mask; The bounding box and fire outline of each fire area in the target sample image are used as the second detection result.

7. The method according to claim 5, characterized in that The backbone module includes a focusing module, a cross-stage partial network and a variable convolutional network; The step of extracting a first down-sampling feature from the enhanced sample image using the backbone module includes: extracting a first sample image feature from the enhanced sample image using the focusing module; Using a cross-stage partial network, extracting second sample image features from the first sample image features; A deformable convolutional network is used to extract the first down-sampled features from the second sample image features.

8. The method according to claim 5, characterized in that The neck module includes a path aggregation network, a feature pyramid network and a dual attention mechanism; The adopting the neck module to perform information fusion based on the first down-sampling features to obtain multi-scale features includes: Using a path aggregation network, performing multi-scale feature fusion on the first down-sampled features to obtain a first fused image feature; Using a feature pyramid network, performing multi-scale feature fusion on the first fused image features to obtain a second fused image feature; A dual attention mechanism is adopted to adjust the channel weights of the second fused image features and generate the multi-scale features.

9. The method according to claim 5, characterized in that The head module includes a classification branch and a regression branch; The step of using the head module to determine the first detection result based on the multi-scale features includes: Determining the fire probability of the enhanced sample image based on the multi-scale features using the classification branch; Determining a bounding box and a confidence score of a fire area in the enhanced sample image based on the multi-scale features using the regression branch; Using a non-maximum suppression algorithm to remove redundant bounding boxes in the enhanced sample image; The fire probability of the enhanced sample image, the bounding box of each fire area, and the confidence score are used as the first detection result.

10. A fire detection device based on YOLO-Unet, characterized in that: The device comprises: A first acquisition module is used to acquire a sample image set; an enhancement module, configured to enhance the sample image set to obtain an enhanced image set; A training module, configured to train an initial fire detection model based on the enhanced image set to obtain a target fire detection model, wherein the initial fire detection model includes a YOLOv11 model and a U-net model; The second acquisition module is used to obtain the image to be detected of the target area collected by the drone; The detection module is used to detect the image to be detected using the target fire detection model to obtain a fire detection result.

Images