Fire detection method and device based on YOLO-Unet

Abstract

The application relates to the technical field of image processing, and discloses a fire detection method and device based on YOLO-Unet, which comprises the following steps: acquiring a sample image set; enhancing the sample image set to obtain an enhanced image set; training an initial fire detection model based on the enhanced image set to obtain a target fire detection model, wherein the initial fire detection model comprises a YOLOv11 model and a U-net model; acquiring an image to be detected collected by a drone; and detecting the image to be detected by using the target fire detection model to obtain a fire detection result. By acquiring the sample image set, the model can understand the fire characteristics; the fire detection model is composed of the YOLOv11 model and the U-net model, the integration of detection and segmentation is realized, the fire area can be quickly located, accurate shapes can be provided, the fire detection accuracy is improved, and important basis is provided for subsequent fire situation analysis, emergency response and prevention and control decision-making.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a fire detection method and device based on YOLO-Unet. BACKGROUND

[0002] Fire detection is of great significance in ecological protection and disaster prevention. At present, deep learning has been applied to fire detection, such as Faster R-CNN or YOLOv5 model. However, these models have poor adaptability to fire environment and are difficult to provide accurate shape of fire area, which limits the effective prediction and prevention ability of fire expansion trend. SUMMARY

[0003] Therefore, the present application provides a fire detection method and device based on YOLO-Unet to solve the problem of poor adaptability to fire environment and inaccurate detection of existing fire detection methods.

[0004] In a first aspect, the present application provides a fire detection method based on YOLO-Unet, which comprises:

[0005] obtaining a sample image set;

[0006] enhancing the sample image set to obtain an enhanced image set;

[0007] training an initial fire detection model based on the enhanced image set to obtain a target fire detection model, the initial fire detection model comprising a YOLOv11 model and a U-net model;

[0008] obtaining a target region image to be detected collected by a UAV;

[0009] detecting the target region image to be detected using the target fire detection model to obtain a fire detection result.

[0010] The present application obtains a sample image set and enhances the data, uses the enhanced image for model training, which helps the model to understand the fire characteristics of the fire environment. The fire detection model is composed of a YOLOv11 model and a U-net model, which realizes the integration of detection and segmentation, can quickly locate the fire area, and can provide its accurate shape. When actual fire detection is performed, the target fire detection model is input into the target region image collected by the UAV, and an accurate fire detection result can be obtained, which provides an important basis for subsequent fire situation analysis, emergency response and prevention decision.

[0011] In an optional embodiment, the sample image set carries sample annotation information;

[0012] enhancing the sample image set to obtain an enhanced image set, comprising:

[0013] randomly cropping the sample image to obtain a plurality of cropped sample images;

[0014] randomly scaling each cropped sample image to obtain a scaled sample image;

[0015] performing mixed augmentation on all scaled sample images and their sample annotation information to obtain a plurality of augmented sample images and their augmented annotation information, forming an augmented image set.

[0016] The present application can simulate different scale and position changes through random cropping, further expand sample diversity through random scaling, combine multiple sample images into new training samples through mixed augmentation, generate more diversified training data, and significantly improve the generalization ability of the model based on these samples, thereby improving the accuracy and reliability of fire detection.

[0017] In an optional implementation, performing mixed augmentation on all scaled sample images and their sample annotation information to obtain a plurality of augmented sample images and their augmented annotation information, forming an augmented image set, comprises:

[0018] dividing all scaled sample images into a first sample set and a second sample set according to a preset ratio;

[0019] using a MixUp interpolation method to interpolate any two scaled sample images in the first sample set and their sample annotation information to obtain corresponding augmented sample images and their augmented annotation information;

[0020] using a CutMix interpolation method to interpolate any two scaled sample images in the second sample set and their sample annotation information to obtain corresponding augmented sample images and their augmented annotation information.

[0021] The present application divides the scaled sample images into two sample sets for MixUp and CutMix augmentation respectively, ensuring that the generated augmented sample images have higher diversity and quality, and enhancing the flexibility and robustness of the training process.

[0022] In an optional implementation, training an initial fire detection model based on the augmented image set to obtain a target fire detection model, comprises:

[0023] using a YOLOv11 model in the initial fire detection model to detect each augmented sample image in the augmented image set to obtain a corresponding first detection result;

[0024] performing confidence screening based on the first detection result of each augmented sample image to obtain a plurality of target sample images;

[0025] The U-net model in the initial fire detection model is used to detect each target sample image to obtain a corresponding second detection result;

[0026] Based on the first detection result of each enhanced sample image and the enhanced annotation information, a first training loss is calculated, and based on the second detection result of each target sample image and the enhanced annotation information, a second training loss is calculated;

[0027] Based on the first training loss and the second training loss, the model parameters of the initial fire detection model are updated to obtain a target fire detection model.

[0028] The YOLOv11 model is used to efficiently preliminarily detect the fire area in the enhanced sample image, the target sample image is obtained through confidence screening, and is input into the U-net model for re-detection. The U-net model focuses on fine-grained feature extraction of the target sample image, generates a boundary box and a fire contour for each fire area, and finally calculates a training loss based on the detected result and sample annotation information, so as to update the model parameters based on the training loss to obtain a target fire detection model. The target fire detection model combines the rapid global detection capability of YOLOv11 and the fine local segmentation capability of U-net, and significantly improves the accuracy of fire area detection and contour extraction.

[0029] In an optional implementation, the YOLOv11 model includes a backbone module, a neck module and a head module;

[0030] The YOLOv11 model in the initial fire detection model is used to detect each enhanced sample image in the enhanced image set to obtain a corresponding first detection result, including:

[0031] The backbone module is used to extract first down-sampling features from the enhanced sample image;

[0032] The neck module is used to perform information fusion based on the first down-sampling features to obtain multi-scale features;

[0033] The head module is used to determine the first detection result based on the multi-scale features.

[0034] The efficient feature extraction of the backbone module, the multi-scale feature fusion of the neck module and the accurate prediction of the head module jointly act together, and the accuracy of fire area detection is significantly improved.

[0035] In an optional implementation, the U-net model includes an encoder and a decoder;

[0036] The U-net model in the initial fire detection model is used to detect each target sample image to obtain a corresponding second detection result, including:

[0037] The encoder is used to perform convolution and pooling on the target sample image to obtain a second down-sampling feature after each convolution and pooling;

[0038] The decoder is used to perform deconvolution on each second down-sampling feature to obtain a corresponding deconvolution feature map;

[0039] Each second down-sampling feature in the encoder and each deconvolution feature map in the decoder are connected by jumping and multi-level feature fusion to obtain a plurality of fusion feature maps;

[0040] The plurality of fusion feature maps are stacked to obtain a comprehensive feature map, and each pixel in the comprehensive feature map is classified to obtain a classification result;

[0041] Based on the classification result of the comprehensive feature map, a fire area mask of the target sample image is generated;

[0042] Based on the fire area mask, the fire contour and the bounding box of each fire area are extracted;

[0043] The bounding box and the fire contour of each fire area in the target sample image are taken as the second detection result.

[0044] The present application improves the accuracy of fire area detection by the efficient feature extraction of the encoder, the high-resolution feature recovery of the decoder, and the multi-level feature fusion of the jump connection.

[0045] In an optional embodiment, the backbone module includes a focusing module, a cross-stage partial network, and a variability convolution network;

[0046] The first down-sampling feature is extracted from the enhanced sample image by using the backbone module, including:

[0047] The first sample image feature is extracted from the enhanced sample image by using the focusing module;

[0048] The second sample image feature is extracted from the first sample image feature by using the cross-stage partial network;

[0049] The first down-sampling feature is extracted from the second sample image feature by using the variability convolution network.

[0050] The present application enables the backbone module to extract high-quality feature information layer by layer through the cooperative work of the focusing module, the cross-stage partial network, and the variability convolution network, providing a basis for subsequent detection tasks.

[0051] In an optional implementation, the neck module comprises a path aggregation network, a feature pyramid network, and a double attention mechanism.

[0052] With the neck module, information fusion is performed based on the first down-sampled features to obtain multi-scale features, including:

[0053] With the path aggregation network, multi-scale feature fusion is performed on the first down-sampled features to obtain first fused image features.

[0054] With the feature pyramid network, multi-scale feature fusion is performed on the first fused image features to obtain second fused image features.

[0055] With the double attention mechanism, the channel weights of the second fused image features are adjusted, and multi-scale features are generated.

[0056] Through the cooperative work of the path aggregation network, the feature pyramid network, and the double attention mechanism, the neck module can efficiently perform multi-scale feature fusion to generate high-quality multi-scale features, and the adaptability of the model to complex fire scenes is significantly improved, providing a high-quality feature representation basis for the fire detection task.

[0057] In an optional implementation, the head module comprises a classification branch and a regression branch.

[0058] With the head module, the first detection result is determined based on the multi-scale features, including:

[0059] With the classification branch, the fire probability of the enhanced sample image is determined based on the multi-scale features.

[0060] With the regression branch, the bounding box and the confidence score of the fire area in the enhanced sample image are determined based on the multi-scale features.

[0061] With the non-maximum suppression algorithm, redundant bounding boxes in the enhanced sample image are removed.

[0062] The fire probability of the enhanced sample image, the bounding box and the confidence score of each fire area are taken as the first detection result.

[0063] Through the cooperative work of the classification branch and the regression branch, combined with the optimization of the non-maximum suppression algorithm, the head module can accurately determine the fire probability and the position of the fire area, and precise and efficient fire detection is achieved.

[0064] In a second aspect, the present application provides a fire detection device based on YOLO-Unet, which comprises:

[0065] A first acquisition module is configured to acquire a set of sample images.

[0066] an enhancement module, configured to enhance the sample image set to obtain an enhanced image set;

[0067] a training module, configured to train an initial fire detection model based on the enhanced image set to obtain a target fire detection model, the initial fire detection model comprising a YOLOv11 model and a U-net model;

[0068] a second acquisition module, configured to acquire a to-be-detected image of a target region collected by a UAV;

[0069] a detection module, configured to detect the to-be-detected image by using the target fire detection model to obtain a fire detection result.

[0070] In a third aspect, the present application provides a computer device, comprising a memory and a processor, which are communicatively connected with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the fire detection method based on YOLO-Unet according to the first aspect or any one of the corresponding embodiments thereof.

[0071] In a fourth aspect, the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the fire detection method based on YOLO-Unet according to the first aspect or any one of the corresponding embodiments thereof. BRIEF DESCRIPTION OF DRAWINGS

[0072] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0073] Figure 1 is a flow chart of the fire detection method based on YOLO-Unet according to the embodiments of the present application;

[0074] Figure 2 is a work flow chart of the initial fire detection model according to the embodiments of the present application;

[0075] Figure 3 is a structural block diagram of the fire detection device based on YOLO-Unet according to the embodiments of the present application;

[0076] Figure 4 is a hardware structure schematic diagram of the computer device according to the embodiments of the present application. DETAILED DESCRIPTION

[0077] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0078] The existing fire detection model has the problems of poor adaptability to fire environment and inaccurate detection. The present application helps the model to understand the fire characteristics of the fire environment by obtaining a sample image set. The fire detection model used is composed of a YOLOv11 model and a U-net model, which realizes the integration of detection and segmentation, can quickly locate the fire area, provide the accurate shape thereof, and obtain accurate fire detection results.

[0079] According to the embodiments of the present application, a fire detection method based on YOLO-Unet is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0080] In the present embodiment, a fire detection method based on YOLO-Unet is provided, which can be used in terminals such as computers, etc. Figure 1 The flowchart of the fire detection method based on YOLO-Unet according to the embodiments of the present application is shown in FIG. 1, which includes the following steps: Figure 1

[0081] Step S101, obtaining a sample image set.

[0082] Specifically, the sample image set has a rich source, covering various terrains such as lakes, marshes, mudflats, meadows, etc., and covering fire images and non-fire images of different terrains. By obtaining the sample image set, a fire detection model suitable for various fire environments is constructed, and the sample image set is used for training to improve the generalization ability of the model.

[0083] Step S102, enhancing the sample image set to obtain an enhanced image set.

[0084] ​Specifically, the sample image set usually has problems of insufficient sample quantity and uneven image quality. If such a sample image set is directly used for model training, the trained model may have weak generalization ability and low detection accuracy. Therefore, by enhancing the sample image set, the diversity of samples can be greatly increased, the quality of images can be improved, more abundant and representative samples can be provided for subsequent model training, and the training effect can be effectively improved.

[0085] In step S103, the initial fire detection model is trained based on the enhanced image set to obtain a target fire detection model, and the initial fire detection model includes a YOLOv11 model and a U-net model.

[0086] Specifically, the fire detection model in the related art generally adopts Faster R-CNN or YOLOv5, etc., and has limitations in detecting fire, such as insufficient integration of detection and segmentation, poor adaptability to fire environment, difficulty in identifying underground fire, delay of diffusion phenomenon, inability to accurately provide shape information of the fire area, and the like, which is not conducive to the prediction of the fire expansion trend. Therefore, the embodiment of the present application adopts a combination of the YOLOv11 model and the U-net model to constitute the initial fire detection model. Through this combination, the global positioning provided by the YOLOv11 model and the local details of the U-Net model complement each other, which can effectively process fire features in complex scenes, and the U-Net model can help verify the continuity of the fire area and exclude isolated bright spots such as water surface reflection, thereby reducing false positives and improving the accuracy and reliability of fire detection.

[0087] In step S104, a to-be-detected image of a target area collected by a UAV is obtained.

[0088] Specifically, traditional manual inspection cannot obtain fast and comprehensive images. By mounting a high-definition camera or an infrared sensor on a UAV, fast cruising and real-time transmission of the to-be-detected image of the target area can be achieved, thereby effectively improving the efficiency and accuracy of fire detection.

[0089] In step S105, the to-be-detected image is detected by using the target fire detection model to obtain a fire detection result.

[0090] Specifically, the to-be-detected image is detected by using the trained target fire detection model, and the final fire detection result not only includes whether there is a fire area in the image, but also accurately gives the bounding box information of the fire area, which is conducive to more comprehensively understanding the fire situation and providing a reliable basis for fire situation analysis and emergency response.

[0091] The application obtains a sample image set and performs data enhancement on the sample image set, uses the enhanced image for model training, helps the model to understand the fire characteristics of the fire environment, and the fire detection model is composed of a YOLOv11 model and a U-net model, realizes the integration of detection and segmentation, can quickly locate the fire area, and can provide the accurate shape thereof, when actual fire detection is performed, the image to be detected collected by the unmanned aerial vehicle is input into the target fire detection model, accurate fire detection results can be obtained, and important basis is provided for subsequent fire situation analysis, emergency response and prevention and control decision.

[0092] In the embodiment, a fire detection method based on YOLO-Unet is provided, which can be used in the terminal described above, and the method specifically includes the following steps:

[0093] In step S201, a sample image set is obtained. For details, refer to Figure 1 The step S101 of the embodiment is not described here again.

[0094] In step S202, the sample image set is enhanced to obtain an enhanced image set.

[0095] Specifically, the step S202 includes the following steps:

[0096] In step S2021, for each sample image in the sample image set, the sample image is randomly cropped to obtain a plurality of cropped sample images.

[0097] Specifically, considering that in an actual fire scene, the fire area can appear at any position, random cropping can help simulate this phenomenon, thereby enhancing the generalization ability of the model. For each sample image, a plurality of starting positions of the cropping area are randomly generated, thereby simulating the observation from different angles and positions. Optionally, the cropping area is generally a rectangle, the starting position of which is located at the upper left corner of the rectangle, and the size of the rectangle is determined by the preset height and width . Assuming that the sample image is , where is the height of the sample image, is the width of the sample image, and is the number of channels of the sample image. For an RGB image, the number of channels is 3, and the cropped image can be represented by the following formula (1).

[0098] (1)

[0099] wherein, represents the cropped sample image; represents the upper left corner coordinate of the cropping area, and in order to ensure that it is located in the sample image, it needs to satisfy the constraint:​ ; represents the number of channels; represents the height of the cropped region; represents the width of the cropped region.

[0100] In step S2022, for each cropped sample image, the cropped sample image is randomly scaled to obtain a scaled sample image.

[0101] Specifically, in a real fire scene, the size of the fire region in the image will vary due to the distance from the observation point. For example, a fire region at close range usually appears larger in the image, while a fire region at a long distance appears smaller. By adjusting the size of the cropped sample image, the change of the fire region at different distances is simulated. More specifically, for each cropped sample image, a scaling factor is randomly selected from a pre-set scaling factor range (such as 0.8-1.2), and the cropped sample image is scaled to a corresponding size to obtain a scaled sample image. By random scaling, the sample diversity is increased, which helps to improve the recognition ability of the model for target objects of different scales.

[0102] In step S2023, all the scaled sample images and their sample label information are mixed and enhanced to obtain a plurality of enhanced sample images and their enhanced label information, forming an enhanced image set.

[0103] In some optional embodiments, the above step S2023 comprises:

[0104] In step S20231, all the scaled sample images are divided into a first sample set and a second sample set according to a pre-set proportion.

[0105] Specifically, since the number of non-fire images in the sample image set is much larger than that of fire images, in order to make full use of limited fire samples and balance the data distribution, the scaled sample images are divided into two sample sets according to a pre-set proportion (such as 7:3), and different mixed enhancement methods are used for processing, so as to ensure the diversity of the overall data and the robustness of the model to complex scenes.

[0106] In step S20232, the MixUp interpolation method is used to interpolate any two scaled sample images and their sample label information in the first sample set to obtain corresponding enhanced sample images and their enhanced label information.

[0107] Specifically, MixUp is a linear interpolation-based method that can not only fuse images but also interpolate their annotation information to simulate mixed scenes between different target objects. Each image in the sample image set carries sample annotation information, which not only explicitly labels whether there is a fire area in the image but also accurately circumscribes the fire area using a bounding box, including key parts such as flames and smoke. For the fire area, MixUp can effectively simulate the mixing of the fire area with other non-fire areas, such as the overlap of the fire area with plant, land, sky, and other areas, thereby helping the model learn the features of the fire area in complex environments. Optionally, the following equation (2) is the mixed augmented sample image, and the following equation (3) is the mixed augmented annotation information.

[0108] (2)

[0109] (3)

[0110] wherein, represents an augmented sample image; represents a mixing coefficient, which is usually randomly sampled from a Beta distribution to control the mixing degree. A larger will make closer to 0 or 1, resulting in greater image and annotation information differences. A smaller will make tend to 0.5, thereby making the mixing of the two images and annotation information more balanced; and represent any two scaled sample images; represents the mixed augmented annotation information; and represent the sample annotation information corresponding to the two scaled sample images, respectively.

[0111] Step S20233, using the CutMix interpolation method, interpolates any two scaled sample images and their sample annotation information in the second sample set to obtain the corresponding augmented sample image and its augmented annotation information.

[0112] Specifically, CutMix generates new samples by replacing a part of the region of one image with the corresponding region of another image to simulate local mixed scenes between target objects, such as the local mixing of fire areas with other areas. Optionally, the following equation (4) is the mixed augmented sample image, and the mixed augmented annotation information is obtained by referring to the above equation (3).

[0113] (4)

[0114] wherein, represents an enhanced sample image; represents the top-left corner coordinate of the cropped region; represents the number of channels; represents the height of the cropped region; represents the width of the cropped region; and represents any two scaled sample images.

[0115] In step S203, the initial fire detection model is trained based on the enhanced image set to obtain a target fire detection model, and the initial fire detection model includes a YOLOv11 model and a U-net model.

[0116] Specifically, the above step S203 includes:

[0117] In step S2031, the YOLOv11 model in the initial fire detection model is used to detect each enhanced sample image in the enhanced image set to obtain a corresponding first detection result, and the YOLOv11 model includes a backbone module, a neck module and a head module.

[0118] In some optional embodiments, the above step S2031 includes:

[0119] In step S20311, the first down-sampling feature is extracted from the enhanced sample image by using the backbone module, and the backbone module includes a focus module, a cross-stage partial network and a variability convolution network.

[0120] In some optional embodiments, the above step S20311 includes:

[0121] In step a1, the first sample image feature is extracted from the enhanced sample image by using the focus module.

[0122] Specifically, the focus module is an important component of the backbone module of the YOLOv11 model. In the fire detection scene, the enhanced sample image contains a large amount of background information, and the fire region may only occupy a small part of the image. The focus module can accurately locate the potential fire region in the complex image background and extract the first sample image feature related thereto, thereby reducing the interference of redundant background on the model judgment.

[0123] In step a2, the second sample image feature is extracted from the first sample image feature by using the cross-stage partial network.

[0124] Specifically, the Cross Stage Partial Network (CSP) plays a key role in optimizing feature extraction in the backbone module. The first sample image features contain information at different levels and dimensions. The CSP further processes these features through a specific network structure and processing method, effectively fusing features at different stages, preserving the detailed information in low-level features, and combining the semantic information of high-level features, thereby extracting more rich and representative second sample image features from the first sample image features, which helps to enhance the model's understanding and grasp of fire region features and improve the model's ability to detect fires in complex environments.

[0125] Step a3, using a deformable convolutional network to extract first down-sampling features from the second sample image features.

[0126] Specifically, the Deformable Convolutional Network Version 4 (DCNv4) is one of the core technologies of the backbone module to deal with complex backgrounds. In fire environments, complex background factors such as water surface reflection and vegetation interlacing can interfere with the detection of fire regions. Traditional convolutional operations use fixed convolutional kernel shapes and positions, making it difficult to adapt to target deformation and complex background changes. The DCNv4 allows the shape and position of the convolutional kernel to be dynamically adjusted according to the content of the second sample image features by introducing deformable convolution. The deformable convolutional network obtains the offset of the convolutional kernel position shown in equation (5) through an additional offset prediction network, allowing the convolutional kernel to move and deform freely on the second sample image features, thereby better capturing the local features of the target and the context information of the complex background. In this process, the second sample image features are down-sampled to obtain the first down-sampling features, significantly improving the model's detection accuracy and robustness for fire regions in complex backgrounds, and providing high-quality feature data for subsequent fire detection tasks of the model.

[0127] (5)

[0128] wherein, and denote the offset of the convolutional kernel position in the horizontal and vertical directions; denotes each position in the second sample image features; denotes the offset prediction network, such as an activation function; and denote the offset of the second sample image features in the horizontal and vertical directions.

[0129] In step S20312, a neck module is adopted to perform information fusion based on the first down-sampled features to obtain multi-scale features, and the neck module includes a path aggregation network, a feature pyramid network, and a double attention mechanism.

[0130] In some optional embodiments, step S20312 includes:

[0131] In step b1, a path aggregation network is adopted to perform multi-scale feature fusion on the first down-sampled features to obtain first fused image features.

[0132] Specifically, the path aggregation network (PANet) plays a key role in the neck module. The first down-sampled features contain information of different scales, but these information is relatively scattered. The role of PANet is to effectively integrate these multi-scale information through specific structure and algorithm. It uses a bottom-up path to fuse the features with rich details at the bottom layer and the features with strong semantic information at the high layer. In this process, different scale features complement each other, so that the model can pay attention to both the details and the overall characteristics of the fire area. For example, small-scale features can help identify the details of small fire sources, while large-scale features can help grasp the overall situation of large-area fires. Through this fusion method, the first fused image features obtained integrate the advantages of different scales, providing more comprehensive information for subsequent processing and improving the model's detection ability for different scale fires.

[0133] In step b2, a feature pyramid network is adopted to perform multi-scale feature fusion on the first fused image features to obtain second fused image features.

[0134] Specifically, the feature pyramid network (FPN) is a classic multi-scale feature fusion method that further optimizes the first fused image features. FPN constructs a multi-scale feature pyramid, starting from the high-resolution feature map at the top of the network, and performs upsampling layer by layer. The up-sampled high-level feature map is combined with the feature map from the low layer through horizontal connection to realize the re-fusion of features at different levels. It can utilize the semantic information of high-level feature maps and retain the spatial detail information of low-level feature maps. In fire detection, fire areas may appear at different scales, and FPN can capture these multi-scale information to enhance the detection ability of low-visibility fires such as underground fire sources or thin smoke. Through the processing of FPN, the second fused image features obtained are more perfect in the fusion of multi-scale information, providing stronger support for accurate detection of fire areas.

[0135] Step b3, a dual attention mechanism is used to adjust the channel weight of the second fused image feature and generate a multi-scale feature.

[0136] Specifically, the dual attention mechanism combines channel attention and spatial attention to further optimize the second fused image feature to generate more targeted multi-scale features. In terms of channel attention, the feature representation of each channel is obtained by calculating the global average pooling value of each channel as shown in equation (6) below, and then an attention vector is generated by an activation function as shown in equation (7) below, dynamically adjusting the weight of each channel, which can enhance the attention to the channel containing important fire information, suppress the influence of irrelevant channels, and highlight the key features of the fire area.

[0137] (6)

[0138] wherein, represents the feature representation of each channel; represents the height of the second fused image; represents the height of the second fused image; represents the channel of the second fused image; represents the feature map value at position .

[0139] (7)

[0140] wherein, represents the attention vector; represents an activation function such as sigmoid; represents a learned weight matrix; represents the feature representation of each channel.

[0141] In terms of spatial attention, the average pooling and maximum pooling in the channel dimension of the feature map are calculated to generate channel statistics, and then a spatial weight map is generated by an activation function such as a sigmoid function as shown in equation (8) below, dynamically adjusting the weight of each spatial position, thereby enhancing the attention to the important spatial position where the fire area is located. The dual attention mechanism comprehensively captures the global and local information of the feature map, enabling the model to focus more accurately on the fire area, and the generated multi-scale feature can be more effectively used for fire area detection, significantly improving the model's detection ability of fire areas in complex scenes.

[0142] (8)

[0143] wherein, represents the spatial weight map; represents an activation function such as sigmoid; represents a learned spatial weight matrix; represents an average pooling feature representation; represents a max pooling feature representation; represents a concatenation operation.

[0144] At step S20313, a first detection result is determined based on the multi-scale feature by using a head module, the head module including a classification branch and a regression branch.

[0145] In some optional embodiments, the step S20313 includes:

[0146] At step c1, a fire probability of the enhanced sample image is determined based on the multi-scale feature by using the classification branch.

[0147] Specifically, in the head module, the classification branch is used to determine whether a fire exists in the enhanced sample image and to determine a fire probability. The classification branch receives the multi-scale feature, which integrates information of different levels and scales and contains rich clues about the content of the image. Through a series of neural network structures such as convolutional layers and fully connected layers, the multi-scale feature is deeply analyzed and processed, the correlation pattern between different feature combinations and the existence of a fire is learned, and the multi-scale feature is mapped to a probability value, which represents the possibility of the existence of a fire in the image. For example, when the model learns that the typical features of flames and smoke (such as specific colors, textures, shapes, etc.) appear in a certain combination in the multi-scale feature, the predicted fire probability is increased; on the contrary, if these key features are not detected, the fire probability is reduced.

[0148] At step c2, a bounding box of a fire region in the enhanced sample image and a confidence score are determined based on the multi-scale feature by using the regression branch.

[0149] Specifically, the regression branch is also based on the multi-scale feature of the neck module, and the multi-scale feature is processed through a specific network structure to predict the bounding box coordinates of the fire region. The bounding box coordinates define a rectangular region, which is considered as a region that may contain a fire. At the same time, the regression branch also generates a confidence score for each predicted bounding box. This score reflects the degree of confidence of the model that a fire indeed exists within the bounding box, and the higher the score, the greater the possibility that the model considers that a fire exists within the bounding box.

[0150] At step c3, a non-maximum suppression algorithm is used to remove redundant bounding boxes in the enhanced sample image.

[0151] Specifically, after being processed by the classification branch and the regression branch, the model can predict multiple bounding boxes, some of which can have overlaps or redundancies, that is, multiple bounding boxes can point to the same fire area. The Non-Maximum Suppression (NMS) algorithm can remove these redundant bounding boxes and only keep one or several optimal bounding boxes to improve the accuracy and efficiency of the detection result. More specifically, the NMS algorithm sorts all predicted bounding boxes according to the confidence score of each bounding box and selects the bounding box with the highest confidence score as the current retained box. Then, the overlap between other bounding boxes and the retained box is calculated. If the overlap between a bounding box and the retained box exceeds a preset threshold (for example, 0.5), the bounding box is considered redundant and is removed. By repeatedly processing all bounding boxes in this way, the final result is a set of bounding boxes that have been filtered and removed of redundancies, avoiding repeated detection of the same fire area and improving the accuracy and practicality of fire detection.

[0152] Step c4, taking the fire probability of the enhanced sample image, the bounding box of each fire area, and the confidence score as the first detection result.

[0153] Specifically, the bounding box of each fire area after removing the redundant bounding box and its confidence score are taken together with the fire probability as the first detection result to achieve preliminary detection of the fire area.

[0154] Step S2032, performing confidence screening based on the first detection result of each enhanced sample image to obtain multiple target sample images.

[0155] Specifically, since the detection result can have false positives, not all detection results are accurate and reliable, so it is necessary to screen based on the confidence score. A suitable confidence threshold is set, and the confidence score in each first detection result is compared with the threshold one by one. The image corresponding to the first detection result whose confidence score reaches the threshold is retained, that is, the target sample image, as the input of the subsequent U-net model to further accurately segment the fire area and improve the accuracy of fire detection.

[0156] Step S2033, using the U-net model in the initial fire detection model to detect each target sample image to obtain the corresponding second detection result, and the U-net model includes an encoder and a decoder.

[0157] Specifically, the above step S2033 includes:

[0158] Step S20331, using the encoder to perform convolution and pooling on the target sample image to obtain the second down-sampling feature after each convolution and pooling.

[0159] Specifically, in the U-net model, the encoder performs preliminary feature extraction and dimension reduction on the target sample image. Convolution operation extracts various features in the image, such as texture, color, shape, etc., by sliding on the image using different convolution kernels, capturing different local patterns in the image. The pooling operation is used to reduce the resolution and reduce the amount of data while retaining the main features. After multiple convolution and pooling operations on the target sample image, the corresponding second down-sampling feature is obtained, which contains more representative and discriminative features in the target sample image, providing an important data basis for subsequent decoder operations.

[0160] In step S20332, the decoder is used to perform deconvolution on each second down-sampling feature to obtain the corresponding deconvolution feature map.

[0161] Specifically, the decoder corresponds to the encoder and is used to restore the resolution and accurately segment the fire area. Deconvolution expands and reconstructs the second down-sampling feature by increasing the size of the feature map, gradually restoring the detailed information of the image.

[0162] In step S20333, each second down-sampling feature in the encoder and each deconvolution feature map in the decoder are connected by jumping and multi-level feature fusion to obtain multiple fusion feature maps.

[0163] Specifically, underground fire spread or local low-temperature combustion may exhibit irregular morphology, while the diffusion of smoke and steam may make the fire source unclear. Jump connection can help the model retain these detailed information, thereby improving the segmentation accuracy and reducing the risk of misidentification when dealing with complex fire morphology. Multi-level feature fusion introduces deconvolution feature maps from different encoder layers between each convolution layer in the decoder in proportion, fuses low-level detailed features and high-level semantic features, as shown in equation (9), to obtain fusion feature maps containing both low-level details and high-level semantics.

[0164] (9)

[0165] wherein, denotes the fusion feature map of the i-th convolution layer after fusion; denotes the fusion feature map of the i-th convolution layer after fusion; denotes the hyperparameter used to control the fusion ratio between low-level features and high-level features. By adjusting , the model can find the best balance between different levels of features, using low-level features to capture details and high-level features to obtain global semantic information; denotes the second down-sampling feature obtained by the i-th convolution layer in the encoder; denotes the second down-sampling feature obtained by the i-th convolution layer in the encoder; denotes the deconvolution feature map of the i-th convolution layer in the decoder. a de-convolutional feature map corresponding to the convolutional layer.

[0166] At step S20334, the plurality of fusion feature maps are stacked to obtain a comprehensive feature map, and each pixel in the comprehensive feature map is classified to obtain a classification result.

[0167] Specifically, the plurality of fusion feature maps are spliced in the channel dimension, or different levels of feature maps are aggregated into a comprehensive feature map through convolution. Then, the comprehensive feature map can be classified by using a convolutional layer and an activation function to obtain a classification result of each pixel. The classification result is used to represent the probability that the pixel belongs to a fire area.

[0168] At step S20335, a fire area mask of the target sample image is generated based on the classification result of the comprehensive feature map.

[0169] Specifically, masks are set for fire areas and non-fire areas, for example, 1 for fire areas and 0 for non-fire areas. At the same time, a preset threshold is set, and pixels with a classification result exceeding the preset threshold are marked as 1 to represent that the pixel belongs to a fire area, and pixels below the preset threshold are marked as 0 to represent that the pixel belongs to a non-fire area, thereby obtaining a binary mask of the target sample image, i.e., a fire area mask.

[0170] At step S20336, a fire contour and a bounding box of each fire area are extracted based on the fire area mask.

[0171] Specifically, an image processing algorithm such as a Canny edge detection algorithm is used to perform connected component analysis on the fire area mask to extract the edge contour of the fire area as the fire contour. The minimum circumscribed rectangle of each connected component is calculated as the bounding box, and the coordinates thereof are determined. Optionally, an area threshold is set, and the area of the region enclosed by each bounding box is compared with the area threshold to remove the bounding box smaller than the area threshold.

[0172] At step S20337, the bounding box and the fire contour of each fire area in the target sample image are taken as the second detection result of the target sample image.

[0173] Specifically, the bounding box and the fire contour of each fire area in the target sample image are taken as the second detection result of the target sample image to achieve accurate detection of the fire area.

[0174] At step S2034, a first training loss is calculated based on the first detection result of each augmented sample image and the augmented annotation information, and a second training loss is calculated based on the second detection result of each target sample image and the augmented annotation information.

[0175] Specifically, the first detection result is generated by the YOLOv11 model, containing fire probability, bounding box coordinates and confidence score. The first detection result is compared with the augmented annotation information, and the first training loss is calculated using a suitable loss function (such as cross-entropy loss function for classification task, mean square error loss function for regression task). The second detection result is generated by the U-net model, and the second training loss is calculated by comparing the second detection result with the augmented annotation information using the corresponding loss function (such as Dice loss function commonly used for image segmentation task). The two training losses are calculated to evaluate the performance of the model at different stages, providing a basis for subsequent model parameter update, so that the model can better fit the real data and improve the accuracy of fire detection.

[0176] In step S2035, the model parameters of the initial fire detection model are updated based on the first training loss and the second training loss to obtain a target fire detection model.

[0177] Specifically, the direction and step of parameter update are determined by calculating the gradient of the loss function with respect to the model parameters. In each iteration, the parameters of the model are adjusted according to the gradient information, so that the first training loss and the second training loss gradually decrease. As the training continues, the detection and segmentation ability of the model for fire area gradually improves. When the training of the model reaches a certain stopping condition, the training ends to obtain the target fire detection model, which has higher accuracy and robustness in the fire detection task and can be better applied to actual fire monitoring scenarios.

[0178] In some optional embodiments, Figure 2 is a workflow diagram of the initial fire detection model according to an embodiment of the present application, as shown in Figure 2As shown, first, data processing is performed on the sample image set, random cropping, random scaling and data enhancement are performed, and an enhanced data set is obtained. The initial fire detection model includes a YOLOv11 model and a U-net model. First, the enhanced data set is input to the backbone module of the YOLOv11 model, the first sample image features are extracted through the focusing module, the second sample image features are extracted through the cross-stage partial network, and finally the first down-sampling features are obtained through the variable convolution network by dynamically adjusting the features, and input to the neck module. First, the first fused image features are obtained by passing through the path aggregation network, and then the second fused image features are obtained by passing through the feature pyramid network again, and finally the multi-scale features are obtained by adjusting through the double attention mechanism, and input to the head module. First, the fire probability is obtained through the classification branch, and then the boundary box coordinates and confidence score of the fire area are determined through the regression branch, and the redundant processing is performed through the non-maximum suppression, and finally the first detection result is obtained. The target sample image input to the U-net model is obtained by screening according to the confidence. First, the second down-sampling features are obtained through convolution and pooling by the encoder, and input to the decoder. The decoder obtains the deconvolution feature map through deconvolution. The encoder and the decoder are connected through the jump connection and the multi-level feature fusion, and the second detection result is obtained.

[0179] In step S204, the image to be detected of the target area collected by the unmanned aerial vehicle is obtained. For details, please refer to Figure 1 The step S104 of the embodiment shown will not be repeated here.

[0180] In step S205, the target fire detection model is used to detect the image to be detected, and the fire detection result is obtained. For details, please refer to Figure 1 The step S105 of the embodiment shown will not be repeated here.

[0181] The present application obtains a sample image set and performs data enhancement on it, uses the enhanced image to train the model, which helps the model to understand the fire characteristics of the fire environment. The fire detection model is composed of a YOLOv11 model and a U-net model, which realizes the integration of detection and segmentation, can quickly locate the fire area, and can provide the accurate shape. When performing actual fire detection, the image to be detected collected by the unmanned aerial vehicle is input into the target fire detection model, and an accurate fire detection result can be obtained, which provides an important basis for subsequent fire situation analysis, emergency response and prevention and control decision.

[0182] A YOLO-Unet-based fire detection device is also provided in the embodiment, which is used to implement the above-mentioned embodiments and preferred embodiments and has been described above. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware or a combination of software and hardware is also possible and contemplated.

[0183] The embodiment provides a YOLO-Unet-based fire detection device, which comprises: Figure 3 as shown in the figure, comprising:

[0184] The first acquisition module 301 is configured to acquire a sample image set.

[0185] The enhancement module 302 is configured to enhance the sample image set to obtain an enhanced image set.

[0186] The training module 303 is configured to train an initial fire detection model based on the enhanced image set to obtain a target fire detection model, wherein the initial fire detection model comprises a YOLOv11 model and a U-net model.

[0187] The second acquisition module 304 is configured to acquire a to-be-detected image of a target region collected by a UAV.

[0188] The detection module 305 is configured to detect the to-be-detected image by using the target fire detection model to obtain a fire detection result.

[0189] In some optional embodiments, the sample image set carries sample annotation information.

[0190] The enhancement module 302 comprises:

[0191] The cropping unit is configured to randomly crop each sample image in the sample image set to obtain a plurality of cropped sample images.

[0192] The scaling unit is configured to randomly scale each cropped sample image to obtain a scaled sample image.

[0193] The enhancement unit is configured to perform mixed enhancement on all scaled sample images and their sample annotation information to obtain a plurality of enhanced sample images and their enhanced annotation information, forming the enhanced image set.

[0194] In some optional embodiments, the enhancement unit comprises:

[0195] The division subunit is configured to divide all scaled sample images into a first sample set and a second sample set according to a preset ratio.

[0196] The first enhancement subunit is configured to perform interpolation on any two scaled sample images and sample label information of the first sample set by using a MixUp interpolation method to obtain corresponding enhanced sample images and enhanced label information.

[0197] The second enhancement subunit is configured to perform interpolation on any two scaled sample images and sample label information of the second sample set by using a CutMix interpolation method to obtain corresponding enhanced sample images and enhanced label information.

[0198] In some optional embodiments, the training module 303 comprises:

[0199] The first detection unit is configured to perform detection on each enhanced sample image in the enhanced image set by using a YOLOv11 model in the initial fire detection model to obtain a corresponding first detection result.

[0200] The screening unit is configured to perform confidence screening based on the first detection result of each enhanced sample image to obtain a plurality of target sample images.

[0201] The second detection unit is configured to perform detection on each target sample image by using a U-net model in the initial fire detection model to obtain a corresponding second detection result.

[0202] The calculation unit is configured to calculate a first training loss based on the first detection result and the enhanced label information of each enhanced sample image, and calculate a second training loss based on the second detection result and the enhanced label information of each target sample image.

[0203] The training unit is configured to update the model parameters of the initial fire detection model based on the first training loss and the second training loss to obtain a target fire detection model.

[0204] In some optional embodiments, the YOLOv11 model comprises a backbone module, a neck module, and a head module.

[0205] The first detection unit comprises:

[0206] The first extraction subunit is configured to extract a first down-sampling feature from the enhanced sample image by using the backbone module.

[0207] The first fusion subunit is configured to perform information fusion based on the first down-sampling feature by using the neck module to obtain a multi-scale feature.

[0208] The first determination subunit is configured to determine the first detection result based on the multi-scale feature by using the head module.

[0209] In some optional embodiments, the U-net model comprises an encoder and a decoder.

[0210] The second detection unit comprises:

[0211] The second extraction subunit is configured to adopt the encoder to perform convolution and pooling on the target sample image to obtain second down-sampling features after each convolution and pooling.

[0212] The deconvolution subunit is configured to adopt the decoder to perform deconvolution on each second down-sampling feature to obtain a corresponding deconvolution feature map.

[0213] The second fusion subunit is configured to perform skip connection and multi-level feature fusion on each second down-sampling feature in the encoder and each deconvolution feature map in the decoder to obtain a plurality of fusion feature maps.

[0214] The classification subunit is configured to stack the plurality of fusion feature maps to obtain a comprehensive feature map, and perform classification on each pixel in the comprehensive feature map to obtain a classification result.

[0215] The generation subunit is configured to generate a fire region mask of the target sample image based on the classification result of the comprehensive feature map.

[0216] The third extraction subunit is configured to extract a fire contour and a bounding box of each fire region based on the fire region mask.

[0217] The second determination subunit is configured to take the bounding box and the fire contour of each fire region in the target sample image as a second detection result.

[0218] In some optional embodiments, the backbone module comprises a focal module, a cross-stage partial network, and a malleable convolution network.

[0219] The first extraction subunit is specifically configured to extract a first sample image feature from the enhanced sample image by adopting the focal module, extract a second sample image feature from the first sample image feature by adopting the cross-stage partial network, and extract the first down-sampling feature from the second sample image feature by adopting the malleable convolution network.

[0220] In some optional embodiments, the neck module comprises a path aggregation network, a feature pyramid network, and a dual attention mechanism.

[0221] The first fusion subunit is specifically configured to perform multi-scale feature fusion on the first down-sampling feature by adopting the path aggregation network to obtain a first fused image feature, perform multi-scale feature fusion on the first fused image feature by adopting the feature pyramid network to obtain a second fused image feature, and adjust a channel weight of the second fused image feature by adopting the dual attention mechanism and generate a multi-scale feature.

[0222] In some alternative implementations, the head module includes a classification branch and a regression branch;

[0223] The first determining subunit is specifically used to determine the fire probability of the enhanced sample image by using a classification branch based on multi-scale features; to determine the bounding box and confidence score of the fire area in the enhanced sample image by using a regression branch based on multi-scale features; to remove redundant bounding boxes in the enhanced sample image by using a non-maximum suppression algorithm; and to take the fire probability of the enhanced sample image, the bounding box of each fire area, and the confidence score as the first detection result.

[0224] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0225] The fire detection device based on YOLO-Unet in this embodiment is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0226] This invention also provides a computer device having the above-described features. Figure 3 The fire detection device shown is based on YOLO-Unet.

[0227] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 4 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 4 Take a processor 10 as an example.

[0228] The processor 10 can be a central processing unit, a network processing unit, or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic device, a general array logic, or any combination thereof.

[0229] The memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform the methods illustrated in the above embodiments.

[0230] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system and application programs required by at least one function. The data storage area can store data created according to the use of the computer device, and the like. In addition, the memory 20 can include a high-speed random access memory, and can further include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some alternative embodiments, the memory 20 can optionally include a memory disposed remotely with respect to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0231] The memory 20 can include a volatile memory, such as a random access memory, and can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid state disk. The memory 20 can further include a combination of the above-mentioned types of memories.

[0232] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30, and the output device 40 can be connected through a bus or other means, Figure 4 For example, the connection through the bus is taken as an example.

[0233] The input device 30 can receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, and the like. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), a tactile feedback device (e.g., a vibration motor), and the like. The display device includes, but is not limited to, a liquid crystal display, a light emitting diode, a display, and a plasma display. In some alternative embodiments, the display device can be a touch screen.

[0234] The embodiments of the present application further provide a computer readable storage medium, and the method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded to a local storage medium through network, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0235] Part of the present application can be applied as a computer program product, for example, computer program instructions, when executed by a computer, the operation of the computer can invoke or provide the method and / or technical solutions according to the present application. Those skilled in the art should understand that the form of computer program instructions in computer readable medium includes but is not limited to source file, executable file, installation package file, etc. Correspondingly, the way of computer program instructions executed by computer includes but is not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.

[0236] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A fire detection method based on YOLO-Unet, characterized in that, The method comprises: acquiring a sample image set carrying sample annotation information, including fire images and non-fire images of various terrains; enhancing the sample image set to obtain an enhanced image set; training an initial fire detection model based on the enhanced image set to obtain a target fire detection model, the initial fire detection model comprising a YOLOv11 model and a U-net model; acquiring a target region image to be detected collected by a UAV; detecting the target region image to be detected using the target fire detection model to obtain a fire detection result; wherein the enhancement of the sample image set to obtain the enhanced image set comprises: randomly cropping each sample image in the sample image set to obtain a plurality of cropped sample images; randomly scaling each cropped sample image to obtain a scaled sample image; mixing and enhancing all scaled sample images and their sample annotation information to obtain a plurality of enhanced sample images and their enhanced annotation information, forming the enhanced image set; the training of the initial fire detection model based on the enhanced image set to obtain the target fire detection model comprises: detecting each enhanced sample image in the enhanced image set using the YOLOv11 model in the initial fire detection model to obtain a corresponding first detection result; confidence screening based on the first detection result of each enhanced sample image to obtain a plurality of target sample images; detecting each target sample image using the U-net model in the initial fire detection model to obtain a corresponding second detection result; calculating a first training loss based on the first detection result and the enhanced annotation information of each enhanced sample image, and calculating a second training loss based on the second detection result and the enhanced annotation information of each target sample image; updating the model parameters of the initial fire detection model based on the first training loss and the second training loss to obtain the target fire detection model.

2. The method of claim 1, wherein, the mixing and enhancing of all scaled sample images and their sample annotation information to obtain a plurality of enhanced sample images and their enhanced annotation information, forming the enhanced image set, comprises: dividing all scaled sample images into a first sample set and a second sample set according to a preset ratio; using a MixUp interpolation method to interpolate any two scaled sample images and their sample annotation information in the first sample set to obtain corresponding enhanced sample images and their enhanced annotation information; using a CutMix interpolation method to interpolate any two scaled sample images and their sample annotation information in the second sample set to obtain corresponding enhanced sample images and their enhanced annotation information.

3. The method of claim 1, wherein, The YOLOv11 model comprises a backbone module, a neck module, and a head module; the detection of each enhanced sample image in the enhanced image set using the YOLOv11 model in the initial fire detection model to obtain a corresponding first detection result comprises: The backbone module is adopted to extract first down-sampling features from the enhanced sample image; The neck module is adopted to perform information fusion based on the first down-sampling features to obtain multi-scale features; The head module is adopted to determine the first detection result based on the multi-scale features.

4. The method of claim 1, wherein, The U-net model includes an encoder and a decoder; The U-net model in the initial fire detection model is adopted to detect each target sample image to obtain a corresponding second detection result, including: The encoder is adopted to perform convolution and pooling on the target sample image to obtain second down-sampling features after each convolution and pooling; The decoder is adopted to perform deconvolution on each second down-sampling feature to obtain a corresponding deconvolution feature map; Each second down-sampling feature in the encoder and each deconvolution feature map in the decoder are connected by jumping and multi-level feature fusion to obtain a plurality of fusion feature maps; The plurality of fusion feature maps are stacked to obtain a comprehensive feature map, and each pixel in the comprehensive feature map is classified to obtain a classification result; Based on the classification result of the comprehensive feature map, a fire region mask of the target sample image is generated; Based on the fire region mask, a fire contour and a bounding box of each fire region are extracted; The bounding box and the fire contour of each fire region in the target sample image are taken as the second detection result.

5. The method of claim 3, wherein, The backbone module includes a focus module, a cross-stage partial network and a variability convolution network; The backbone module is adopted to extract first down-sampling features from the enhanced sample image, including: The focus module is adopted to extract first sample image features from the enhanced sample image; The cross-stage partial network is adopted to extract second sample image features from the first sample image features; The variability convolution network is adopted to extract the first down-sampling features from the second sample image features.

6. The method of claim 3, wherein, The neck module includes a path aggregation network, a feature pyramid network and a double attention mechanism; The neck module is adopted to perform information fusion based on the first down-sampling features to obtain multi-scale features, including: The path aggregation network is adopted to perform multi-scale feature fusion on the first down-sampling features to obtain first fusion image features; The feature pyramid network is adopted to perform multi-scale feature fusion on the first fusion image features to obtain second fusion image features; The double attention mechanism is adopted to adjust the channel weight of the second fusion image features and generate the multi-scale features.

7. The method of claim 3, wherein, The head module includes a classification branch and a regression branch; The head module is adopted to determine the first detection result based on the multi-scale features, including: The classification branch is adopted to determine the fire probability of the enhanced sample image based on the multi-scale features; The regression branch is adopted to determine the bounding box and the confidence score of the fire region in the enhanced sample image based on the multi-scale features; A non-maximum suppression algorithm is adopted to remove redundant bounding boxes in the enhanced sample image; The fire probability of the enhanced sample image, the boundary box of each fire area, and the confidence score are taken as the first detection result.

8. A YOLO-Unet-based fire detection device, characterized in that, The device comprises: A first acquisition module is configured to acquire a sample image set, the sample image set carrying sample annotation information, including fire images and non-fire images of multiple terrains; An enhancement module is configured to enhance the sample image set to obtain an enhanced image set; A training module is configured to train an initial fire detection model based on the enhanced image set to obtain a target fire detection model, the initial fire detection model including a YOLOv11 model and a U-net model; A second acquisition module is configured to acquire a to-be-detected image of a target area collected by a UAV; A detection module is configured to detect the to-be-detected image by using the target fire detection model to obtain a fire detection result; The enhancement module is specifically configured to: randomly crop each sample image in the sample image set to obtain multiple cropped sample images; randomly scale each cropped sample image to obtain a scaled sample image; perform mixed enhancement on all scaled sample images and their sample annotation information to obtain multiple enhanced sample images and their enhanced annotation information, forming the enhanced image set; The training module is specifically configured to: detect each enhanced sample image in the enhanced image set by using the YOLOv11 model in the initial fire detection model to obtain a corresponding first detection result; perform confidence screening based on the first detection result of each enhanced sample image to obtain multiple target sample images; detect each target sample image by using the U-net model in the initial fire detection model to obtain a corresponding second detection result; calculate a first training loss based on the first detection result and the enhanced annotation information of each enhanced sample image, and calculate a second training loss based on the second detection result and the enhanced annotation information of each target sample image; update the model parameters of the initial fire detection model based on the first training loss and the second training loss to obtain the target fire detection model.

Images