A fire point detection method, device, system and storage medium based on multi-band infrared images

Abstract

This application relates to a method, apparatus, system, and storage medium for fire detection based on multi-band infrared images, and pertains to the field of image data processing technology. The method includes: acquiring multi-band infrared image data and preprocessing the multi-band infrared image data; employing a preset fire detection model to perform feature classification, extraction, and feature fusion on fire features in fire areas and background features in non-fire areas of the preprocessed multi-band infrared image data to obtain a target fire point segmentation probability map; wherein the preset fire detection model includes an encoder, a fire point segmentation decoder, a background reconstruction decoder, and a fusion module; and determining the fire point pixel detection result based on the target fire point segmentation probability map and a segmentation threshold.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, in particular to a fire point detection method, device and system based on multi-band infrared images and a storage medium. BACKGROUND

[0002] Satellite infrared remote sensing technology provides a key means for forest, grassland and other large-area fire monitoring. With the development of artificial intelligence, semantic segmentation models based on deep learning, such as U-Net model, have become the mainstream method for satellite infrared remote sensing fire point detection, which can realize end-to-end pixel-level recognition and significantly outperform traditional threshold method and machine learning method based on hand-crafted features in performance. However, the fire point detection method based on the model has the following two problems: first, the single-branch segmentation model lacks explicit modeling ability for interference backgrounds such as clouds and high-temperature ground, resulting in confusion between target features and background features and high false detection rate; second, the multi-scale down-sampling in the encoder-decoder structure dilutes the key features of small fire points at the pixel level, causing loss of small target information and prominent missing detection problem. These two problems restrict each other, making it difficult for the model to balance the precision and recall of the detection result in complex scenes, restricting the actual reliability of the automatic fire point monitoring system and greatly reducing the fire point detection accuracy. SUMMARY

[0003] The present application provides a fire point detection method, device, system and storage medium based on multi-band infrared images to at least solve the problem of low detection accuracy of small target fire points in complex backgrounds and susceptibility to background interference. The technical solution of the present application is as follows:

[0004] According to a first aspect of an embodiment of the present application, a fire point detection method based on multi-band infrared images is provided, which comprises: acquiring multi-band infrared image data and pre-processing the multi-band infrared image data; using a preset fire point detection model to perform feature classification extraction and feature fusion on the fire point features of the fire point region and the background features of the non-fire point region in the pre-processed multi-band infrared image data to obtain a target fire point segmentation probability map; wherein the feature classification extraction and feature fusion using the preset fire point detection model comprises: calling an encoder to extract multi-scale features of the pre-processed multi-band infrared image data to obtain a multi-scale feature map; calling a fire point segmentation decoder to perform preliminary segmentation on the fire point features of the multi-scale feature map to obtain a preliminary fire point segmentation probability map; calling a background reconstruction decoder to extract and reconstruct the background features of the background region in the multi-scale feature map to obtain a background region reconstruction probability map; calling a fusion module to perform difference fusion on the preliminary fire point segmentation probability map and the background region reconstruction probability map to obtain the target fire point segmentation probability map; and determining a fire point pixel detection result according to the target fire point segmentation probability map.

[0005] The fire point pixel detection result can be determined by threshold segmentation according to the target fire point segmentation probability map.

[0006] Specifically, threshold judgment is performed according to the target fire point segmentation probability map and the segmentation threshold, and pixels with a segmentation probability greater than or equal to the segmentation threshold are identified as fire point pixels, and pixels with a segmentation probability less than the segmentation threshold are identified as background pixels.

[0007] The preset fire point detection model includes a shared encoder, a fire point segmentation decoder, a background reconstruction decoder, and a fusion module.

[0008] The shared encoder is configured to extract multi-scale features of the preprocessed multi-band infrared image data to obtain a multi-scale feature map.

[0009] The fire point segmentation decoder is configured to preliminarily segment fire point features of the multi-scale feature map to obtain a preliminary fire point segmentation map.

[0010] The background reconstruction decoder is configured to extract and reconstruct background features of a background region in the multi-scale feature map to obtain a background region reconstruction map.

[0011] The fusion module is configured to perform differential feature fusion on the preliminary fire point segmentation map and the background region reconstruction map to obtain a target fire point segmentation map.

[0012] In an implementation manner, the encoder includes N down-sampling blocks and a first convolution block connected in series according to a preset bit sequence, the fire point segmentation decoder includes N first up-sampling blocks, the background reconstruction decoder includes (N+1) background feature transformation modules and N second up-sampling blocks connected in series, the first sub-output end of each down-sampling block is connected to the first sub-input end of a first up-sampling block in one-to-one correspondence, the second sub-output end of each down-sampling block is connected to the second sub-input end of a second up-sampling block through a background feature transformation module in one-to-one correspondence, the first output end of the first convolution block is connected to the third sub-input end of the first first up-sampling block, the second output end of the first convolution block is connected to the fourth sub-input end of the first second up-sampling block through a background feature transformation module, and the background feature transformation module is configured to model a local context dependency relationship of a non-fire point region.

[0013] In another implementation, the N first up-sampling blocks are connected in series in a preset bit sequence; the fire point segmentation decoder further comprises a second convolutional block connected to the first up-sampling output end of the Nth first up-sampling block, and the background reconstruction decoder further comprises a third convolutional block connected to the second up-sampling output end of the Nth second up-sampling block; the fusion module is called to perform difference fusion on the preliminary fire point segmentation probability map and the background region reconstruction probability map to obtain a target fire point segmentation probability map, including: performing a scaling operation on the background region reconstruction probability map output by the third convolutional block according to a first learnable scalar parameter, the first learnable scalar parameter being used to control the intensity of background suppression and being positively correlated with the background brightness; and performing element-by-element subtraction between the preliminary fire point segmentation probability map output by the second convolutional block and the background region reconstruction probability map after the scaling operation to obtain the target fire point segmentation probability map.

[0014] In another implementation, the N first up-sampling blocks are independently arranged in sequence according to a preset bit sequence; the fusion module comprises a fourth convolution block and N sub-fusion blocks independently arranged in sequence according to the preset bit sequence, and a sub-fusion output end of the Kth sub-fusion block is connected to a fifth sub-input end of the (K-1)th first up-sampling block, K being any positive integer in 1 to (N-1); a sub-fusion output end of the Nth sub-fusion block is connected to an input end of the fourth convolution block; the initial fire point segmentation probability map comprises first sub-fire point segmentation probability maps output by the first sub-sampling output ends of the first up-sampling blocks; the fire point segmentation decoder is called to preliminarily segment the fire point features of the multi-scale feature maps to obtain a preliminary fire point segmentation probability map, comprising: for any first up-sampling block, preliminarily segmenting the multi-scale feature map output by the down-sampling block connected to the first up-sampling block and the second sub-fire point segmentation probability map output by the sub-fusion output end connected to the first up-sampling block to obtain the first sub-fire point segmentation probability map output by the first sub-sampling output end of the first up-sampling block; the sub-fusion output end connected to any first up-sampling block is a sub-fusion output end of the sub-fusion block connected to the first up-sampling block; the fusion module is called to difference fuse the preliminary fire point segmentation probability map and the background region reconstruction probability map to obtain a target fire point segmentation probability map, comprising: sequentially calling each sub-fusion block to perform the following fusion operation according to the preset bit sequence until the Nth sub-fusion block is executed, and performing a convolution operation on the sub-fire point segmentation probability map output by the sub-fusion output end of the Nth sub-fusion block by using the fourth convolution block to obtain the target fire point segmentation probability map; wherein, the fusion operation performed by any sub-fusion block comprises: performing a scaling operation on the sub-reconstruction probability map output by the second sub-sampling output end connected to the sub-fusion block according to the second learnable scalar parameter, and performing an element-by-element subtraction between the first sub-fire point segmentation probability map output by the first sub-sampling output end connected to the sub-fusion block and the sub-reconstruction probability map after the scaling operation to obtain the second sub-fire point segmentation probability map output by the sub-fusion block.

[0015] In another implementation, the background reconstruction decoder is invoked to extract and reconstruct the background features of the background region in the multi-scale feature map to obtain a background region reconstruction probability map, including: invoking each background feature transformation module to perform the following reconstruction operation: dividing the input feature map input to each background feature transformation module into a plurality of non-overlapping local windows; performing multi-head self-attention calculation in each local window; sequentially performing layer normalization, feedforward neural network processing and residual connection on the results of the multi-head self-attention calculation to obtain each first reconstruction probability map; invoking a plurality of second upsampling blocks and third convolutional blocks to perform upsampling processing and convolution processing on each first reconstruction probability map to obtain the background region reconstruction probability map; wherein the input feature map of each background feature transformation module is the output of the first convolutional block connected to each background feature transformation module or the output of the downsampling block connected to each background feature transformation module.

[0016] In another implementation, the method further includes: training the model parameters of the initial preset fire point detection model based on a spatial weighted loss and a Tversky loss to obtain a trained preset fire point detection model; the spatial weighted loss is a weighted sum result of the prediction confidence of each output fire point pixel output by the model and the spatial weight of each output fire point pixel; wherein the prediction confidence represents the confidence of each output fire point pixel output by the initial preset fire point detection model in the prediction process relative to the actual fire point pixel in the pixel space; the spatial weight represents the attenuation degree of the area of the connected domain to which each output fire point pixel belongs in the image; the spatial weight of each output fire point pixel is negatively related to the area of the connected domain; the Tversky loss is a fire point loss rate determined based on the number of real fire points, the number of background fire points and the number of missed real fire points in each output fire point pixel.

[0017] The preset fire point detection model is trained based on the spatial weighted loss and the Tversky loss.

[0018] In another implementation, the preprocessing of the multi-band infrared image data includes: combining the data of a plurality of bands with a fire point sensitivity higher than a preset sensitivity in the multi-band infrared image data, and performing normalization and / or cropping operation processing on the combined multi-band infrared image data; determining the fire point pixel detection result according to the target fire point segmentation probability map, including: performing threshold segmentation and / or morphological filtering on the target fire point segmentation probability map to obtain the fire point pixel detection result.

[0019] The plurality of bands can be three bands of different spectral ranges.

[0020] For example, the data of bands 5, 6 and 7 in Landsat8 image are selected for band combination to obtain the combined multi-band infrared image data.

[0021] According to a second aspect of the present invention, a fire detection device based on multi-band infrared images is provided. The fire detection device based on multi-band infrared images includes: an acquisition unit configured to acquire multi-band infrared image data and preprocess the multi-band infrared image data.

[0022] The fire detection unit is configured to use a preset fire detection model to perform feature classification, extraction, and feature fusion on the fire features of the fire region and the background features of the non-fire region in the preprocessed multi-band infrared image data to obtain a target fire point segmentation probability map. The preset fire detection model includes an encoder, a fire point segmentation decoder, a background reconstruction decoder, and a fusion module. The encoder is used to extract multi-scale features from the preprocessed multi-band infrared image data to obtain a multi-scale feature map. The fire point segmentation decoder is used to perform preliminary segmentation on the fire features of the multi-scale feature map to obtain a preliminary fire point segmentation probability map. The background reconstruction decoder is used to extract and reconstruct the background features of the background region in the multi-scale feature map to obtain a background region reconstruction probability map. The fusion module is used to perform differential fusion on the preliminary fire point segmentation probability map and the background region reconstruction probability map to obtain the target fire point segmentation probability map. The determination unit is configured to determine the fire point pixel detection result based on the target fire point segmentation probability map.

[0023] The device is configured to perform the fire detection method based on multi-band infrared images, which is described in the first aspect above and any possible implementation thereof.

[0024] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which instructions are stored, such that when the instructions in the computer-readable storage medium are executed by a processor of a fire detection device based on multi-band infrared images, the fire detection device based on multi-band infrared images is able to perform a fire detection method based on multi-band infrared images as described in the first aspect and any possible implementation thereof.

[0025] According to a fourth aspect of the present disclosure, a computer program product is provided, the computer program product including computer instructions, which, when executed on a fire detection device based on multi-band infrared images, cause the fire detection device based on multi-band infrared images to perform the fire detection method based on multi-band infrared images described in the first aspect and any possible implementation thereof.

[0026] The technical solution provided by the embodiments of the present invention brings at least the following beneficial effects: The multi-band infrared image data on which this application is based provides rich spectral information of fire points and background, which is beneficial for distinguishing fire points from background. Based on this, an encoder is first used to extract multi-scale feature maps, which can capture fire point features and background features at different scales; then, a fire point segmentation decoder is used to specifically extract fire point features to obtain a preliminary fire point segmentation probability map, focusing on the segmentation of fire points to accurately extract fire point information; at the same time, a background reconstruction decoder is used to specifically extract background features and reconstruct the background region to obtain a background region reconstruction probability map to accurately extract background information. Furthermore, a fusion module performs differential fusion on the two probability maps to remove background information from the background region reconstruction probability map from the preliminary fire point segmentation probability map. This can suppress the response of the background region in fire point segmentation, thereby reducing background interference and highlighting the real fire point, so that the fire point pixel detection results obtained based on the target fire point segmentation probability map are more accurate, thereby reducing the probability of missed and false fire points.

[0027] Therefore, this application constructs a collaborative architecture comprising an encoder, a fire point segmentation decoder, a background reconstruction decoder, and a fusion module. Based on a differential fusion mechanism, it separates the target fire point from the background, avoiding semantic confusion in feature learning. This effectively identifies and suppresses interference from background information on real fire point information, reducing the false negative and false positive rates for fire point detection. Simultaneously, both the fire point segmentation decoder and the background reconstruction decoder receive features from each layer of the encoder via skip connections, ensuring that high-resolution detail features from lower layers are directly injected into the decoding process. This avoids information attenuation of small target features in a single decoding path. Therefore, through multi-scale feature extraction and a collaborative decoding architecture, the spatial details of small targets are recovered while preserving high-level semantics, thereby reducing the false negative rate for small target fire points.

[0028] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0029] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0030] Figure 1 This is a flowchart illustrating a fire detection method based on multi-band infrared images according to an exemplary embodiment;

[0031] Figure 2 This is a schematic diagram of the architecture of a fire detection model according to an exemplary embodiment. Figure 1 ;

[0032] Figure 3This is a schematic diagram of the architecture of a fire detection model according to an exemplary embodiment. Figure 2 ;

[0033] Figure 4 This is a block diagram of a fire detection device based on multi-band infrared images, according to an exemplary embodiment.

[0034] Figure 5 This is a schematic diagram of a fire detection device based on multi-band infrared images, according to an exemplary embodiment. Detailed Implementation

[0035] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0036] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0037] Before providing a detailed description of the fire detection method based on multi-band infrared images provided in the embodiments of this application, a brief introduction to the application scenarios and implementation environment involved in the embodiments of this application is given.

[0038] Current mainstream remote sensing fire detection methods are mainly divided into thresholding methods, traditional machine learning models (such as SVM and random forest), and deep learning models. It has been found that encoder-decoder structures, represented by U-Net, have achieved significant results in fire semantic segmentation tasks. For example, the U-Net model accompanying the ActiveFire dataset, trained end-to-end on Landsat-8 multispectral images using the standard U-Net architecture, achieves detection performance superior to traditional thresholding algorithms (such as Schroeder and Kumar-Roy), but still suffers from insufficient sensitivity to small fire spots and susceptibility to background interference from reflective urban buildings.

[0039] in U 2 Based on Net, a multi-level aggregation mechanism and a BCE+IoU hybrid loss function are introduced, achieving high mIoU and localization accuracy on datasets such as LC8FPS and ActiveFire-Oceania, especially in multi-fire points and irregularly shaped fire areas.

[0040] While the above methods can optimize performance, they still face the following problems: First, insufficient small target detection capability. Fire points in remote sensing images are usually pixel-level targets, making it difficult to capture their detailed information (for example, most fires occupy only 10 or fewer pixels in remote sensing images, while the image size is generally 65,535 pixels, so it is difficult to learn the detailed features of such very small fire point pixels). This easily leads to missed detections and false detections. Missed detections refer to situations where there is actually a fire point in the image, but the model fails to detect it; false detections refer to situations where there is actually no fire point in the image, but the model incorrectly detects it as a fire point. Second, weak background interference suppression. Bright clouds, heat sources, and other non-fire point areas are easily misjudged. The lack of a targeted background modeling mechanism easily leads to false detections of fire point pixels. Third, the class imbalance problem has not been effectively optimized. Traditional loss functions, such as Focal Loss, Dice Loss, and BCE Loss, are not good at focusing on difficult samples (small fire points). They only perform loss calculations at the single pixel level and fail to effectively consider the connectivity of the fire point area, which can easily lead to missed detection of fire point pixels.

[0041] To address the aforementioned issues, this application proposes a fire detection method based on multi-band infrared images. It utilizes a deep learning model with a constructed segmentation-reconstruction dual-decoding architecture, combined with a dual-branch output feature differential fusion mechanism and a spatially weighted loss function, to achieve high-precision detection of pixel-level fire points while effectively suppressing background interference such as bright clouds and heat sources. This technology is applicable to scenarios such as remote sensing fire monitoring and forest fire early warning, and especially under extreme class imbalance conditions (positive-to-negative sample ratio reaching 1:10^4), it can effectively improve the detection accuracy of small targets and the ability to model background patterns.

[0042] For ease of understanding, the fire detection method based on multi-band infrared images provided in this application will be described in detail below with reference to the accompanying drawings.

[0043] Figure 1 This is a flowchart illustrating a fire detection method based on multi-band infrared images according to an exemplary embodiment.

[0044] S11: Acquire multi-band infrared image data and preprocess the multi-band infrared image data.

[0045] The multi-band infrared image data on which it is based provides rich spectral information on the fire point and the background, which is helpful in distinguishing the fire point from the background.

[0046] S12, using a preset fire detection model, performs feature classification, extraction, and feature fusion on the fire features of the fire area and the background features of the non-fire area in the preprocessed multi-band infrared image data to obtain a target fire point segmentation probability map.

[0047] The process involves using a pre-defined fire detection model for feature classification, extraction, and fusion. This includes: calling the encoder to extract multi-scale features from the pre-processed multi-band infrared image data to obtain a multi-scale feature map; calling the fire segmentation decoder to perform preliminary segmentation of the fire features in the multi-scale feature map to obtain a preliminary fire segmentation probability map; calling the background reconstruction decoder to extract and reconstruct the background features in the background region of the multi-scale feature map to obtain a background region reconstruction probability map; and calling the fusion module to perform differential fusion of the preliminary fire segmentation probability map and the background region reconstruction probability map to obtain a target fire segmentation probability map.

[0048] The shared encoder is used to extract multi-scale features from preprocessed multi-band infrared image data to obtain multi-scale feature maps.

[0049] The fire point segmentation decoder is used to perform preliminary segmentation of fire point features in a multi-scale feature map to obtain a preliminary fire point segmentation map.

[0050] The background reconstruction decoder is used to extract and reconstruct background features of the background region in a multi-scale feature map to obtain a reconstructed background region map.

[0051] The fusion module is used to perform differential feature fusion on the preliminary fire point segmentation map and the background region reconstruction map to obtain the target fire point segmentation map.

[0052] The aforementioned preset fire detection model includes a shared encoder, a fire segmentation decoder, a background reconstruction decoder, and a fusion module.

[0053] This pre-defined fire detection model introduces a segmentation-reconstruction dual decoding architecture, which is implemented through a specific network structure and training strategy.

[0054] In this implementation process, an encoder is first used to extract multi-scale feature maps, capturing fire point features and background features at different scales. Then, a fire point segmentation decoder is used to specifically extract fire point features, obtaining a preliminary fire point segmentation probability map, focusing on fire point segmentation to accurately extract fire point information. Simultaneously, a background reconstruction decoder is used to specifically extract background features and reconstruct the background region, obtaining a background region reconstruction probability map to accurately extract background information. Furthermore, a fusion module performs differential fusion on the two probability maps to remove background information from the background region reconstruction probability map from the preliminary fire point segmentation probability map. This suppresses the response of the background region in fire point segmentation, thereby reducing background interference and highlighting the true fire point.

[0055] S13, Determine the fire point pixel detection result based on the target fire point segmentation probability map.

[0056] Based on the target fire point segmentation probability map, the fire point pixel detection results are more accurate, thereby reducing the probability of missed fire points and false fire points.

[0057] In the aforementioned fire detection method, a collaborative architecture comprising an encoder, a fire point segmentation decoder, a background reconstruction decoder, and a fusion module is constructed. Based on a differential fusion mechanism, the target fire point is separated from the background, avoiding semantic confusion in feature learning. This effectively identifies and suppresses interference from background information on real fire point information, reducing the false negative and false positive rates for fire point detection. Simultaneously, both the fire point segmentation decoder and the background reconstruction decoder receive features from each layer of the encoder via skip connections, ensuring that high-resolution detail features from lower layers are directly injected into the decoding process. This avoids information attenuation of small target features in a single decoding path. Therefore, through multi-scale feature extraction and a collaborative decoding architecture, the spatial details of small targets are recovered while preserving high-level semantics, thereby reducing the false negative rate for small target fire points.

[0058] In one embodiment, the encoder includes N downsampling blocks and a first convolutional block connected in a preset bit order, the fire point segmentation decoder includes N first upsampling blocks, and the background reconstruction decoder includes (N+1) background feature transformation modules that are independently distributed and N second upsampling blocks connected in a preset bit order.

[0059] The first sub-output of each downsampling block is connected to the first sub-input of a first upsampling block in a one-to-one correspondence.

[0060] The second sub-output of each downsampling block is connected to the second sub-input of a second upsampling block through a background feature transformation module.

[0061] The first output of the first convolutional block is connected to the third sub-input of the first first upsampling block. The second output of the first convolutional block is connected to the fourth sub-input of the first second upsampling block through a background feature transformation module.

[0062] The background feature transformation module is configured to model the local context dependencies of non-fire point regions.

[0063] In one implementation, a dual-branch collaborative decoding network architecture is constructed, comprising a shared encoder, a fire point segmentation decoder, and a background reconstruction decoder. A Transformer-Based Background Module (TBM) is embedded before each upsampling stage of the background reconstruction decoder. This TBM module divides the input features into 8×8 non-overlapping local windows, performs multi-head self-attention computation within each window, and sequentially connects layer normalization, a feedforward neural network, and residual connections to model the local contextual dependencies of non-fire point regions (such as clouds or hot surfaces). This implementation addresses the problem of poor ability of existing single-branch models to distinguish between fire points and bright interference backgrounds (such as bright clouds or thermal anomalies)—related technologies model all pixels uniformly, leading to background semantics being mixed into foreground prediction; while this application improves the accuracy of model prediction by learning the background distribution of non-fire points through an independent reconstruction branch.

[0064] Based on the model structure settings of this implementation method, the model structure is further implemented using the following two implementation methods.

[0065] In the first embodiment, the N first upsampling blocks are connected in sequence according to a preset bit order; the fire point segmentation decoder also includes a second convolutional block connected to the first upsampling output of the Nth first upsampling block, and the background reconstruction decoder also includes a third convolutional block connected to the second upsampling output of the Nth second upsampling block.

[0066] The Nth first upsampling block is the last first upsampling block from 1 to N.

[0067] Under this model structure, the specific process of calling the fusion module to perform differential fusion in step S12 above is as follows.

[0068] First, based on the first learnable scalar parameter, the background region reconstruction probability map output by the third convolutional block is scaled.

[0069] The first learnable scalar parameter mentioned above is used to control the intensity of background suppression and is positively correlated with background brightness.

[0070] Secondly, the initial fire point segmentation probability map output by the second convolutional block is subtracted from the background region reconstruction probability map after scaling operation on an element-by-element weighted basis to obtain the target fire point segmentation probability map.

[0071] In this implementation, the specific process of introducing the feature difference fusion mechanism can be as follows: subtract the segmentation probability map (SegLogits) output by the fire point segmentation decoder from the reconstruction probability map (RecLogits) output by the background reconstruction decoder after scaling the first learnable scalar parameter recWeight element-wise to generate the final fused probability map. This refers to the target fire point segmentation probability map. Its mathematical expression is: The first learnable scalar parameter can be understood as the recWeight, which participates in end-to-end training as a learnable parameter in the network.

[0072] This implementation solves the problem that related fusion methods (such as feature splicing and weighted addition) are prone to cause foreground-background semantic confusion—addition operations cannot effectively suppress strong response background regions; while this application directly cancels background activation in the probability precursor space through difference operations, thereby enhancing the fire point foreground and suppressing the interfering background.

[0073] Based on the U-Net single-branch semantic segmentation model and combined with the modules proposed in this application, the model structure of the preset fire detection model in this implementation is as follows: Figure 2 As shown. The background feature transformation module can also be called the background feature conversion module. The fusion module is also called the fusion head. The first learnable scalar parameter is also called the reconstruction weight. For example... Figure 2 As shown, the encoder includes four downsampling blocks and one convolutional block; the segmentation decoder includes four upsampling blocks and one 1x1 convolutional block; the background reconstruction decoder, or simply the reconstruction decoder, includes five TBM modules, four upsampling blocks, and one 1x1 convolutional block. In the second embodiment, the N first upsampling blocks are independently distributed in a preset position order. The fusion module includes a fourth convolutional block and N sub-fusion blocks independently distributed in a preset position order. The first sub-sampling output of each first upsampling block is connected to the second sub-sampling output of each corresponding second upsampling block through a sub-fusion block. The sub-fusion output of the Kth sub-fusion block is connected to the fifth sub-input of the (K-1)th first upsampling block, where K is any positive integer from 1 to (N-1). The sub-fusion output of the Nth sub-fusion block is connected to the input of the fourth convolutional block.

[0074] Based on this model structure, step S12 above is further implemented.

[0075] The initial fire point segmentation probability map includes the first sub-fire point segmentation probability map output from the first sub-sampling output terminal of each first upsampling block.

[0076] The specific process of calling the fire point segmentation decoder for preliminary segmentation in step S12 above is as follows: First, for any call to any first upsampling block, the multi-scale feature map output by the downsampling block connected to the first upsampling block and the second sub-fire point segmentation probability map output by the sub-fusion output terminal connected to the first upsampling block are subjected to preliminary segmentation processing to obtain the first sub-fire point segmentation probability map output by the first sub-sampling output terminal of the first upsampling block. Second, the sub-fusion output terminal connected to the first upsampling block is the sub-fusion output terminal of the sub-fusion block connected to the first upsampling block.

[0077] The specific process of calling the fusion module to perform differential fusion in step S12 above is as follows.

[0078] According to the preset order, each sub-fusion block is called sequentially to perform the following fusion operation until the Nth sub-fusion block is completed. Then, the fourth convolution block is used to perform a convolution operation on the sub-fire point segmentation probability map output by the sub-fusion output of the Nth sub-fusion block to obtain the target fire point segmentation probability map.

[0079] The fusion operation performed by any sub-fusion block includes: scaling the sub-reconstruction probability map output by the second sub-sampling output terminal connected to the corresponding sub-fusion block according to the second learnable scalar parameter, and subtracting the first sub-fire point segmentation probability map output by the first sub-sampling output terminal connected to the corresponding sub-fusion block from the scaled sub-reconstruction probability map element by element to obtain the second sub-fire point segmentation probability map output by any sub-fusion block.

[0080] Based on the U-Net single-branch semantic segmentation model and combined with the modules proposed in this application, the model structure of the preset fire detection model in this implementation is as follows: Figure 3 As shown. The background feature transformation module can also be called the background feature conversion module. The fusion module is also called the fusion head. The first learnable scalar parameter is also called the reconstruction weight. For example... Figure 3 As shown, the encoder includes 4 downsampling blocks and one convolutional block; the segmentation decoder includes 4 upsampling blocks and one 1x1 convolutional block; the background reconstruction decoder, or simply the reconstruction decoder, includes 5 TBM modules, 4 upsampling blocks and one 1x1 convolutional block. Figure 3 China also adopts the same approach as Figure 2 The basic framework of the encoder and two parallel decoders is retained, but the feature fusion method is adjusted: instead of placing the fusion operation at the end of the network's logit layer, a learnable differential fusion is performed on the intermediate feature maps output by the segmentation decoder and the background reconstruction decoder at each upsampling stage (i.e., after each Up Block). Specifically, after the k-th upsampling stage, features of the same scale are obtained from both decoders. and Then, a feature difference fusion operation is performed to generate fused features: This fusion feature It is directly used as the input to the next-level segment decoder (replacing the original). This allows background suppression signals to be injected step by step during the decoding process.

[0081] The following provides a further explanation of each module in the preset fire detection model. The preset fire detection model includes the following modules: Encoder, Segmentation Decoder, Reconstruction Decoder, and Fusion Head. These modules are connected through a feature transfer path, forming an end-to-end trainable architecture.

[0082] First, the encoder consists of a 5-level downsampling structure, with the input being a remote sensing image X ∈ ^{H×W×C} (using bands 7, 6, and 5 of Landsat 8 remote sensing imagery, with a size of 256 x 256), outputting five sets of multi-scale feature maps {f1, f2, f3, f4, f5}, with resolutions of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively.

[0083] Each downsampling block (e.g.) Figure 2 The structure is as follows: The input features first pass through a ConvBlock (convolutional module), which contains two consecutive convolutional layers (Conv (convolution) → BatchNorm (batch normalization) → ReLU (activation) → Conv (convolution) → BatchNorm (batch normalization) → ReLU (activation)); the output is connected to a MaxPool layer (maximum pooling, kernel size 2×2, stride 2) to achieve spatial downsampling; finally, it is connected to a DropOut layer (random deactivation layer, drop rate set to 0.1~0.3, optional, used for regularization).

[0084] The connection relationships between the blocks are as follows: The input image first enters the first Down Block (downsampling module); the output of each Down Block serves as the input of the next Down Block; at the same time, the feature maps output by each Down Block (except for the last layer) are sent to the corresponding Up Block of the segmentation decoder and reconstruction decoder, forming skip connections; the output of the fifth Down Block enters an independent Conv Block (convolution block, without downsampling), generating the deepest feature f5, which is used by both decoders.

[0085] This encoder can perform the following functions: extract multi-scale semantic features of the input image and output low-level details and high-level semantics in layers, providing structured information support for the subsequent decoding stage.

[0086] Secondly, the segment decoder is a standard U-Net upsampling structure, consisting of 4 Up Blocks (upsampling blocks or upsampling modules) connected in series, which restore the spatial resolution step by step and fuse the encoder features of the corresponding level.

[0087] Each Up Block has the following structure: Input is the upsampled features from the previous level or the encoder skip connection features; first, it passes through the ConvTranspose layer (transposed convolutional layer, kernel size 2×2, stride 2) to achieve upsampling; then it passes through the Concat layer (concatenation) to concatenate the upsampled results with the encoder features of the corresponding level (such as f3) along the channel dimension; then it passes through the DropOut layer (optional); finally, it passes through a Conv Block (convolutional block, two convolutional layers + BN + ReLU) to output the current scale features.

[0088] Connection relationship: Starting from the deepest layer f5, it passes through the 4th→3rd→2nd→1st Up Block in sequence; each Up Block receives the jump connection features from the corresponding layer of the encoder (f4→f3→f2→f1); the final output is compressed to 1 channel number by a 1×1 convolutional layer (representing the logits of fire point and non-fire point categories), denoted as SegLogits (segmentation probability map).

[0089] The segmentation decoder performs the following functions: based on the semantic features extracted by the encoder, it gradually recovers pixel-level segmentation predictions and generates a preliminary fire point segmentation probability map.

[0090] Third, the background reconstruction decoder structure is symmetrical to the segmentation decoder, but a background feature transformation module is added before each Up Block to enhance the ability to model the semantic structure of non-fire point regions.

[0091] like Figure 2 and Figure 3The TBM module structure in the lower right corner is as follows: The input features first pass through the Padding layer (a padding layer that adds zeros to make the size divisible by 8, facilitating window partitioning); then into the Window Partition layer, which divides the features into 8×8 non-overlapping local windows; LayerNorm (layer normalization) is performed on the features within each window; then the features are fed into the Multi-head Attention module (a multi-head self-attention mechanism, the number of heads can be set to 4 or 8, and the attention dimension is consistent with the input channel); the attention output is added to the original input as a residual; LayerNorm is performed again; then the Feed-Forward Net layer (two fully connected layers + GELU activation + Dropout) is connected; the residual is added to the original input again; finally, the WindowReverse layer restores the original shape features.

[0092] Connections between blocks: The output features of each layer of the encoder (f5→f4→f3→f2→f1) are input into the corresponding TBM module; the features processed by the TBM are then fed into the corresponding Up Block for upsampling and splicing; after all Up Blocks are concatenated, the final output is a reconstruction probability map, denoted as RecLogits, through a 1×1 convolutional layer.

[0093] The background reconstruction decoder performs the following functions: it explicitly learns the spatial distribution and local semantic dependencies of the background regions of non-fire points, providing a high-quality background suppression signal for subsequent subtraction fusion.

[0094] Third, the fusion head is used to achieve foreground enhancement and background suppression in the logit space. Its structure is simple but crucial: the fusion head does not rely on complex feature concatenation or weighted averaging, but directly performs probability difference operations in the probability space: subtracting the product of a learnable scalar parameter recWeight (reconstruction weight) and RecLogits (reconstruction probability map) from SegLogits (segmentation probability map), i.e. recWeight, as a trainable parameter in the network, is automatically adjusted during end-to-end training to adapt to the background suppression intensity under different scenarios. This subtraction mechanism requires no additional modules yet effectively achieves simultaneous foreground enhancement and background suppression—when RecLogits has a high value in a certain region, it indicates that the region belongs to a "strong background," and the subtraction operation will correspondingly reduce the final response in that region, effectively suppressing the segmentation decoder from misdetecting background interference factors as fire points and improving the accuracy of model prediction.

[0095] Each convolutional block includes the following layers in sequence according to the execution flow: a convolutional layer that performs convolution operations, a normalization layer that performs batch normalization operations, an activation layer that performs activation operations, a convolutional layer that performs convolution operations, a normalization layer that performs batch normalization operations, and an activation layer that performs activation operations.

[0096] Each downsampling block, in the order of execution, includes the following layers: the aforementioned convolutional block, the max pooling layer, and the random deactivation layer.

[0097] Each upsampling block includes the following layers in sequence according to the execution flow: a transposed convolutional layer that performs transpose and convolution operations, a splicing layer that performs splicing operations, a random deactivation layer, and a convolutional block.

[0098] Each background feature transformation module includes the following layers in sequence according to the execution flow: a filling layer for feature filling, a window partitioning layer for dividing the feature map into windows, a layer normalization layer, a multi-head self-attention layer, a fusion layer that fuses the outputs of the multi-head self-attention layer and the window partitioning layer, a layer normalization layer, a feedforward network, a fusion layer that fuses the outputs of the above fusion layers with the outputs of the feedforward network, and a flipping layer that flips the window of the fusion map.

[0099] As one implementation method, the background reconstruction decoder is invoked to extract and reconstruct the background features of the background region in the multi-scale feature map to obtain the background region reconstruction probability map. Specifically, this includes: sequentially invoking each background feature transformation module to perform the following reconstruction operation.

[0100] Each background feature transformation module is executed according to the following reconstruction operation steps.

[0101] First, the input feature map input to each background feature transformation module is divided into multiple non-overlapping local windows.

[0102] Secondly, multi-head self-attention computation is performed within each local window.

[0103] Third, the results of multi-head self-attention calculation are sequentially processed by layer normalization, feedforward neural network processing, and residual connection to obtain each first reconstruction probability map.

[0104] Fourth, multiple second upsampling blocks and third convolutional blocks are invoked to process each first reconstruction probability map through upsampling and convolution to obtain the background region reconstruction probability map.

[0105] The input feature map of the first background feature transformation module (ranked first) is the output of the first convolutional block connected to each background feature transformation module. The outputs of the downsampling blocks connected to each background feature transformation module (ranked first) are the outputs of other background feature transformation modules.

[0106] As a model training method, the model parameters of the initial preset fire detection model are trained based on spatial weighted loss and Tversky loss to obtain the trained preset fire detection model.

[0107] The spatially weighted loss is the weighted sum of the prediction confidence of each output fire pixel and its spatial weight. The prediction confidence represents the confidence of each output fire pixel in the pixel space relative to the actual fire pixel during the prediction process. The spatial weight represents the degree of attenuation of the area of ​​the connected region to which each output fire pixel belongs in the image. The spatial weight of each output fire pixel is negatively correlated with the area of ​​its connected region.

[0108] Tversky loss is the true fire loss rate determined based on the number of true fire points, the number of background fire points, and the number of missed true fire points in each output fire point pixel.

[0109] In this implementation, a spatially weighted hybrid loss function, SpatialFocalTverskyLoss, is constructed for remote sensing fire point segmentation tasks. This loss consists of two weighted components: spatially weighted loss and Tversky loss.

[0110] In some implementations, the spatially weighted loss is constructed by introducing spatial weights based on the area of ​​the connected regions of the fire points, on top of the standard Focal Loss. Higher loss weights are assigned to fire points in small connected domains, and numerical stability is ensured through normalization. and These are the minimum and maximum values ​​of the spatial weights, respectively. Let be the area of ​​the connected region to which the i-th pixel belongs in the image; This is the spatial weight decay coefficient, used to control the rate at which spatial weights decay with increasing area. The expression for its spatial weighted loss is: ; It is a constant. Let be the pixel loss of the i-th pixel.

[0111] In this implementation, by using a spatially weighted mixed loss function with different weights, the model can explicitly focus on hard samples (small-area fire points) during training, avoiding the problem that the model is completely biased towards predicting negative samples due to the dominance of background pixels.

[0112] In some implementations, the Tversky loss is expressed using adjustable parameters. and The penalty strength for false negatives (FN) and false positives (FP) is controlled separately to improve the model's segmentation robustness under extreme class imbalance. The Tversky loss expression is: Where TP represents true positives, indicating the number of fire points correctly predicted by the model; FN represents false negatives, indicating the number of fire points missed by the model; FP represents false positives, indicating the number of fire points falsely reported by the model; and smooth is the smoothing term.

[0113] The overall loss function is defined by the following formula.

[0114] .

[0115] The loss function is obtained through and By adjusting the weights of the two losses, the learning of small fire points is explicitly enhanced while taking into account the overall segmentation quality, effectively mitigating the problems of missed detections and false detections.

[0116] As one implementation method, the above-mentioned preprocessing of multi-band infrared image data includes: combining the bands in the multi-band infrared image data whose fire sensitivity is higher than a preset sensitivity, and normalizing and / or cropping the combined multi-band infrared image data.

[0117] As another implementation, in step S13 above, the fire point pixel detection result is determined based on the target fire point segmentation probability map. The specific process includes: performing threshold segmentation and / or morphological filtering on the target fire point segmentation probability map to obtain the fire point pixel detection result.

[0118] To achieve the above functions, the fire detection device based on multi-band infrared images includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art will readily recognize that, based on the algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0119] like Figure 4 As shown in the embodiments of this application, a fire detection device based on multi-band infrared images is also proposed. The device includes: an acquisition unit 401, a fire detection unit 402, and a determination unit 403.

[0120] The acquisition unit 401 is configured to acquire multi-band infrared image data and preprocess the multi-band infrared image data.

[0121] The fire detection unit 402 is configured to use a preset fire detection model to perform feature classification, extraction and feature fusion on the fire features of the fire area and the background features of the non-fire area in the preprocessed multi-band infrared image data to obtain a target fire point segmentation probability map.

[0122] The preset fire detection model includes an encoder, a fire segmentation decoder, a background reconstruction decoder, and a fusion module.

[0123] The encoder is used to extract multi-scale features from the preprocessed multi-band infrared image data to obtain a multi-scale feature map; the fire point segmentation decoder is used to perform preliminary segmentation of the fire point features in the multi-scale feature map to obtain a preliminary fire point segmentation probability map; the background reconstruction decoder is used to extract and reconstruct the background features of the background region in the multi-scale feature map to obtain a background region reconstruction probability map; and the fusion module is used to perform differential fusion of the preliminary fire point segmentation probability map and the background region reconstruction probability map to obtain the target fire point segmentation probability map.

[0124] The determining unit 403 is configured to determine the fire point pixel detection result based on the target fire point segmentation probability map.

[0125] Regarding the apparatus in the above embodiments, the specific manner in which each unit module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0126] Figure 5 This is a schematic diagram of a fire detection device based on multi-band infrared images provided in this application. Figure 5 The fire detection device 50 based on multi-band infrared images includes: a first processor 501, a communication bus 502, a memory 503, a communication interface 504, an output device 505, an input device 506, and a second processor 507.

[0127] The fire detection device 50 based on multi-band infrared images may include at least one first processor 501 and a memory 503 for storing processor-executable instructions. The first processor 501 is configured to execute the instructions in the memory 503 to implement the fire detection method based on multi-band infrared images in the following embodiments.

[0128] In addition, the fire detection device 50 based on multi-band infrared images may also include a communication bus 502, at least one communication interface 504, an input device 506, and an output device 505.

[0129] The first processor 501 may be a processor (central processing unit, CPU), a microprocessor unit, an ASIC, or one or more integrated circuits for controlling the execution of programs according to the present application.

[0130] The communication bus 502 may include a path for transmitting information between the aforementioned components.

[0131] Communication interface 504 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0132] Input device 506 is used to receive input signals and output device 505 is used to output signals.

[0133] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital universal discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory may exist independently and be connected to the processing unit via a bus. Memory may also be integrated with the processing unit.

[0134] The memory 503 stores instructions for executing the scheme of this application, and the execution is controlled by the first processor 501. The first processor 501 executes the instructions stored in the memory 503 to realize the functions of the method of this application.

[0135] In a specific implementation, as one example, the first processor 501 may include one or more CPUs, for example... Figure 5 CPU0 and CPU1 in the CPU.

[0136] In a specific implementation, as one example, the fire detection device 50 based on multi-band infrared images may include multiple processors, such as... Figure 5 The first processor 501 and the second processor 507 are described. Each of these processors can be a single-core processor or a multi-core processor. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0137] This fire detection device based on multi-band infrared images, such as Figure 5 The diagram includes a first processor 501 and a memory 503 for storing executable instructions of the first processor 501. The first processor 501 is configured to execute the executable instructions to implement a fire detection method based on multi-band infrared images as described in any of the possible embodiments above. Furthermore, it achieves the same technical effect, and to avoid repetition, will not be elaborated further here.

[0138] This application also provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by the processor of a fire detection device based on multi-band infrared images, the fire detection device based on multi-band infrared images is able to perform the fire detection method based on multi-band infrared images as described in any of the possible embodiments above. And it can achieve the same technical effect; to avoid repetition, it will not be described again here.

[0139] This application also provides a computer program product, including a computer program or instructions, which are executed by a processor as described in any of the possible implementations above, for a fire detection method based on multi-band infrared images. This achieves the same technical effect, and to avoid repetition, it will not be described again here.

[0140] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0141] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A fire detection method based on multi-band infrared images, characterized in that, The method includes: Acquire multi-band infrared image data and preprocess the multi-band infrared image data; A preset fire detection model is used to classify, extract, and fuse fire features in fire areas and background features in non-fire areas of the preprocessed multi-band infrared image data to obtain a target fire point segmentation probability map. The feature classification, extraction, and fusion using the preset fire detection model includes: calling an encoder to extract multi-scale features from the preprocessed multi-band infrared image data to obtain a multi-scale feature map; calling a fire point segmentation decoder to perform preliminary segmentation of the fire features in the multi-scale feature map to obtain a preliminary fire point segmentation probability map; calling a background reconstruction decoder to extract and reconstruct background features in the background areas of the multi-scale feature map to obtain a background area reconstruction probability map; and calling a fusion module to perform differential fusion of the preliminary fire point segmentation probability map and the background area reconstruction probability map to obtain the target fire point segmentation probability map. Based on the target fire point segmentation probability map, the fire point pixel detection result is determined; the encoder includes N downsampling blocks and a first convolutional block connected in a preset position sequence; the fire point segmentation decoder includes N first upsampling blocks; the background reconstruction decoder includes (N+1) independently distributed background feature transformation modules and multiple N second upsampling blocks connected in a preset position sequence; the first sub-output of each downsampling block is connected to the first sub-input of a first upsampling block; the second sub-output of each downsampling block is connected to the second sub-input of a second upsampling block through a background feature transformation module; the first output of the first convolutional block is connected to the third sub-input of the first first upsampling block; the second output of the first convolutional block and the fourth sub-input of the first second upsampling block are connected through a background feature transformation module; the background feature transformation module is configured to model the local contextual dependencies of non-fire point regions. The N first upsampling blocks are connected sequentially in the preset bit order; the fire point segmentation decoder also includes a second convolutional block connected to the first upsampling output of the Nth first upsampling block, and the background reconstruction decoder also includes a third convolutional block connected to the second upsampling output of the Nth second upsampling block; The invocation of the fusion module performs differential fusion on the preliminary fire point segmentation probability map and the background region reconstruction probability map to obtain the target fire point segmentation probability map, including: The background region reconstruction probability map output by the third convolutional block is scaled according to the first learnable scalar parameter, which is used to control the intensity of background suppression and is positively correlated with the background brightness. The initial fire point segmentation probability map output by the second convolutional block is subtracted element-wise from the background region reconstruction probability map after scaling to obtain the target fire point segmentation probability map.

2. The fire detection method according to claim 1, characterized in that, The N first upsampling blocks are arranged independently and sequentially according to the preset position order; the fusion module includes a fourth convolutional block and N sub-fusion blocks arranged independently and sequentially according to the preset position order. The first sub-sampling output of each first upsampling block is connected to the second sub-sampling output of each corresponding second upsampling block through a sub-fusion block; the sub-fusion output of the Kth sub-fusion block is connected to the fifth sub-input of the (K-1)th first upsampling block, where K is any positive integer from 1 to (N-1); the sub-fusion output of the Nth sub-fusion block is connected to the input of the fourth convolutional block; The initial fire point segmentation probability map includes the first sub-fire point segmentation probability map output by the first sub-sampling output terminal of each first upsampling block; The step of calling the fire point segmentation decoder is used to perform preliminary segmentation of the fire point features in the multi-scale feature map to obtain a preliminary fire point segmentation probability map, including: For any call to any first upsampling block, the multi-scale feature map output by the downsampling block connected to the first upsampling block and the second sub-fire point segmentation probability map output by the sub-fusion output terminal connected to the first upsampling block are subjected to preliminary segmentation processing to obtain the first sub-fire point segmentation probability map output by the first sub-sampling output terminal of the first upsampling block; the sub-fusion output terminal connected to the first upsampling block is the sub-fusion output terminal of the sub-fusion block connected to the first upsampling block; The invocation of the fusion module performs differential fusion on the preliminary fire point segmentation probability map and the background region reconstruction probability map to obtain the target fire point segmentation probability map, including: According to the preset position order, each sub-fusion block is called sequentially to perform the following fusion operation until the Nth sub-fusion block is completed. Then, the fourth convolution block is used to perform a convolution operation on the sub-fire point segmentation probability map output by the sub-fusion output end of the Nth sub-fusion block to obtain the target fire point segmentation probability map. The fusion operation performed by any sub-fusion block includes: Based on the second learnable scalar parameter, the sub-reconstruction probability map output by the second sub-sampling output terminal connected to any sub-fusion block is scaled, and the first sub-fire point segmentation probability map output by the first sub-sampling output terminal connected to any sub-fusion block is subtracted element-wise from the scaled sub-reconstruction probability map to obtain the second sub-fire point segmentation probability map output by any sub-fusion block.

3. The fire detection method according to claim 1 or 2, characterized in that, The background reconstruction decoder is invoked to extract and reconstruct background features of the background region in the multi-scale feature map, resulting in a background region reconstruction probability map, including: Each background feature transformation module is invoked, and the following reconstruction operation is performed: the input feature map input to each background feature transformation module is divided into multiple non-overlapping local windows; multi-head self-attention calculation is performed within each local window; the result of the multi-head self-attention calculation is sequentially subjected to layer normalization, feedforward neural network processing, and residual connection to obtain each first reconstruction probability map; multiple second upsampling blocks and third convolutional blocks are invoked to process each first reconstruction probability map through upsampling and convolution processing to obtain the background region reconstruction probability map; The input feature map of each background feature transformation module is the output of the first convolutional block connected to each background feature transformation module or the output of the downsampling block connected to each background feature transformation module.

4. The fire detection method according to claim 1 or 2, characterized in that, The method further includes: Based on spatial weighted loss and Tversky loss, the model parameters of the initial preset fire detection model are trained to obtain the trained preset fire detection model. The spatially weighted loss is the weighted sum of the prediction confidence of each output fire pixel and its spatial weight. The prediction confidence represents the confidence of each output fire pixel relative to the actual fire pixel in the pixel space during the prediction process. The spatial weight represents the degree of attenuation of the area of ​​the connected region to which each output fire pixel belongs in the image. The spatial weight of each output fire pixel is negatively correlated with the area of ​​its connected region. The Tversky loss is a fire point loss rate determined based on the number of real fire points, the number of background fire points, and the number of missed real fire points in each output fire point pixel.

5. The fire detection method according to claim 1 or 2, characterized in that, The preprocessing of the multi-band infrared image data includes: combining the data of multiple bands in the multi-band infrared image data whose fire sensitivity is higher than a preset sensitivity, and normalizing and / or cropping the combined multi-band infrared image data. The step of determining the fire point pixel detection result based on the target fire point segmentation probability map includes: Threshold segmentation and / or morphological filtering are performed on the target fire point segmentation probability map to obtain the fire point pixel detection results.

6. A fire detection device based on multi-band infrared images, using the fire detection method based on multi-band infrared images as described in any one of claims 1 to 5, characterized in that, The device includes: The acquisition unit is configured to acquire multi-band infrared image data and preprocess the multi-band infrared image data. The fire detection unit is configured to use a preset fire detection model to perform feature classification, extraction, and feature fusion on the fire features of the fire region and the background features of the non-fire region in the preprocessed multi-band infrared image data to obtain a target fire point segmentation probability map. The preset fire detection model includes an encoder, a fire point segmentation decoder, a background reconstruction decoder, and a fusion module. The encoder is used to extract multi-scale features from the preprocessed multi-band infrared image data to obtain a multi-scale feature map. The fire point segmentation decoder is used to perform preliminary segmentation of the fire features in the multi-scale feature map to obtain a preliminary fire point segmentation probability map. The background reconstruction decoder is used to extract and reconstruct the background features of the background region in the multi-scale feature map to obtain a background region reconstruction probability map. The fusion module is used to perform differential fusion of the preliminary fire point segmentation probability map and the background region reconstruction probability map to obtain the target fire point segmentation probability map. The determining unit is configured to determine the fire point pixel detection result based on the target fire point segmentation probability map.

7. A fire detection system, characterized in that, It is configured to perform the fire detection method based on multi-band infrared images as described in any one of claims 1-5.

8. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of a computer device, the computer device is enabled to perform the fire detection method based on multi-band infrared images as described in any one of claims 1-5.

Images