A forest grassland fire monitoring method and a model training method

Abstract

The application discloses a forest and grassland fire smoke and fire point simultaneous detection method. The method obtains visible light and infrared band images of a target area, registers and synthesizes into multi-band synthetic data; the data is input into a pre-trained multi-modal smoke and fire simultaneous detection network for feature extraction, the network adopts a feature fidelity down-sampling module containing a fixed kernel low-pass filter and average pooling during down-sampling, and is combined with a cross-layer connected background suppression and smoke and fire focusing module, a multi-scale smoke and fire shape perception module and a smoke and fire space information rearrangement amplification module for joint processing; finally, according to the feature channel set, the smoke and the fire point area are respectively located, and a fire monitoring distribution map is generated by superposition. The application realizes the simultaneous detection and mutual verification of multi-modal features of smoke and fire, effectively overcomes the loss problem of small fire points and smoke plume details, and improves the accuracy of early fire warning.

Description

Technical Field

[0001] This invention relates to the field of intelligent satellite remote sensing monitoring of forest and grassland fires, and more specifically, to a method for monitoring forest and grassland fires that integrates multimodal satellite imagery with deep learning image processing technology. Background Technology

[0002] Forest and grassland fires are characterized by their suddenness and persistence. Early detection and timely extinguishing can prevent small fires from escalating into large-scale conflagrations. Satellite remote sensing has been widely proven to be one of the most effective methods for monitoring forest and grassland fires. The smoke and fire spots produced by forest and grassland combustion exhibit spectral differences between satellite imagery and the background, providing important indicators of the occurrence and development of forest and grassland fires. Currently, the industry typically utilizes these differences in satellite imagery to extract real-time information about the fire scene, such as smoke and fire spots, for fire monitoring.

[0003] However, in routine satellite remote sensing monitoring, existing technologies face the following major technical bottlenecks: First, fire detection is prone to missed detection. Current detection mainly relies on kilometer-level thermal infrared imagery. Due to the low spatial resolution of this type of imagery, when the fire intensity is weak and the area is less than one pixel (i.e., small fire spots / sub-pixel fire spots), it is very easy to miss the detection due to the weak thermal radiation signal, which can even lead to major forest and grassland fires. Second, the accuracy of smoke detection is low. Although smoke is often clearly visible in multispectral imagery with medium to high spatial resolution, the complexity and variability of smoke spectral characteristics, as well as its susceptibility to interference from cloud cover or ground background, result in low accuracy for methods that use satellite remote sensing reflectance differences to detect smoke.

[0004] To address these issues, existing detection methods typically utilize the spectral differences between smoke or fire points and background features, independently judging each by setting physical thresholds. While machine learning and deep learning have been introduced into some research in recent years, they haven't yet combined smoke and fire detection methods for mutual complementarity. These existing solutions suffer from the following significant drawbacks: First, deficiencies in the underlying feature extraction mechanism. Existing conventional deep learning image segmentation models (such as the traditional UNet framework) are prone to aliasing and truncation of high-frequency information during network downsampling feature dimensionality reduction (e.g., max pooling). This directly leads to the disappearance of already fragile micro-fire point features or severely disrupts the irregular, gradient smoke plume edges. Second, a lack of spatial verification logic for multi-source heterogeneous data. Due to limitations in model transferability, existing technologies mostly treat smoke and fire points as separate tasks. The failure to spatially align and feature-level fuse multispectral and infrared images prevents the system from mutually verifying and complementing visible light reflection (smoke) and thermal radiation (fire) at the same geographical location. Under complex conditions such as lightning strikes and initial fires in mountainous areas, it is difficult to achieve rapid and accurate monitoring and early warning of fires. Summary of the Invention

[0005] The purpose of this invention is to provide a method for simultaneously detecting smoke and fire points in forest and grassland fires, integrating spectral features and deep learning, as well as a model training method. This invention aims to address the technical problems in existing routine satellite remote sensing monitoring work, such as the tendency to miss sub-pixel-level small fire points when relying on kilometer-level thermal infrared imagery, the low accuracy of smoke detection using multispectral imagery, and the easy loss of plume edges and small fire point features during conventional deep learning downsampling processes.

[0006] To address the above problems, this invention provides a method for monitoring forest and grassland fires, comprising the following steps: The visible light band image and infrared band image of the target area are acquired, spatially registered at a uniform spatial resolution, and converted into multi-band composite data containing a first feature channel set corresponding to the visible light band image and a second feature channel set corresponding to the infrared band image. The multi-band synthesized data is input into a pre-trained multimodal fireworks detection network for feature extraction to obtain feature extraction results. The multimodal fireworks detection network uses a fireworks feature fidelity downsampling module that includes fixed kernel low-pass filtering and average pooling operations for feature processing when performing feature downsampling and scaling. Based on the feature extraction results, the smoke area is located according to the first feature channel set in the extraction results, and the fire area is located according to the second feature channel set. The smoke area and the fire area are then spatially superimposed to generate a forest and grassland fire monitoring distribution map.

[0007] Furthermore, in the above-mentioned forest and grassland fire monitoring method, the feature processing using a smoke and fire feature fidelity downsampling module that includes fixed kernel low-pass filtering and average pooling operations includes: The downsampled input feature map input to the fireworks feature fidelity downsampling module is low-pass filtered to obtain the first feature map; Simultaneously, average pooling is performed on the downsampled input feature map to obtain a second feature map; The first feature map and the second feature map are fused together.

[0008] Furthermore, in the aforementioned forest and grassland fire monitoring method, the feature processing using a smoke and fire feature fidelity downsampling module that includes fixed-kernel low-pass filtering and average pooling operations specifically includes: The downsampled input feature map is convolved using a preset fixed-weight smoothing matrix, and the feature map obtained after the convolution process is then subjected to downsampling convolution and channel convolution in sequence to obtain the first feature map. After performing average pooling on the downsampled input feature map, the channel dimensions are aligned through convolutional mapping to obtain the second feature map; The first feature map and the second feature map are added element by element, and the added feature map is then subjected to feature normalization and nonlinear activation processing in sequence to obtain the output feature map of the fireworks feature fidelity downsampling module.

[0009] Furthermore, in the above-mentioned forest and grassland fire monitoring method, the multimodal smoke and fire detection network is equipped with a cross-layer connection background suppression and smoke and fire focusing module between the corresponding layers of the feature downsampling stage and the feature upsampling stage; The background suppression and fireworks focusing module's processing includes: Receive the cross-layer input feature map output from the corresponding layer during the feature downsampling stage, perform global spatial pooling on the cross-layer input feature map and calculate the channel dimension weights to obtain a channel-weighted feature map; Simultaneously, the cross-layer input feature map is subjected to single-channel convolution mapping and spatial dimension weights are calculated to obtain a spatially weighted feature map; The channel-weighted feature map and the spatial-weighted feature map are added element by element to obtain a fused feature map, and the fused feature map is output to the corresponding layer of the feature upsampling stage.

[0010] Furthermore, in the aforementioned forest and grassland fire monitoring method, a bridging module is provided between the feature downsampling stage and the feature upsampling stage of the multimodal smoke and fire detection network. This bridging module includes a multi-scale smoke and fire morphology sensing module. The processing procedure of the multi-scale smoke and fire morphology sensing module includes: By using multiple dilated convolution branches with different expansion rates, parallel convolution extraction is performed on the intermediate layer input feature map input to the multi-scale smoke and fire morphology perception module to obtain the smoke and fire morphology features corresponding to different receptive fields, resulting in multiple morphology feature maps. The multiple morphological feature maps are spliced ​​together along the channel dimension to obtain a multi-scale spliced ​​feature map. The multi-scale stitched feature map is subjected to channel dimensionality reduction and feature fusion by convolutional mapping to obtain a multi-scale perceptual feature map, and the multi-scale perceptual feature map is output to the feature upsampling stage.

[0011] Furthermore, in the above-mentioned forest and grassland fire monitoring method, the output end of the bridging module of the multimodal smoke and fire detection network and the corresponding layer of the feature upsampling stage are both equipped with a large smoke and fire spatial information re-emission module; the processing of the large smoke and fire spatial information re-emission module includes: The feature map to be enlarged, which is input to the large-scale emission module of the fireworks space information, is subjected to channel expansion convolution to obtain a wide-channel feature map with a preset number of channels; According to the preset spatial rearrangement rules, the pixels of the wide channel feature map in the channel dimension are filled to the corresponding sub-pixel positions in the spatial dimension to obtain a preliminary rearranged feature map with increased spatial resolution. The preliminary rearranged feature map is subjected to group normalization and nonlinear activation processing in sequence to obtain the voltage-stabilized activation feature map; The voltage-stabilized activation feature map is extracted channel-by-channel independently using depthwise convolution to obtain a spatially refined feature map. The spatially refined feature map is fused across channels using pointwise convolution, and the feature map obtained after the cross-channel feature fusion is then subjected to group normalization and nonlinear activation processing in sequence to obtain the high-resolution feature map output by the large-scale smoke and fire spatial information re-emission module.

[0012] Furthermore, in the above-mentioned forest and grassland fire monitoring method, the backbone feature extraction unit of the multimodal smoke and fire detection network is a dual residual depthwise separable convolutional module; the processing procedure of the dual residual depthwise separable convolutional module includes: The backbone input feature map input to the double residual depth-separable convolution module is subjected to depth convolution, and then group normalization and nonlinear activation are performed sequentially to obtain the first process feature map. The first process feature map is fused across channels using pointwise convolution to obtain the second process feature map; After performing group normalization and nonlinear activation processing on the feature map of the second process in sequence, the processed feature map is added to the main input feature map by the first residual based on the input-output channel consistency gating mechanism to obtain the first-order output feature map; The first-order output feature map is subjected to depthwise convolution, followed by group normalization and nonlinear activation to obtain the third-order feature map. The feature map of the third process is fused across channels using pointwise convolution to obtain the feature map of the fourth process; After performing group normalization and nonlinear activation processing on the feature map of the fourth process in sequence, based on the input-output channel consistency gating mechanism, the processed feature map is added to the first-order output feature map by the second residual to obtain the final output feature map of the double residual depth separable convolution module. The input / output channel consistency gating mechanism refers to the following: when the number of feature map channels input to this stage is equal to the number of feature map channels about to be output to this stage, the corresponding first residual addition or second residual addition is initiated.

[0013] Furthermore, in the above-mentioned forest and grassland fire monitoring method, the total number of feature downsampling and scaling operations of the multimodal smoke and fire detection network is three.

[0014] This invention also provides a model training method, comprising the following steps: Obtain a training sample set, which includes multiple multi-band composite data containing visible light and infrared bands, as well as real labeled masks of smoke and fire points corresponding to the multi-band composite data. The multi-band synthetic data in the training sample set is input into the multi-modal smoke and fire detection network to be trained for forward propagation. During the forward propagation process, the multi-modal smoke and fire detection network to be trained extracts and restores features from the multi-band synthetic data through a smoke and fire feature fidelity downsampling module, a background suppression and smoke and fire focusing module, a multi-scale smoke and fire morphology perception module, and a large smoke and fire spatial information re-emission module. The main probability image is output through the terminal output layer of the multimodal fireworks detection network to be trained, and auxiliary probability images of different scales are output through multiple bridging modules of the multimodal fireworks detection network to be trained. Based on the main probability image and the real labeled mask, calculate the main global pixel classification loss and the smoke region overlap loss; Based on the auxiliary probability image and the ground truth labeled mask, the auxiliary global pixel classification loss is calculated respectively. The main global pixel classification loss, the smoke region overlap loss, and the weighted auxiliary global pixel classification loss are summed to obtain the mixed weighted total loss. Gradient backpropagation is performed based on the hybrid weighted total loss to update the weight parameters of the multimodal fireworks detection network to be trained until the model converges, thus obtaining the trained multimodal fireworks detection network.

[0015] Furthermore, in the above model training method, the main global pixel classification loss and the auxiliary global pixel classification loss are both calculated using the binary cross-entropy loss function, and the smoke and fire area overlap loss is calculated using the Dice loss function. The weighted total loss is obtained by summing the main global pixel classification loss, the overlap loss of the fireworks region with a preset overlap weight, and multiple auxiliary global pixel classification losses with different scales and corresponding auxiliary weight coefficients. The auxiliary global pixel classification loss at different scales includes: a first auxiliary global pixel classification loss corresponding to the output of the half-scale feature layer, and a second auxiliary global pixel classification loss corresponding to the output of the quarter-scale feature layer.

[0016] The above-described technical solution of the present invention has the following beneficial technical effects: This application extracts visible light bands to form a first feature channel set for smoke identification and extracts infrared bands to form a second feature channel set for fire point identification. Both are input into the same network for feature fusion and independent localization, and the results are spatially superimposed. Since smoke is often a precursor to early fires, this multimodal detection mechanism allows the two to mutually verify and complement each other spatially, solving the problem of missed detections with single infrared fire point detection.

[0017] This application controls the total number of downsampling and scaling operations of the overall network to three, and creatively designs a smoke feature fidelity downsampling module that includes fixed-kernel low-pass filtering and average pooling operations. This module filters out high-frequency noise and suppresses aliasing through low-pass filtering, and combines smooth average pooling features to reduce the dimensionality of the feature map while firmly locking the fine edges of the smoke plume and small fire point pixels.

[0018] This solution introduces a background suppression and smoke / fire focusing module into the cross-layer connection path of the network. This module calculates feature weights through a parallel channel-space dual attention mechanism, which can not only automatically select spectral channels that are sensitive to smoke / fire, but also suppress interference noise such as clouds and bright ground surfaces in the spatial dimension, guiding the model to accurately focus on the real smoke / fire area.

[0019] To address the challenge of extremely small proportions of smoke and fire areas in satellite imagery, this solution introduces deep supervision and a hybrid weighted loss function during the training phase. By extracting probability images of different scales at the network's endpoints and hidden bridging layers, and combining binary cross-entropy with Dice loss (which measures overlap) to constrain gradient updates, the model is forced to focus not only on global classification accuracy but also on the spatial overlap of extremely small target regions, significantly improving model convergence stability and detection accuracy. Attached Figure Description

[0020] Figure 1 This is a structural block diagram of a forest and grassland fire monitoring system according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the steps according to an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0022] To adapt to the forest and grassland fire monitoring method provided in the embodiments of this application, this application provides a forest and grassland fire monitoring method that can be applied to a pre-trained multimodal smoke and fire detection network provided in the embodiments of this application. For example... Figure 1 The forest and grassland fire monitoring system includes an image acquisition module, an encoding module, a bridging module, a decoding module, a jump connection module, an output activation module, and an output result module. These modules are connected sequentially, with the jump connection module positioned between the encoding and decoding modules.

[0023] Specifically, the image acquisition module receives preprocessed multi-band composite data. The multi-band composite data is composed of spatially registered and stacked visible light band images (such as the red, green, and blue bands of satellite imagery) and infrared band images (such as the near-infrared band) at the same spatial resolution. This data includes a first feature channel set (corresponding to visible light, used for smoke identification) and a second feature channel set (corresponding to infrared, used for fire detection).

[0024] Specifically, the encoding module can be divided into three sequentially connected feature extraction units. The first feature extraction unit is used to initially extract the shallow texture and edge information of the input image, and then perform downsampling processing on the extracted features. While compressing the spatial size, it preserves the integrity of the plume edges and small fire points as much as possible, and suppresses the aliasing phenomenon that is easily generated during ordinary downsampling. The second feature extraction unit is used to further extract mid-level semantic features based on the features output by the first feature extraction unit, and continues to perform downsampling processing to perceive the smoke diffusion pattern, fire point neighborhood response and its relationship with the complex background from a larger spatial range, while continuing to reduce the impact of aliasing on the recognition of target boundaries and small targets. The third feature extraction unit is used to further extract deeper features from the features processed above to obtain more discriminative high-level semantic features, and output them to the bridging module.

[0025] Furthermore, both the first and second feature extraction units include a double residual depthwise separable convolution module and a smoke feature fidelity downsampling module connected in sequence. The double residual depthwise separable convolution module enhances the extraction capabilities of smoke texture, small fire points, and their differences from background features. The smoke feature fidelity downsampling module suppresses aliasing during downsampling while reducing the spatial resolution of the feature map, maintaining the structural integrity of the plume edges and small fire points.

[0026] The third feature extraction unit includes a dual residual depthwise separable convolution module, which is used to further enhance the semantics and learn the deep representation of the input features after completing the first two levels of feature extraction and downsampling, so as to form an intermediate feature representation for the bridging module to perform multi-scale fireworks morphology perception.

[0027] Specifically, the bridging module is located between the encoding module and the decoding module. It is used to perform transition processing on the deep features output by the encoding module, enhance the multi-scale expressive ability of the features, and provide the decoding module with intermediate features suitable for step-by-step recovery.

[0028] Furthermore, the bridging module comprises a smoke feature fidelity downsampling module, a multi-scale smoke morphology perception module, and a large smoke spatial information re-emission module, which are connected in series. The smoke feature fidelity downsampling module downsamples and integrates the multi-scale fused features, reducing the feature map scale while suppressing aliasing, thus obtaining bridging output features that combine scale representation capability and structural stability. The multi-scale smoke morphology perception module extracts multi-scale receptive field features from the upsampled features to enhance the network's ability to perceive the morphology of smoke regions and fire points at different scales. The large smoke spatial information re-emission module upsamples the output features of the encoding module to improve feature map resolution and enhance the recovery of smoke texture and fire point details.

[0029] After being processed sequentially by the smoke and fire feature fidelity downsampling module, the multi-scale smoke and fire morphology perception module, and the smoke and fire spatial information redemission module, the features output by the bridging module are transmitted to the decoding module for subsequent step-by-step reconstruction and identification of smoke and fire targets.

[0030] Specifically, the decoding module is used to upsample and reconstruct the deep features output by the bridging module step by step, and combine them with the feature information transmitted by the corresponding layer of the encoding module to gradually restore the spatial resolution and boundary details of the flue gas area and the fire point target, so as to obtain a high-resolution feature representation suitable for output discrimination.

[0031] Specifically, the decoding module includes three levels of feature recovery units connected in sequence. The first feature recovery unit includes a double residual depth separable convolution module, and the second and third feature recovery units include a large smoke and fire spatial information re-emission module and a double residual depth separable convolution module connected in sequence.

[0032] After being restored and fused step by step by the decoding module, the output features are transmitted to the output module, and after activation mapping, the prediction results of flue gas and fire point targets are generated.

[0033] Specifically, a skip connection module is set between the encoding and decoding modules. This skip connection module includes three background suppression and fireworks focusing modules. The first background suppression and fireworks focusing module is connected at one end to the double residual depthwise separable convolutional module in the first feature extraction unit of the encoding module, and at the other end to the third recovery unit of the decoding module. Correspondingly, the fireworks spatial information re-emission module and the double residual depthwise separable convolutional module in the third recovery unit are connected to the background suppression and fireworks focusing module. The second background suppression and fireworks focusing module is connected at one end to the fireworks feature fidelity downsampling module in the second feature extraction unit of the encoding module, and at the other end to the second recovery unit of the decoding module. Correspondingly, the fireworks spatial information re-emission module and the double residual depthwise separable convolutional module in the third recovery unit are connected to the background suppression and fireworks focusing module. The third background suppression and fireworks focusing module is connected at one end to the double residual depthwise separable convolutional module in the third feature extraction unit of the encoding module, and at the other end through the connection module of the decoding module, which is located at the input end of the decoding module.

[0034] In this embodiment, the double residual depthwise separable convolution module is used to enhance the extraction capability of fireworks features. The internal data processing process includes the following steps: First, perform the first residual depthwise separable convolution extraction, receiving the input variables. Apply scaling factor to it The kernel size is The first channel convolution feature is obtained by performing a channel convolution operation. Its mathematical expression is: Convolutional features of the first channel The input is fed into a GroupNorm (GN) layer for normalization, and then subjected to a non-linear mapping using an activation function to obtain the first intermediate feature. Its mathematical expression is: For the first intermediate feature Execution kernel size The point convolution operation is performed to obtain the first point convolution feature. Its mathematical expression is: Convolve the first point of the feature Passing through group normalization layer and activation function in sequence After processing, compared with the residual indicator function Weighted input variables Perform pixel-by-pixel addition and fusion to obtain the first convolution result variable. Its mathematical expression is: .

[0035] Next, a second residual depthwise separable convolution extraction is performed, extracting the variables from the first convolution result. As the new input, apply a scaling factor to it. The kernel size is The first channel convolution operation is performed to obtain the second channel convolution feature. Its mathematical expression is: Convolutional features of the second channel The input is fed into a group normalization layer for normalization and then processed by an activation function. A nonlinear mapping is performed to obtain the second intermediate feature. Its mathematical expression is: For the second intermediate feature The kernel size is [size to be filled in] The point convolution operation is used to obtain the second point convolution feature. Its mathematical expression is: Convolutional features at the second point Passing through group normalization layer and activation function in sequence After processing, compared with the residual indicator function The weighted first convolution result variable The pixels are added and fused one by one, and the final output is the variable of the second convolution. Its mathematical expression is: .

[0036] During the processing of the above modules, Let be the residual indicator function, where When the number of input channels of the module Number of output channels When they are equal, the residual indicator function A value of 1 initiates the corresponding residual connection operation; conversely, when the number of input channels is 1, the corresponding residual connection operation is initiated. Number of output channels When they are not equal, the residual indicator function The value is 2, thus disabling the corresponding residual join operation.

[0037] In this embodiment, the smoke feature fidelity downsampling module is used to preserve the subtle smoke edges and small fire points in forest and grassland fires as much as possible while downsampling, preventing information loss (i.e., aliasing effect) caused by downsampling. The internal data processing includes the following steps: First, receive input variables. Then, a fixed-kernel low-pass filter is applied to obtain a smoothed process variable. Its mathematical expression is: Preferably, a fixed-core low-pass filter. use The weight matrix is ​​specifically defined as follows:

[0038] Subsequently, the smoothing process variables were... Execute scaling factor The kernel size is The channel convolution operation is used to perform spatial downsampling to obtain the first downsampled variable. Its mathematical expression is: .

[0039] Next, for the first downsampled variable The kernel size is [size to be filled in] The point convolution operation yields the main branch process variables. Its mathematical expression is: .

[0040] In parallel, for the input variables Execute scaling factor Window size is The average pooling operation is used to perform spatial downsampling to obtain the second downsampled variable. Its mathematical expression is: .

[0041] Subsequently, the second downsampled variable The kernel size is [size to be filled in] The conventional convolution operation is used to adjust the channels, resulting in residual branch process variables. Its mathematical expression is: .

[0042] Main branch procedure variables With residual branch process variables Element-wise addition and fusion are performed, and the fused features are sequentially input into a group normalization layer and an activation function for nonlinear mapping. The final output is the downsampling result variable of this module. Its mathematical expression is:

[0043] In this embodiment, the large module for re-arranging the spatial information of fireworks uses channel rearrangement technology instead of traditional interpolation or deconvolution, effectively avoiding the checkerboard effect in the upsampling process. The internal data processing includes the following steps: First, regarding the input variables The kernel size is [size to be filled in] The conventional convolution operation reduces its number of channels from Expand to To obtain channel amplification variables Its mathematical expression is:

[0044] Next, channel amplification variables were applied. By applying a channel rearrangement operation with a scaling factor of 2, the elements in the channel dimension are periodically rearranged to the spatial dimension, thereby increasing the spatial resolution of the feature map by a factor of 2, and obtaining the variables of the upsampling process. Its mathematical expression is:

[0045] Next, the upsampling process variables The data are sequentially input into a group normalization layer for normalization, and then nonlinearly mapped using an activation function to obtain intermediate process variables. Its mathematical expression is:

[0046] Finally, to further eliminate potential spatial artifacts after rearrangement and restore details of smoke and fire points, intermediate process variables were adjusted. Execute scaling factor The kernel size is Channel convolution operation yields spatially refined variables. Its mathematical expression is:

[0047] Subsequently, spatial refinement variables were applied. The kernel size is [size to be filled in] The point convolution operation performs cross-channel fusion, and the fusion result is processed sequentially through a group normalization layer and an activation function, finally outputting the upsampling result variable of this module. Its mathematical expression is:

[0048] In this embodiment, the multi-scale smoke and fire morphology perception module effectively utilizes the fire scene characteristics under different sensing fields by constructing a multi-branch parallel structure. The internal data processing includes the following steps: First, multi-scale feature extraction is performed in parallel on the input variables. The kernel size is [size to be filled in] The regular convolution is then performed, followed by group normalization and activation processing to obtain the first intermediate process variable. Its mathematical expression is:

[0049] At the same time, the input variables were respectively... Perform three sets of convolutional kernels with a size of [size missing]. And the expansion rates are respectively , , The dilated convolution operation captures target features at different scales through differentiated receptive fields, yielding the second to fourth intermediate process variables. Their mathematical expressions are as follows:

[0050]

[0051]

[0052] Next, for the input variables Perform global average pooling to extract global context information, then execute... Convolution, and finally upsampled to the level of the input variable using bilinear interpolation. Same spatial resolution ( ), to obtain the fifth intermediate process variable Its mathematical expression is:

[0053] In this embodiment, the background suppression and fireworks focusing module reconstructs and weights the feature map through parallel channel attention and spatial attention mechanisms. The internal data processing includes the following steps: For input variables Global average pooling is performed to compress the spatial dimension, followed by sequential processing through the first weight matrix. Mapping, activation function processing, and the second weight matrix The mapping is finally passed through the Sigmoid activation function. Output channel attention weights Its mathematical expression is:

[0054] Input variables With channel attention weights Element-wise multiplication is performed to recalibrate the features of each channel, resulting in channel-weighted features. Its mathematical expression is:

[0055] In parallel, for the input variables Perform space mapping operations And through the Sigmoid activation function Generate spatial attention weights Its mathematical expression is:

[0056] Input variables Spatial attention weights Element-wise multiplication is performed to highlight key spatial regions in the feature map, resulting in spatially weighted features. Its mathematical expression is:

[0057] Finally, the channel weighted features are... Spatial weighted features The module performs pixel-by-pixel addition and fusion, and finally outputs the result variable, the mathematical expression of which is:

[0058] The following is for reference. Figure 2 In another embodiment of the present invention, a method for monitoring forest and grassland fires is provided, comprising the following steps: S1: Acquire visible light and infrared images of the target area. Specifically, multispectral camera images with a spatial resolution of 16m provided by the Environment and Disaster Reduction Satellite-2 are used as visible light images, and infrared camera images with a spatial resolution of 48m or 96m are used as infrared images. To perform spatial registration of the two at a unified spatial resolution, the infrared camera images are resampled to a spatial resolution of 16m.

[0059] Subsequently, the data was converted into multi-band composite data: the blue (0.48μm), green (0.56μm), and red (0.66μm) bands from the multispectral camera images were extracted as the first feature channel set for the corresponding visible light band images; the near-infrared, shortwave infrared 1 (1.62μm), and shortwave infrared 2 (2.22μm) bands from the resampled infrared camera images were extracted as the second feature channel set for the corresponding infrared band images. After band registration of the first and second feature channel sets, they were synthesized into multi-band apparent reflectance data, which served as the multi-band composite data.

[0060] S2: Input the multi-band apparent reflectance data into the pre-trained multimodal fireworks detection network for feature extraction to obtain the feature extraction results.

[0061] In the multimodal smoke and fire detection network, a smoke and fire feature fidelity downsampling module, which includes fixed-kernel low-pass filtering and average pooling operations, is used for feature processing during feature downsampling and scaling. By combining the anti-aliasing properties of low-pass filtering with average pooling, this module can effectively improve the feature integrity of plume edges and small fire points while reducing the spatial resolution of the feature map, avoiding the loss of key subtle features caused by conventional max pooling.

[0062] In a preferred embodiment of the present invention, the specific process of feature processing using a fireworks feature fidelity downsampling module that includes fixed kernel low-pass filtering and average pooling operations is as follows: First, the downsampled input feature map input to the module is processed. Parallel processing is performed. In the main branch, a fixed-core low-pass filter is used. Downsampled input feature map Convolutional smoothing is performed to filter out high-frequency noise and suppress aliasing, thereby obtaining the first feature map. (i.e., smoothing process variables), its mathematical expression is:

[0063] Meanwhile, in the residual branch, average pooling operations (e.g., using...) are employed. (Average pooling with a window size of 2) is used to downsample the input feature map. The second feature map is obtained by directly downsampling and scaling the spatial dimension. Its mathematical expression is:

[0064] Finally, the first feature map (or intermediate features obtained after further feature extraction) and the second feature map (Or features adjusted by the channel) are fused. Through this dual-branch fusion method, the module can reduce spatial resolution while preserving the continuity of plume edges and the pixel integrity of small fire points to the greatest extent.

[0065] In a preferred embodiment of the present invention, to further improve the integrity of the plume edge and small fire points, a smoke feature fidelity downsampling module including fixed kernel low-pass filtering and average pooling operations is used for feature processing. The micro-calculation process specifically includes: in the main feature extraction branch: Using a preset fixed-weight smoothing matrix (i.e., fixed-kernel low-pass filter) ) for downsampled input feature map Perform convolution processing to obtain a smooth feature map. Preferably, the specific numerical configuration of the fixed-weight smoothing matrix is ​​as follows:

[0066] Subsequently, the smoothed feature map obtained after convolution processing... Perform downsampling convolutions with a scaling factor of 2 sequentially (i.e., depthwise separable channel convolutions). Obtain intermediate variables and performing convolution with a kernel size of Channel convolution (i.e., point convolution) Finally, the first feature map is obtained. Its corresponding mathematical expression is: , In the parallel residual downsampling branch: Downsampled input feature map Perform window as And average pooling with a step size of 2 ( ) Obtain the pooling feature map Subsequently, through Regular convolution ( Channel mapping is performed to align with the main branch channel dimensions, resulting in the second feature map. Its corresponding mathematical expression is: , In the feature fusion and output stage: The first feature map With the second feature map Perform element-wise addition ( The summed feature maps are sequentially input into a group normalization layer for feature normalization, and then subjected to nonlinear activation processing using an activation function to obtain the output feature map of the fireworks feature fidelity downsampling module. The overall mathematical expression for its integration is:

[0067] In a preferred embodiment of the present invention, in order to effectively preserve the underlying visual features and overcome the information loss caused by network deepening, the multimodal smoke and fire detection network includes a cross-layer connection background suppression and smoke and fire focusing module between corresponding layers in the feature downsampling and feature upsampling stages. This module can help the model effectively focus on smoke and fire information and suppress background noise from other ground features. Specifically, for the received cross-layer input feature map... The background suppression and fireworks focusing module's processing steps include: First, a channel feature weighted branch based on global spatial pooling: For cross-layer input feature maps Global spatial pooling is performed to extract global context information and compress spatial dimensions; subsequently, a two-level weight mapping (matrix) is applied. and ) and nonlinear activation (including activation functions) and Sigmoid activation function Calculate the channel dimension weights Input feature maps across layers With channel dimension weight Element-wise multiplication is performed to enhance key channels, resulting in a channel-weighted feature map. The mathematical expression for this branch is:

[0068]

[0069] The second step is a weighted branch based on spatial features derived from single-channel convolutional mapping: Simultaneously, for cross-layer input feature maps Perform single-channel convolution mapping (i.e., spatial convolution) ), and activated by the Sigmoid function Calculate the spatial dimension weights Input feature maps across layers Spatial Dimension Weights Pixel-by-pixel multiplication is performed to guide the network to focus on key spatial regions (such as smoke edges and tiny fire points), resulting in a spatially weighted feature map. The mathematical expression for this branch is:

[0070]

[0071] The third step is the fusion of dual attention features and cross-layer output: The channel weighted feature map obtained from the above independent calculations Spatial weighted feature map The feature maps are obtained by adding each pixel individually. Its mathematical expression is:

[0072] Finally, the fused feature maps As the output of the skip connection, it is passed to the corresponding layer in the feature upsampling stage to achieve the concatenation of low-level detailed features and high-level semantic features.

[0073] In a preferred embodiment of the present invention, to enhance the network's ability to perceive and separate the morphology of smoke and fire targets at different scales, the multimodal smoke and fire detection network includes a bridging module between the feature downsampling stage and the feature upsampling stage. This bridging module includes a multi-scale smoke and fire morphology perception module. Specifically, for the intermediate layer input feature map input to this module... The processing steps specifically include: First, multi-branch parallel feature extraction: By utilizing multiple dilated convolutional branches with different dilation rates (i.e., dilation rates), the input feature map of the intermediate layer is processed. Parallel convolution extraction is performed to obtain the morphological features of smoke and fire points corresponding to different receptive fields, resulting in multiple morphological feature maps. In this embodiment, it specifically includes five parallel branches: The kernel size is [size to be filled in] The first morphological feature map is obtained by conventional convolution. ; Execute three commands, each corresponding to a different void ratio. , , of Dilated convolution yields the second to fourth morphological feature maps. , , ; Global average pooling is performed to extract global contextual features, and then bilinear interpolation is used to upsample the features to the original size to obtain the fifth morphological feature map. .

[0074] After convolution or pooling, each of the above branches undergoes group normalization and non-linear activation processing in sequence. The second step is feature channel concatenation: The multiple morphological feature maps obtained in parallel above (i.e. , , , and The first step involves concatenating features along the channel dimension to integrate local details and macroscopic semantics, resulting in a multi-scale concatenated feature map. The third step is feature dimensionality reduction, fusion, and output. Through convolution mapping (i.e., projective convolution) The multi-scale stitched feature map is subjected to channel dimensionality reduction and feature fusion, followed by group normalization and nonlinear activation processing to finally obtain the multi-scale perceptual feature map. Its overall mathematical expression is:

[0075] Finally, the multi-scale perceived feature map The output is sent to the feature upsampling stage and used as the input features for the decoder.

[0076] In a preferred embodiment of the present invention, in order to better recover the detailed texture features of smoke and fire points during the feature restoration (decoding) stage and avoid the "chessboard effect" easily caused by traditional deconvolution (transposed convolution), a large smoke and fire spatial information re-emission module is set at the output end of the bridging module of the multimodal smoke and fire detection network and the corresponding layer in the feature upsampling stage. Specifically, for the feature map to be enlarged input to the large smoke and fire spatial information re-emission module... The processing steps include: First, feature channel expansion: To enlarge feature map Perform channel augmentation convolution (i.e.) Regular convolution ), and change its feature channel number from Expand to a preset multiple (e.g., expand to 4 times the original number of channels, i.e.) Thus, a wide-channel feature map is obtained. Its mathematical expression is:

[0077] The second step involves large-scale subpixel space re-arrangement. According to the preset spatial rearrangement rules, the wide-channel feature map is rearranged using channel rearrangement operations. Pixels in the channel dimension are periodically filled to the corresponding sub-pixel positions in the spatial dimension to achieve lossless scaling of the spatial resolution of the feature map (e.g., scaling up the length and width by a factor of 2), resulting in a preliminary rearranged feature map with scaled spatial resolution. Its mathematical expression is:

[0078] The third step, voltage regulation activation and nonlinear mapping: For the initial rearrangement of feature maps The data is sequentially input into a group normalization layer for processing, and then processed through an activation function ( Nonlinear activation processing is performed to stabilize the data distribution and improve expressive power, resulting in a voltage-regulated activation feature map. Its mathematical expression is:

[0079] Step 4: Feature smoothing and detail refinement: To eliminate pixel discontinuities or artifacts that may result from spatial rearrangement, depthwise convolution (i.e., scaling factor) is used. The kernel size is Channel convolution ) Activation feature map of voltage regulation Perform independent spatial feature extraction for each channel to obtain a refined spatial feature map. Its mathematical expression is:

[0080] Step 5, cross-channel fusion and high-resolution output: Finally, pointwise convolution (i.e., convolution kernel size is...) is used. Point convolution Spatial refinement feature map Cross-channel feature fusion is performed, and the feature maps obtained after cross-channel feature fusion are then subjected to group normalization and nonlinear activation processing in sequence to obtain the high-resolution feature map output by the large-scale smoke and fire spatial information re-emission module. Its mathematical expression is:

[0081] In a preferred embodiment of the present invention, in order to enhance the ability to extract fireworks features and retain more details while reducing the number of parameters, the backbone feature extraction unit of the multimodal fireworks detection network is a double residual depthwise separable convolutional module. The backbone input feature map is input to the double residual depthwise separable convolutional module. Its processing includes two cascaded residual extraction stages: Phase 1: Initial feature extraction and addition of the first residual to the main input feature map. Scaling factor The kernel size is Depth convolution ( Then, group normalization and nonlinear activation processing are performed sequentially to obtain the first process feature map. Its corresponding mathematical expression is: , .

[0082] Using convolution kernel size Pointwise convolution ( ) Feature map of the first process Cross-channel fusion is performed to obtain the second process feature map. Its corresponding mathematical expression is: .

[0083] Feature map of the second process After performing group normalization and nonlinear activation processing in sequence, a gating mechanism based on input-output channel consistency is applied. The processed feature map is compared with the backbone input feature map. The first residuals are summed to obtain the first-order output feature map. Its corresponding mathematical expression is: Second stage: The second-order feature map is extracted and added to the second residual. As input, perform a depthwise convolution ( Then, group normalization and nonlinear activation are performed sequentially to obtain the feature map of the third process. Its corresponding mathematical expression is:

[0084]

[0085] Using pointwise convolution ( ) for the third process feature map Cross-channel fusion is performed to obtain the fourth process feature map. Its corresponding mathematical expression is:

[0086] Feature map of the fourth process After performing group normalization and nonlinear activation processing in sequence, a gating mechanism based on input-output channel consistency is applied. The processed feature map is compared with the first-order output feature map. A second residual summation is performed to obtain the final output feature map of the double residual depthwise separable convolutional module. Its corresponding mathematical expression is:

[0087] Regarding the gating mechanism: Preferably, an input / output channel consistency gating mechanism is used. (in The mathematical definition of ) is: This mechanism means that the input feature map channel number () is valid if and only if the input ... ) and the number of feature map channels to be output in this stage ( When they are equal, If the value is 1, then the corresponding first residual addition or second residual addition is initiated to fuse multi-level features; conversely, if the number of channels of the two are not equal (i.e., the dimensions do not match), then... The value is 0, thus automatically disabling the corresponding residual addition operation.

[0088] In conventional UNet and other classic basic models, downsampling operations are typically performed four or more times to acquire deep semantic information. However, in routine satellite remote sensing monitoring of forest and grassland fires, early initial fire points are often extremely small, frequently going undetected due to fire areas being less than a pixel in size (i.e., small fire points). Considering that excessive spatial compression can easily lead to blurring or even irreversible loss of these tiny fire point pixels and subtle smoke plume edge features in deep networks, this invention specifically modifies the number of downsampling operations in the basic model from the conventional four to three. By limiting the total number of feature downsampling scaling operations to three, the degree of spatial resolution attenuation of the feature map can be effectively controlled, thereby enhancing the semantic perception of smoke and fire point detection while preserving more smoke and fire details. This improvement at the macroscopic architectural level significantly enhances the model's sensitivity and detection accuracy for small fires under complex conditions such as lightning strikes and initial fires in mountainous areas.

[0089] S3: Based on the feature extraction results output by the multimodal smoke and fire detection network, smoke and fire points are identified and located separately. Specifically, based on the feature responses of the first feature channel set (blue, green, and red bands) in the extraction results, smoke region pixels in the image are detected and located; simultaneously, based on the feature responses of the second feature channel set (near-infrared, shortwave infrared 1, and shortwave infrared 2 bands) in the extraction results, fire point region pixels in the image are detected and located.

[0090] Finally, the smoke area and the fire point area are spatially overlaid to generate a forest and grassland fire monitoring distribution map of the target area (i.e., a forest and grassland fire smoke and fire point distribution map), thereby achieving efficient extraction and utilization of real-time fire information. The two results can both verify each other and complement each other.

[0091] To verify the detection performance of the multimodal smoke and fire detection model described in this invention under complex conditions, this embodiment uses four comprehensive accuracy indicators to quantitatively evaluate the detection results of smoke and fire points. The specific calculation formulas are as follows: Precision rate ( ):

[0092] Recall rate ( ):

[0093] Comprehensive evaluation indicators:

[0094] (Intersection over Union) Evaluation Indicators:

[0095] In the above formula, To accurately predict the number of pixels in the foreground (i.e., the number of correctly detected smoke or fire pixels); The number of mispredicted pixels in the foreground (i.e., the number of falsely reported pixels); The model predicts the number of background erroneous pixels (i.e., the number of missed pixels). By introducing core modules such as improved anti-aliasing residual downsampling, multi-scale morphology perception, and sub-pixel upsampling, the model of this invention has achieved significant improvements in all the above indicators, effectively realizing the purpose of accurate monitoring and early warning of real-time fire information.

[0096] In one embodiment of the present invention, a model training method for the pre-trained multimodal smoke and fire detection network described above is provided. This method addresses the problem of model convergence difficulties or low detection accuracy caused by the extremely small proportion of smoke and fire points in satellite imagery by introducing a deep supervision mechanism and a hybrid weighted loss function. The specific training steps are as follows: First, a training sample set was acquired and constructed. This set included multiple multi-band composite images containing visible and infrared wavelengths, along with corresponding real-world masking for smoke and fire points. Specifically, using the previously collected Landsat8OLI fire point dataset and the self-constructed GF-6WFV smoke dataset, smoke and fire regions in the corresponding images were manually delineated. A binary masking model with the same size as the input image was then generated, and pixels representing smoke and fire regions and background features were labeled separately.

[0097] The multi-band synthetic data from the training sample set is input into the multimodal smoke detection network to be trained for forward propagation. During the forward propagation process, the network intelligently mines smoke features through the following core components: Fireworks feature fidelity downsampling module: Reduces spatial dimension during the encoding stage, while using low-pass filtering and residual branching to improve the feature integrity of plume edges and small fire points.

[0098] Multi-scale smoke and fire morphology perception module: By using convolutional branches with different porosity at network bottlenecks, the ability to perceive and separate the morphology of smoke and fire targets at different scales is enhanced.

[0099] Background suppression and smoke focusing module: On the jump connection path, background noise of ground objects is suppressed through channel and spatial attention mechanisms, guiding the model to focus on key smoke areas.

[0100] The large module for re-arranging spatial information of smoke and fire: During the decoding stage, the spatial resolution is restored through channel re-arrangement technology to better recover the detailed texture features of smoke and fire points.

[0101] To achieve deep supervision, the multimodal fireworks detection network to be trained outputs multiple branches at the output end: the final output layer of the network outputs a principal probability image with the same size as the input image. By using different scale feature layers (e.g., half-scale and quarter-scale) within the network's internal bridging module, corresponding auxiliary probability images are output. and .

[0102] The mixed weighted total loss is calculated based on the output probability image and the ground truth labeled mask. This constrains the semantic learning process of the network. The calculation process is as follows: 1. Global pixel classification loss: The classification loss of the main output and auxiliary output is calculated using the binary cross-entropy loss function. Its mathematical expression is:

[0103] in, The probability value output by the network. For the corresponding actual labeled value, This represents the total number of pixels at the corresponding scale. The main classification loss is calculated. First auxiliary classification loss and second auxiliary classification loss .

[0104] 2. Loss of overlap between smoke and fire areas: The Dice loss is calculated on the main output probability image to assess the overlap between the predicted smoke area and the actual area, thereby improving the focus on small targets. Its mathematical expression is:

[0105] in, This is a smoothing factor to prevent the denominator from being zero.

[0106] 3. Mixed weighted total loss: The total loss is obtained by weighted summing of the above losses:

[0107] in, The pre-defined overlap weights, and These are auxiliary weighting coefficients at different scales.

[0108] Based on the calculated mixed weighted total loss The gradient backpropagation algorithm is executed to update the convolutional kernel weights, normalization layer parameters, etc. in the network to be trained through the optimizer. Steps one to five above are repeated to iterate the training sample set until the classification loss of the model on the validation set no longer decreases or reaches the preset number of training rounds (model convergence), and finally the trained multimodal fireworks detection network is obtained.

[0109] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A method for monitoring forest and grassland fires, characterized in that, Includes the following steps: The visible light band image and infrared band image of the target area are acquired, spatially registered at a uniform spatial resolution, and converted into multi-band composite data containing a first feature channel set corresponding to the visible light band image and a second feature channel set corresponding to the infrared band image. The multi-band synthesized data is input into a pre-trained multimodal fireworks detection network for feature extraction to obtain feature extraction results. The multimodal fireworks detection network uses a fireworks feature fidelity downsampling module that includes fixed kernel low-pass filtering and average pooling operations for feature processing when performing feature downsampling and scaling. Based on the feature extraction results, the smoke area is located according to the first feature channel set in the extraction results, and the fire area is located according to the second feature channel set. The smoke area and the fire area are then spatially superimposed to generate a forest and grassland fire monitoring distribution map.

2. The forest and grassland fire monitoring method according to claim 1, characterized in that, The feature processing employs a smoke feature fidelity downsampling module that includes fixed-kernel low-pass filtering and average pooling operations, including: The downsampled input feature map input to the fireworks feature fidelity downsampling module is low-pass filtered to obtain the first feature map; Simultaneously, average pooling is performed on the downsampled input feature map to obtain a second feature map; The first feature map and the second feature map are fused together.

3. The forest and grassland fire monitoring method according to claim 2, characterized in that, The feature processing employs a fireworks feature fidelity downsampling module that includes fixed-kernel low-pass filtering and average pooling operations, specifically including: The downsampled input feature map is convolved using a preset fixed-weight smoothing matrix, and the feature map obtained after the convolution process is then subjected to downsampling convolution and channel convolution in sequence to obtain the first feature map. After performing average pooling on the downsampled input feature map, the channel dimensions are aligned through convolutional mapping to obtain the second feature map; The first feature map and the second feature map are added element by element, and the added feature map is then subjected to feature normalization and nonlinear activation processing in sequence to obtain the output feature map of the fireworks feature fidelity downsampling module.

4. The forest and grassland fire monitoring method according to claim 1, characterized in that, The multimodal fireworks detection network has a cross-layer connection between the corresponding layers of the feature downsampling stage and the feature upsampling stage, which includes background suppression and fireworks focusing modules. The background suppression and fireworks focusing module's processing includes: Receive the cross-layer input feature map output from the corresponding layer during the feature downsampling stage, perform global spatial pooling on the cross-layer input feature map and calculate the channel dimension weights to obtain a channel-weighted feature map; Simultaneously, the cross-layer input feature map is subjected to single-channel convolution mapping and spatial dimension weights are calculated to obtain a spatially weighted feature map; The channel-weighted feature map and the spatial-weighted feature map are added element by element to obtain a fused feature map, and the fused feature map is output to the corresponding layer of the feature upsampling stage.

5. The forest and grassland fire monitoring method according to claim 1, characterized in that, A bridging module is provided between the feature downsampling stage and the feature upsampling stage of the multimodal smoke and fire detection network. The bridging module is equipped with a multi-scale smoke and fire morphology perception module. The processing steps of the multi-scale smoke and fire pattern sensing module include: By using multiple dilated convolution branches with different expansion rates, parallel convolution extraction is performed on the intermediate layer input feature map input to the multi-scale smoke and fire morphology perception module to obtain the smoke and fire morphology features corresponding to different receptive fields, resulting in multiple morphology feature maps. The multiple morphological feature maps are spliced ​​together along the channel dimension to obtain a multi-scale spliced ​​feature map. The multi-scale stitched feature map is subjected to channel dimensionality reduction and feature fusion by convolutional mapping to obtain a multi-scale perceptual feature map, and the multi-scale perceptual feature map is output to the feature upsampling stage.

6. The forest and grassland fire monitoring method according to claim 1, characterized in that, The output of the bridging module of the multimodal smoke and fire detection network and the corresponding layer of the feature upsampling stage are both equipped with a large smoke and fire spatial information re-emission module. The processing steps of the large module for re-emissioning information on the smoke and fire space include: The feature map to be enlarged, which is input to the large-scale smoke and fire space information re-emission module, is subjected to channel expansion convolution to obtain a wide-channel feature map with a preset number of channels; According to the preset spatial rearrangement rules, the pixels of the wide channel feature map in the channel dimension are filled to the corresponding sub-pixel positions in the spatial dimension to obtain a preliminary rearranged feature map with increased spatial resolution. The preliminary rearranged feature map is subjected to group normalization and nonlinear activation processing in sequence to obtain the voltage-stabilized activation feature map; The voltage-stabilized activation feature map is extracted channel-by-channel independently using depthwise convolution to obtain a spatially refined feature map. The spatially refined feature map is fused across channels using pointwise convolution, and the feature map obtained after the cross-channel feature fusion is then subjected to group normalization and nonlinear activation processing in sequence to obtain the high-resolution feature map output by the large-scale smoke and fire spatial information re-emission module.

7. The forest and grassland fire monitoring method according to claim 1, characterized in that, The multimodal smoke and fire detection network uses a dual residual depthwise separable convolutional module as its backbone feature extraction unit; the processing procedure of the dual residual depthwise separable convolutional module includes: The backbone input feature map input to the double residual depth-separable convolution module is subjected to depth convolution, and then group normalization and nonlinear activation are performed sequentially to obtain the first process feature map. The first process feature map is fused across channels using pointwise convolution to obtain the second process feature map; After performing group normalization and nonlinear activation processing on the feature map of the second process in sequence, the processed feature map is added to the main input feature map by the first residual based on the input-output channel consistency gating mechanism to obtain the first-order output feature map; The first-order output feature map is subjected to depthwise convolution, followed by group normalization and nonlinear activation to obtain the third-order feature map. The feature map of the third process is fused across channels using pointwise convolution to obtain the feature map of the fourth process; After performing group normalization and nonlinear activation processing on the feature map of the fourth process in sequence, based on the input-output channel consistency gating mechanism, the processed feature map is added to the first-order output feature map by the second residual to obtain the final output feature map of the double residual depth separable convolution module. The input / output channel consistency gating mechanism refers to the following: when the number of feature map channels input to this stage is equal to the number of feature map channels about to be output to this stage, the corresponding first residual addition or second residual addition is initiated.

8. The forest and grassland fire monitoring method according to claim 1, characterized in that, The feature downsampling and scaling of the multimodal fireworks detection network is performed a total of three times.

9. A model training method, characterized in that, Includes the following steps: Obtain a training sample set, which includes multiple multi-band composite data containing visible light and infrared bands, as well as real labeled masks of smoke and fire points corresponding to the multi-band composite data. The multi-band synthetic data in the training sample set is input into the multi-modal smoke and fire detection network to be trained for forward propagation. During the forward propagation process, the multi-modal smoke and fire detection network to be trained extracts and restores features from the multi-band synthetic data through a smoke and fire feature fidelity downsampling module, a background suppression and smoke and fire focusing module, a multi-scale smoke and fire morphology perception module, and a large smoke and fire spatial information re-emission module. The main probability image is output through the terminal output layer of the multimodal fireworks detection network to be trained, and auxiliary probability images of different scales are output through multiple bridging modules of the multimodal fireworks detection network to be trained. Based on the main probability image and the real labeled mask, calculate the main global pixel classification loss and the smoke region overlap loss; Based on the auxiliary probability image and the ground truth labeled mask, the auxiliary global pixel classification loss is calculated respectively. The main global pixel classification loss, the smoke region overlap loss, and the weighted auxiliary global pixel classification loss are summed to obtain the mixed weighted total loss. Gradient backpropagation is performed based on the hybrid weighted total loss to update the weight parameters of the multimodal fireworks detection network to be trained until the model converges, thus obtaining the trained multimodal fireworks detection network.

10. The model training method according to claim 9, characterized in that, The primary global pixel classification loss and the auxiliary global pixel classification loss are both calculated using the binary cross-entropy loss function, and the smoke and fire region overlap loss is calculated using the Dice loss function. The weighted total loss is obtained by summing the main global pixel classification loss, the overlap loss of the fireworks region with a preset overlap weight, and multiple auxiliary global pixel classification losses with different scales and corresponding auxiliary weight coefficients. The auxiliary global pixel classification loss at different scales includes: a first auxiliary global pixel classification loss corresponding to the output of the half-scale feature layer, and a second auxiliary global pixel classification loss corresponding to the output of the quarter-scale feature layer.

Images