A remote sensing image enhancement method for forest fire monitoring

Abstract

The present application relates to the technical field of image enhancement, in particular to a remote sensing image enhancement method for forest fire monitoring, which specifically comprises the following steps: firstly, pre-processing multispectral images, digital elevation models and fire point labels to obtain multi-source training samples; then, performing differentiated enhancement on different bands in combination with terrain shadow intensity, normalized burning ratio and short-wave infrared local anomaly response; then, generating a spatially restricted fire line expansion area based on wind direction, slope direction and vegetation conditions, and performing thermal radiation enhancement and smoke plume shielding modulation to obtain physically simulated augmented samples; finally, constructing a double-branch dynamic receptive field network for decoupling fire points and smoke, and jointly training the network using focal loss and boundary consistency loss, and outputting a pixel-level fire danger probability map. The present application can improve the detection accuracy of forest fires under complex terrain and the enhanced results can be directly used for fire point detection or auxiliary manual interpretation.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and in particular to a remote sensing image enhancement method for forest fire monitoring. Background Technology

[0002] In practical applications of mountain forest fire monitoring, commonly used multispectral remote sensing data, digital elevation models, and manually labeled fire points often come from different data sources. These data exhibit significant inconsistencies in spatial resolution, projected coordinate systems, and effective coverage. Directly using this data for subsequent terrain shadow analysis, fire point feature enhancement, and deep learning network training can easily lead to spatial misalignment, boundary drift, and discrepancies between labels and images, severely impacting fire identification accuracy. Furthermore, the complex terrain of mountainous areas results in severe mountain shadow effects. Images of shady slopes and valleys simultaneously exhibit low brightness and low contrast, while understory smoldering fires and smoke-covered areas also display similar spectral characteristics. Existing technologies commonly employ full-band uniform normalization or global enhancement methods, which struggle to distinguish between ordinary terrain shadows and actual combustion anomalies within shadows, easily leading to the suppression of weak fire points or the misdetection of ordinary shadows as fire points. In terms of training samples, the number of samples of small fire spots, thin fire lines and weak fire spots on shady slopes in real forest fire scenarios is extremely small. Conventional data augmentation methods such as rotation and flipping can only change the appearance of the samples and cannot generate fire line expansion patterns that conform to the real wind direction, slope aspect and combustible material distribution patterns, resulting in insufficient learning ability of the model for complex fire line boundaries.

[0003] Existing technologies objectively suffer from the following shortcomings: Multispectral images, digital elevation models, and fire point labels originate from different sources, resulting in inconsistent spatial resolutions and projected coordinate systems. The lack of unified registration in existing technologies leads to spatial misalignment and label mismatches in subsequent processing. Current methods employ uniform normalization across all bands, easily suppressing genuine weak fire points or mistakenly enhancing ordinary mountain shadows as suspected fire points, failing to distinguish between ordinary shadows and actual combustion anomalies within them. Conventional augmentation methods such as rotation and flipping cannot generate fireline patterns that conform to actual wind direction, slope aspect, and combustible material distribution constraints, making it difficult to effectively expand high-value samples such as small fire points and thin fire lines. Existing fire monitoring networks learn all bands within the same branch using the same receptive field, making small fire points easily obscured by smoke textures, and causing unstable plume boundaries due to insufficient receptive field, leading to feature competition between fire points and smoke.

[0004] Therefore, this invention proposes a remote sensing image enhancement method for forest fire monitoring to solve the above problems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a remote sensing image enhancement method for forest fire monitoring. This invention can improve the detection accuracy of forest fires in complex terrain, and the enhancement results can be directly used for fire point detection or to assist in manual interpretation.

[0006] The technical solution of this invention to solve the technical problem is a remote sensing image enhancement method for forest fire monitoring, comprising the following steps:

[0007] S1. Perform unified projection, unified resolution, and sample slicing on multispectral images, digital elevation models, and fire point labels in forest fire monitoring to obtain multi-source training samples for forest fires. Each training sample includes multispectral samples, digital elevation models, effective area masks, and fire point label maps.

[0008] S2. Calculate the terrain shadow intensity based on the digital elevation model, and then combine the normalized combustion ratio and the local anomaly response of the shortwave infrared band to perform differential enhancement on the shortwave infrared band, visible light band and vegetation structure band to obtain enhanced multispectral samples.

[0009] S3. Based on the enhanced multispectral samples and fire point label map, a spatially confined fire line expansion area is generated according to wind direction, slope aspect and vegetation conditions. Thermal radiation enhancement and smoke plume shading modulation are performed on the expansion area to obtain a physically simulated augmented sample.

[0010] S4. Construct a dual-branch dynamic receptive field network for decoupling fire point and smoke, use training samples as input, verify the improvement effect of the enhanced samples on the actual fire point detection performance, and output a pixel-level fire risk probability map; use focus loss and boundary consistency loss to jointly optimize the network parameters and train and optimize the dual-branch dynamic receptive field network.

[0011] S5. Acquire new temporal remote sensing images of the area to be monitored. After image enhancement, use them for the trained bi-branch dynamic receptive field network or as a visualization base map to assist manual interpretation.

[0012] S1 is as follows;

[0013] Atmospheric correction is performed on multispectral images, the required bands are extracted and cropped according to spatial range, and the atmospherically corrected surface reflectance image is defined as a multispectral sample. The multispectral sample represents the multiband input data used for forest fire identification.

[0014] The digital elevation model is reprojected to the same projection coordinate system as the multispectral sample, and then bilinear interpolation or cubic convolution interpolation is used to resample the digital elevation model to a uniform spatial resolution.

[0015] Perform uniform resampling on all bands of the multispectral sample, and then extract cloud, cloud shadow, water body and invalid boundary information from the quality control information corresponding to the multispectral sample to generate an effective region mask;

[0016] Fire point labels, either manually marked or corrected from existing fire records, are defined as fire point label maps.

[0017] A fixed-size sliding window is used to crop training slices from the whole scene image, and each training slice is a training sample.

[0018] In step S2, the light intensity is estimated using a digital elevation model, and then combined with the normalized vegetation index and normalized burn ratio to generate a topographic shadow intensity map and a shadow area type map, as detailed below:

[0019] The solar azimuth and solar altitude angles are read from the image metadata corresponding to the multispectral samples to determine whether each pixel is in the direct sunlight area or the shadow area; the Sobel operator is used to calculate the lateral elevation change and the longitudinal elevation change in the digital elevation model in a 3×3 neighborhood, and then the slope amplitude and aspect angle are synthesized from the lateral gradient and the longitudinal gradient; the light intensity of each pixel is calculated based on the slope, aspect, solar azimuth and solar altitude angle, and then the light intensity is mapped to a normalized shadow intensity map;

[0020] Normalized Difference Vegetation Index (NDVI) and Normalized Burning Ratio (NBR) were calculated based on the near-infrared, red, and short-wave infrared reflectance in the multispectral samples.

[0021] Based on the normalized shadow intensity map S, the normalized burn ratio NBR, and the local anomaly response of shortwave infrared, each pixel is classified into region types, including ordinary shadow area, direct area, transition area, and shadow burn candidate area. The region type results are saved as a region type map.

[0022] The calculation process for enhancing multispectral samples is as follows:

[0023] Effective pixels are screened out from the effective area mask, and then healthy vegetation reference areas are selected from the effective pixels. The mean value of healthy vegetation background in each band is calculated and defined as the vegetation background reference reflectance. At each effective pixel location, a local neighborhood is established with the current location as the center, and the local background fluctuation intensity in each band is calculated.

[0024] Background correction and local normalization are performed on the multispectral samples band by band to obtain intermediate normalization results. The intermediate normalization results represent the multispectral samples after removing vegetation background bias and unifying them according to the local fluctuation scale.

[0025] For each region type in the regional type map, shortwave infrared enhancement coefficient and visible light suppression and compensation coefficient are constructed. Vegetation structure retention coefficient is constructed to adjust the enhancement intensity of the red edge band and near-infrared band. Slope suppression coefficient is constructed according to the terrain slope. The intermediate normalized results are applied to the shortwave infrared enhancement coefficient, visible light suppression and compensation coefficient, and vegetation structure retention coefficient according to the band category, and then multiplied by the slope suppression coefficient to obtain the enhanced multispectral sample.

[0026] Based on the fire point label map, digital elevation model, and vegetation conditions, a spread direction field is generated, and the fire point area is expanded accordingly. The specific operation is as follows:

[0027] Using enhanced multispectral samples, digital elevation models, and fire point label maps as input, the fire point label maps are first subjected to noise removal, and then connected component analysis is performed to extract the ignition source location from each fire point connected component. The ignition source location represents the spread initiation point. Then, the combustible weight map is calculated from the enhanced multispectral samples, and the slope aspect is calculated from the digital elevation model. The wind speed and wind direction data corresponding to the sample imaging time are also read.

[0028] With each ignition source as the center, discrete directions are divided, and the spread priority is calculated in each discrete direction. For each direction, the basic spread length is first set, and then adjusted according to the degree of wind direction consistency, uphill degree and combustible weight.

[0029] Grid expansion is performed based on the spread priority of each direction to obtain an expanded fire point label map. The expanded fire point label map represents the expanded fire line area after being constrained by wind direction, slope aspect, and combustible material. The expansion results are then constrained by low combustible material areas and significant boundaries. Finally, edge smoothing is performed on the expanded fire point label map to obtain a soft boundary expansion map.

[0030] The specific constraints of the extended results are as follows:

[0031] If a location to be expanded falls in a low-combustible area, or is located on a significant land feature boundary characterized by both red-edge and near-infrared band gradients, its priority for inclusion in the expansion area is reduced. If the priority is below a threshold, expansion to that location will no longer proceed.

[0032] The calculation process for augmented samples in physical simulation is as follows:

[0033] Based on the soft boundary expansion map, the fire core area, fire line edge area, and heat-affected zone are constructed. A plume area is constructed on the downwind side of the fire line according to the prevailing wind direction. The plume area represents the background area affected by smoke obscuring. Then, thermal radiation enhancement is performed on the shortwave infrared band, weak enhancement or preservation processing is performed on the red edge band and near-infrared band, and plume transmittance attenuation is performed on the visible light band. Then, thermal radiation enhancement and plume transmittance attenuation are applied together to enhance the multispectral sample to obtain the physically simulated augmented sample. The augmented label corresponding to the physically simulated augmented sample is defined as the augmented fire point label map.

[0034] The dual-branch dynamic receptive field network includes: a band splitting module, a fire point branch, and a smoke branch;

[0035] The band splitting module generates fire point branch weights and smoke branch weights for each band based on band statistics and a learnable routing network.

[0036] The fire point branch receives the weighted fire point input tensor, extracts small-scale high-temperature features through two 3×3 convolutional blocks and multi-scale dilated convolution, and outputs a fire point feature map and a fire point response map.

[0037] The smoke branch receives the weighted smoke input tensor, first extracts preliminary features through two 5×5 convolutional blocks, and then enters three parallel context branches: 3×3 convolution, 5×5 convolution, and 3×3 convolution with a dilation rate of 2. The feature maps output by the three branches are dynamically weighted and fused by the fire response map to obtain the smoke feature map.

[0038] The specific operations for band splitting and bi-branch feature extraction are as follows:

[0039] Training samples include real samples and augmented samples. When using real samples, the corresponding input is an enhanced multispectral sample. When using augmented samples, the corresponding input is a physical simulation augmented sample.

[0040] The training samples are input into the band splitting module. Four global statistics are extracted for each band as the basis for routing decisions, including global mean, global standard deviation, 95th percentile, and center wavelength encoding. The four-dimensional statistical vector of each band is then input into a two-layer fully connected routing network to output the fire branch weight and smoke branch weight.

[0041] Each band of the training input sample is multiplied by its corresponding weight and then stacked in the original band order. A fire point input tensor is generated based on the fire point branch weights of all bands, and a smoke input tensor is generated based on the smoke branch weights of all bands. The fire point input tensor is then input into the fire point branch to obtain the fire point feature map and the fire point response map.

[0042] Input the smoke tensor into the smoke branch, generate dynamic receptive field selection weights based on the fire response map, and perform pixel-by-pixel weighted fusion on the outputs of the three smoke context branches according to the receptive field selection weights to obtain the smoke feature map.

[0043] By fusing the fire point feature map, smoke feature map, and fire point response map, a pixel-level fire risk probability map is obtained.

[0044] The calculation process of the network's loss function is as follows:

[0045] The training label for the enhanced multispectral sample is the fire point label map, and the training label for the physical simulation augmented sample is the augmented fire point label map. The focal loss is calculated based on the training label map and the pixel-level fire risk probability map.

[0046] The system constructs true boundary targets based on the training label map, performs Euclidean distance transformation on the training label map to obtain a distance map, and then uses the Sobel operator to extract the boundary gradient map as the true boundary target. The Sobel operator is also used to extract the predicted boundary gradient map on the pixel-level fire risk probability map. The mean square error of the boundary gradient map and the predicted boundary gradient map is calculated to obtain the boundary consistency loss.

[0047] The total loss is obtained by weighted summation of the focus loss and the boundary consistency loss.

[0048] The effects described in the invention are merely those of the embodiments, and not all the effects of the invention. The above technical solutions have the following advantages or beneficial effects:

[0049] This invention discloses a remote sensing image enhancement method for forest fire monitoring. It performs unified projection, unified resolution, and slicing on multispectral images, digital elevation models, and fire point labels. Simultaneously, it extracts clouds, cloud shadows, and water bodies to generate effective area masks, solving the spatial misalignment problem of multi-source data. Based on the digital elevation model, it calculates the terrain shadow intensity and, combined with the normalized combustion ratio and local anomalies in shortwave infrared, subdivides the shadow area into ordinary shadow areas and shadow combustion candidate areas. Differential enhancement is performed on the shortwave infrared, visible light, and vegetation structure bands. A non-uniform spread direction field is constructed using wind direction, slope aspect, and vegetation conditions to spatially expand the fire point labels. Thermal radiation enhancement and plume masking modulation are simultaneously applied to the expanded area to generate augmented samples that conform to physical laws. A dual-branch network is constructed, assigning fire point weights and smoke weights to each band through band splitting. The fire point response map dynamically guides the smoke branch to select different receptive fields, and focal loss and boundary consistency loss are jointly optimized. Attached Figure Description

[0050] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0051] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0052] Figure 2 The following are sample images: Figure (a) is a schematic diagram of a multispectral sample, Figure (b) is a schematic diagram of the near-infrared band B8, and Figure (c) is a schematic diagram of a digital elevation model.

[0053] Figure 3 This is a characteristic map of light reception.

[0054] Figure 4 This is a normalized shadow intensity feature map.

[0055] Figure 5 This is a schematic diagram of vegetation activity indicators. Detailed Implementation

[0056] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0057] Example 1

[0058] like Figure 1 As shown, a remote sensing image enhancement method for forest fire monitoring includes the following steps:

[0059] S1. Perform unified projection, unified resolution, and sample slicing on multispectral images, digital elevation models, and fire point labels in forest fire monitoring to obtain multi-source training samples for forest fires. Each training sample includes multispectral samples, digital elevation models, effective area masks, and fire point label maps.

[0060] S2. Calculate the terrain shadow intensity based on the digital elevation model, and then combine the normalized combustion ratio and the local anomaly response of the shortwave infrared band to perform differential enhancement on the shortwave infrared band, visible light band and vegetation structure band to obtain enhanced multispectral samples.

[0061] S3. Based on the enhanced multispectral samples and fire point label map, a spatially confined fire line expansion area is generated according to wind direction, slope aspect and vegetation conditions. Thermal radiation enhancement and smoke plume shading modulation are performed on the expansion area to obtain a physically simulated augmented sample.

[0062] S4. Construct a dual-branch dynamic receptive field network for decoupling fire point and smoke, use training samples as input, verify the improvement effect of the enhanced samples on the actual fire point detection performance, and output a pixel-level fire risk probability map; use focus loss and boundary consistency loss to jointly optimize the network parameters and train and optimize the dual-branch dynamic receptive field network.

[0063] S5. Acquire new temporal remote sensing images of the area to be monitored. After image enhancement, use them for the trained bi-branch dynamic receptive field network or as a visualization base map to assist manual interpretation.

[0064] In a specific implementation, S1 is as follows:

[0065] In mountain forest fire monitoring, multispectral images, digital elevation models (DEMs), and fire point labels often come from different sources, and their spatial resolution, projection coordinate systems, and effective area ranges are inconsistent. Without prior unified registration and sample processing, subsequent terrain shadow analysis, fire point augmentation, and network training will encounter problems such as spatial misalignment, boundary drift, and label mismatch. This invention first performs unified projection, unified resolution, and sample slicing on multispectral images, DEMs, and fire point labels to obtain multispectral samples, DEMs, and fire point label maps, providing unified input for subsequent augmentation, broadening, and training. The specific steps are as follows:

[0066] S1.1 This invention takes the Sentinel-2 dataset as an example. It uses the Sen2Cor plugin provided by the European Space Agency or open-source atmospheric correction tools (such as MAJA, iCOR) to perform atmospheric correction on the Sentinel-2 L1C level product (atmospheric top reflectance) to generate L2A level surface reflectance product. The required bands are extracted from it and cropped according to the spatial range. The atmospherically corrected surface reflectance image is defined as multispectral sample X. Multispectral sample X represents the multi-band input data used for forest fire identification. The size is H×W×C, where H represents the sample height, W represents the sample width, and C represents the number of bands.

[0067] Twelve surface reflectance bands were selected as input, namely B1, B2, B3, B4, B5, B6, B7, B8, B8A, B9, B11, and B12, with C=12. Using these twelve surface reflectance bands instead of directly using cirrus bands avoids introducing unstable upper-level atmospheric information directly into the surface fire detection process. The meanings of the twelve surface reflectance bands of the Sentinel-2MSI sensor are shown in Table 1.

[0068] Table 1. Meaning of Reflectivity Bands

[0069]

[0070] S1.2 Define the digital elevation model that is consistent with the spatial coverage of the multispectral sample X as digital elevation model D. Digital elevation model D represents the topographic elevation map with a size of H×W, which is used for subsequent calculations of slope, aspect and topographic illumination.

[0071] In practice, the digital elevation model is first reprojected to the same projection coordinate system as the multispectral sample. Then, bilinear interpolation or cubic convolution interpolation is used to resample the digital elevation model to a uniform spatial resolution. Reprojection refers to converting the original geographic coordinate system of the digital elevation model (such as WGS84 latitude and longitude) into the same projection coordinate system as X (such as UTM zonation). Resampling refers to adjusting the pixel size of the digital elevation model to the target resolution. Bilinear interpolation is performed by weighted averaging of the surrounding 4 pixels, while cubic convolution is performed by fitting a surface to the surrounding 16 pixels. The latter is smoother but slightly slower to compute.

[0072] It should be noted that digital elevation models are raster data representing the elevation of the Earth's surface, which can be obtained from public data sources, such as NASASRTM (30 meters), ESA Copernicus digital elevation model (30 meters), or ALOSAW3D30 (30 meters). This invention reprojects the digital elevation model data corresponding to the spatial range of the multispectral image to the same coordinate system (such as UTM projection), and then resamples it to a spatial resolution consistent with X (such as 20 meters).

[0073] S1.3. Perform uniform resampling on all bands of the multispectral sample X to ensure that all bands are located on the same pixel grid. Specifically, considering that the shortwave infrared bands B11 and B12 are key bands for forest fire identification, in one implementation, all bands are unified to a spatial resolution of 20 meters. Bands with an original resolution of 10 meters are downsampled using area average, while bands with an original resolution of 20 meters remain unchanged. For example, when the original B2 (10 meters) is downsampled to 20 meters, the arithmetic mean (area average) of the four values ​​in each 2×2 pixel window is taken, while the original B11 (20 meters) remains unchanged. Based on this, the 20-meter shortwave infrared band is "magnified" to 10 meters through interpolation, artificially creating non-existent high-resolution details. This avoids simply magnifying key shortwave infrared bands and then mixing them with other bands, thereby reducing false details.

[0074] S1.4 Extract cloud, cloud shadow, water body and invalid boundary information from the quality control information corresponding to the multispectral sample X, and generate effective region mask K. Effective region mask K represents the effective pixel region that can participate in subsequent enhancement and training. The size is H×W, and the value is 0 or 1. The value of 1 represents an effective pixel, and the value of 0 represents a cloud, cloud shadow, water body or invalid boundary pixel.

[0075] In practical implementation, the Sentinel-2 scene classification layer can be used first; if the scene classification layer is not available, a mask can be constructed by combining near-infrared reflectance, short-wave infrared reflectance and blue light band threshold.

[0076] Taking the following scenario as an example, if the scene classification layer is missing, a threshold method can be used, as follows:

[0077] Cloud: B8 reflectance 0.25 and B4 reflectance 0.15;

[0078] Cloud shadow: B8 reflectance <0.10 and neighboring pixels are covered by clouds;

[0079] Water body: NDWI = (B3 - B8) / (B3 + B8)) 0.3;

[0080] Invalid boundary: 1-2 pixels outward from the four edges of the image;

[0081] If any condition is met, the effective region mask K value is set to 0; otherwise, it is set to 1.

[0082] S1.5 Define the manually labeled or corrected fire point labels from existing fire records as fire point label map M. Fire point label map M represents a pixel-level binary label map that corresponds one-to-one with the multispectral sample X. The size is H×W. A value of 1 represents a fire point pixel, and a value of 0 represents a non-fire point pixel.

[0083] In practice, the fire spot label map M can be generated based on short-wave infrared anomalies, time-series comparison results, and manual interpretation. For small fire spots with unclear boundaries, the high-temperature core area can be delineated first in the short-wave infrared pseudo-color combination map, and then the label boundary can be corrected pixel by pixel.

[0084] S1.6. Use a fixed-size sliding window to crop training slices from the whole scene image.

[0085] The window size can be 256×256 pixels, and the step size can be 128 pixels. For windows containing fire point labels, they are retained as positive sample slices. For windows that do not contain fire point labels but have normal vegetation, bare land, mountain shadows and smoke backgrounds, they are retained as negative sample slices according to a set ratio, thereby ensuring that the training set contains both "real fire point samples" and "complex background samples that are easy to be misdetected".

[0086] S1.7 Save four types of data for each training slice, including multispectral sample X, digital elevation model D, effective area mask K, and fire point label map M. At this point, the basic training sample set that can be directly entered into the enhancement module is obtained.

[0087] Taking the following scenario as an example, assuming the window position is (row 1000, column 2000) and the size is 256×256, then:

[0088] Multispectral sample X: a three-dimensional array (256,256,12) containing 12 bands from B1 to B12;

[0089] Digital elevation model D: a two-dimensional array of elevation values ​​(256, 256);

[0090] Effective region mask K: 0 / 1 binary mask two-dimensional array (256, 256);

[0091] Fire point label diagram M: 0 / 1 fire point label two-dimensional array (256, 256).

[0092] It should be noted that this invention solves the problem of "images are aligned but labels are not" that is most likely to occur in mountainous and forested scenes with multi-source data by first unifying spatial resolution and pixel grid, and then constructing effective region masks and fire point label maps, thus providing a unified data benchmark for subsequent steps.

[0093] like Figure 2 The diagram shown is a schematic of the multispectral sample, near-infrared band B8, and digital elevation model during the sample acquisition process in step S1.

[0094] In a specific implementation, S2 is as follows:

[0095] In images of mountain forest fires, shady slopes, valleys, smoke-covered areas, and smoldering areas under the forest can simultaneously exhibit low brightness, low contrast, and localized abnormal reflections. Applying uniform normalization to all bands can easily suppress genuine weak fire points or mistakenly enhance ordinary mountain shadows as suspected fire points. This invention first calculates the terrain shadow intensity based on a digital elevation model, then combines the normalized combustion ratio and local anomaly responses in the shortwave infrared band to perform differentiated enhancement on the shortwave infrared band, visible light band, and vegetation structure band, obtaining enhanced multispectral samples. The specific steps are as follows:

[0096] S2.1, Estimation of Topographic Shadow Intensity and Classification of Shadow Area Types

[0097] Shadows in forest fire scenes may come from mountain terrain or smoke obstruction. The two are similar in visible light, but have different effects on short-wave infrared and flammability index. If "ordinary terrain shadows" and "real combustion anomalies in shadows" are not distinguished first, subsequent enhancement will mix shadows with fire points.

[0098] This invention estimates the amount of sunlight received through a digital elevation model, and then combines this with the normalized vegetation index and normalized combustion ratio to generate a topographic shadow intensity map and a shadow area type map. The specific steps are as follows:

[0099] 1) Read the solar azimuth and solar altitude angle from the image metadata corresponding to the multispectral sample X. The solar azimuth angle is used to indicate the direction of the sun's incidence on the horizontal plane, and the solar altitude angle is used to indicate the angle at which the sun rises relative to the horizon. Based on the relationship between these two angles and the terrain normal, it is possible to determine whether each pixel is in the direct sunlight area or the shadow area.

[0100] In one embodiment, in the metadata file of the Sentinel-2 L1C product, for the <ZENITH_ANGLE> (zenith angle, 90° - altitude angle) and <AZIMUTH_ANGLE> (azimuth angle) under the field name <Mean_Sun_Angle> with the path " / Granule / … / MTD_TL.xml", reading this XML file can obtain the solar azimuth angle and solar altitude angle.

[0101] 2) Calculate the elevation gradient within a 3×3 neighborhood of the digital elevation model D, and obtain the slope and aspect from the elevation gradient. Specifically, the Sobel operator can be used to calculate the lateral elevation change and longitudinal elevation change respectively, and then the slope amplitude and aspect angle are synthesized from the lateral gradient and longitudinal gradient. Here, the slope is used to represent the terrain steepness, and the aspect is used to represent the slope orientation.

[0102] In specific implementation, for each pixel of the digital elevation model, calculate with the Sobel operator:

[0103] Lateral gradient = Elevation of the right pixel - Elevation of the left pixel;

[0104] Longitudinal gradient = Elevation of the lower pixel - Elevation of the upper pixel;

[0105] Slope = ;

[0106] Aspect = (Calculated clockwise from the north).

[0107] 3) According to the slope, aspect, solar azimuth angle and solar altitude angle, calculate the light-receiving degree of each pixel, and then map the light-receiving degree to the normalized shadow intensity map S. The normalized shadow intensity map S represents the shadow strength at each position, with a size of H×W and a value range of 0 to 1. The closer the value is to 1, the stronger the shadow, and the closer the value is to 0, the closer it is to the direct sunlight area.

[0108] In one implementation, first calculate the cosine value of the angle between the slope normal and the solar incident direction; then use piecewise linear mapping to generate S: when the cosine value of the angle is greater than or equal to 0.7, it is recorded as weak shadow or non-shadow; when the cosine value of the angle is less than or equal to 0.2, it is recorded as strong shadow; when it is between the two, it is mapped according to the linear ratio.

[0109] As an example, assume that the calculated cosine value of the angle of a certain pixel is 0.5. Then the calculation of the linear mapping is (0.5 - 0.2) / (0.7 - 0.2) = 0.6, that is, the value of the normalized shadow intensity map S is 0.6, belonging to the transition zone.

[0110] 4) Calculate the Normalized Difference Vegetation Index (NDVI) and Normalized Burn Ratio (NBR) based on the multispectral sample X for subsequent differentiation between ordinary shadows and real burning anomalies. The calculation method is expressed as:

[0111] ;

[0112] ;

[0113] where B8 represents the reflectance of the near-infrared band, B4 represents the reflectance of the red band, and B12 represents the reflectance of the short-wave infrared band. represents the numerical stability constant (which can take ); NDVI is used to characterize vegetation activity, and NBR is used to characterize the degree of burning. The burning area usually shows a decrease in NBR.

[0114] 5) Based on the normalized shadow intensity map S, the normalized burn ratio NBR, and the short-wave infrared local anomaly response, divide the region type of each pixel. The region types include ordinary shadow areas, direct sunlight areas, transition areas, and shadow burning candidate areas, which are used to separate "ordinary dark areas on the shaded slope" and "real weak fire points on the shaded slope"; in specific implementation, it can be executed in the following way:

[0115] Define (i,j) to represent the pixel position at the i-th row and j-th column in the image. S(i,j) represents the normalized shadow intensity value at position (i,j), and NBR(i,j) represents the normalized burn ratio at position (i,j);

[0116] When S(i,j) ≥ 0.7 and NBR(i,j) does not decrease significantly, divide position (i,j) into an ordinary shadow area;

[0117] When S(i,j) ≤ 0.3, divide position (i,j) into a direct sunlight area;

[0118] When 0.3 < S(i,j) < 0.7, divide position (i,j) into a transition area;

[0119] When S(i,j) ≥ 0.7, and NBR(i,j) < 0, and at the same time the B12 band at the current position is significantly higher than the average value of its 9×9 neighborhood, divide position (i,j) into a shadow burning candidate area.

[0120] It should be noted that in the conditions for the shadow burning candidate area, when judging that "the B12 band is significantly higher than the 9×9 neighborhood average value", let the neighborhood average value be , and the neighborhood standard deviation be , when it is considered significantly higher than the neighborhood.

[0121] In one embodiment, for example, at a certain location (i,j), if S(i,j)=0.84, NBR(i,j)=0.28, and the B12 band value is close to the surrounding vegetation area, then this location is closer to a normal shaded area, and subsequent shading compensation should be performed rather than fire point enhancement; if another location satisfies S(i,j)=0.81, NBR(i,j)=-0.15, and the B12 band value is significantly higher than the surrounding locations on the same shady slope, then this location should be classified as a candidate area for shading combustion, and subsequent high response of shortwave infrared should be retained, and it cannot be simply treated as shading suppression.

[0122] 7) Save the region type results as a region type map Z. The region type map Z represents the enhancement strategy category corresponding to each location, with a size of H×W. Subsequent enhancement of all bands will be performed based on the region type map Z.

[0123] It should be noted that "darkness" in mountain forest fire images is not the same as "shadow," and "shadow" may also contain real combustion anomalies. This invention uses a digital elevation model to interpret the terrain's illumination first, and then uses the combustion index and shortwave infrared anomalies to make secondary corrections to areas with strong shadows. This can reduce the problem of weak fire points being falsely suppressed, so that ordinary shady slopes will not be misidentified as fire points, and real fire points in shadows will not be missed due to low brightness.

[0124] like Figure 3 As shown, this is a map illustrating the characteristics of sunlight exposure (the cosine of the angle between direct sunlight and the terrain normal; a higher value indicates more abundant sunlight exposure).

[0125] like Figure 4 As shown, a normalized shadow intensity feature map is displayed (a continuous value of 0-1 mapped from the degree of illumination, with the closer to 1 indicating a stronger terrain shadow).

[0126] S2.2, Band Differentiation Normalization Enhancement

[0127] Different wavelength bands serve different purposes in forest fire scenarios. The shortwave infrared band primarily carries information about high temperature anomalies, while the visible light band is easily affected by shadows and smoke. The red-edge and near-infrared bands mainly carry vegetation structure information. If all bands are enhanced with the same intensity, it can easily lead to insufficient enhancement of the shortwave infrared, over-enhancement of the visible light, and distortion of vegetation structure. This invention performs differentiated enhancement on different wavelength bands based on region type maps, normalized shadow intensity maps, and vegetation background statistics to obtain enhanced multispectral samples. The specific steps are as follows:

[0128] 1) Screen out effective pixels from the effective area mask K, and then select a healthy vegetation reference area from the effective pixels. The healthy vegetation reference area is used to estimate the normal background reflectance baseline.

[0129] In one implementation, locations with NDVI > 0.75 that are neither fire point labels nor shadow burning candidate areas are used as healthy vegetation reference areas.

[0130] 2) Calculate the mean value of healthy vegetation background in the c-th band and define it as the vegetation background reference reflectance. Vegetation background reference reflectance This represents the typical reflectance of the c-th band on healthy vegetation, used to reduce the common background response of large areas of normal vegetation;

[0131] If the number of healthy vegetation reference pixels in a single slice is insufficient, the statistical window should be expanded first; if it is still insufficient, the statistical mean of adjacent slices in the same batch should be used as the fallback value.

[0132] In one embodiment, as an example, the vegetation background reference reflectance Directly equal to the first The average reflectance of all effective pixels within a healthy vegetation reference area for a given band. For example, if the average reflectance of band B8 within the area is 0.42, then... .

[0133] 3) At each pixel location (i,j), establish a local neighborhood centered on the current location, and calculate the local background wave intensity of the c-th band, defined as... Local background fluctuation intensity This represents the natural fluctuation of background reflectance near position (i,j), which is used for subsequent local normalization.

[0134] In practical implementation, the local neighborhood size can be 15×15; during statistics, only pixels with an effective region mask K=1 that are not fire point labels or shadow burning candidate areas are used to avoid real fire points from increasing the local standard deviation. If there are too few effective local pixels, switch to a 21×21 neighborhood for re-statistics.

[0135] 4) Perform background correction and local normalization on the multispectral sample X band by band to obtain the intermediate normalized result R. The intermediate normalized result R represents the multispectral sample after removing the vegetation background bias and unifying it according to the local fluctuation scale. The size is H×W×C, and the calculation method is expressed as follows:

[0136] ;

[0137] in, This represents the local normalization result of the c-th band at position (i,j). This represents the original reflectance of the c-th band at position (i,j). Represents the numerical stability constant, which can be taken as... .

[0138] 5) Construct a shortwave infrared enhancement coefficient. The shortwave infrared enhancement coefficient is used to improve the response of the B11 and B12 bands to weak fire points, while avoiding excessive amplification of ordinary shadow areas.

[0139] In practical implementation, the short-wave infrared enhancement coefficient can be taken as 1.4 to 2.2 for the shadow burning candidate region;

[0140] For ordinary shadow areas, the shortwave infrared enhancement factor can be taken as 1.1 to 1.4;

[0141] For the direct-light region, the short-wave infrared enhancement coefficient can be taken as 1.0 to 1.2;

[0142] For the transition zone, linear interpolation can be performed using the coefficients of the two adjacent zones.

[0143] In one embodiment, as an example, for a certain pixel in the shadow burning candidate region, if Local Original reflectivity Then the normalized value If the enhancement factor for this region is 1.8, then the enhanced value will be... (Subsequent truncation processing).

[0144] Based on this, it can be ensured that the short-wave infrared response of the "shadow area where the real fire point is located" is activated, while the short-wave infrared response of the "ordinary shadow area" is only mildly compensated.

[0145] 6) Construct visible light suppression and compensation coefficients. Visible light suppression and compensation coefficients are used to avoid saturation artifacts caused by blue light, green light and red light in areas with strong direct sunlight or dense smoke.

[0146] In practical implementation, for ordinary shadow areas, weak compensation is used in the visible light band, and the coefficient can be taken as 1.05 to 1.20;

[0147] For the direct sunlight region, the visible light band is subject to upper limit suppression, with a coefficient ranging from 0.85 to 1.00.

[0148] For the candidate region of shadow burning, the visible light band is not strongly enhanced, but only a mild adjustment of 0.95 to 1.05 is maintained to avoid magnifying the texture around the fire point in the dark background into a large false detection area.

[0149] In one embodiment, as an example, the pre-enhancement value of the B4 band in the ordinary shadow area is 0.05 (too low), the compensation coefficient is 1.15, and the enhanced value is 0.0575, which slightly brightens the image but does not overexpose it.

[0150] 7) Construct a vegetation structure retention coefficient, which is used to adjust the enhancement intensity of the red-edge band and near-infrared band to ensure that the vegetation boundary, burned area boundary, and unburned vegetation structure remain stable after enhancement. In one implementation, a moderate adjustment range of 0.95 to 1.15 is uniformly adopted for both the red-edge band and the near-infrared band, without excessive stretching.

[0151] 8) Construct a slope suppression coefficient based on the terrain slope. Slope suppression coefficient This represents the suppression weight applied to the enhancement amplitude due to steep terrain. It has a size of H×W and can be set from 0.7 to 1.0.

[0152] In one implementation, let the slope angle be θ (in degrees), and construct the slope suppression coefficient using the following formula:

[0153] ;

[0154] That is, once the slope exceeds 10°, the inhibition coefficient decreases by 0.03 for every 1° increase, down to a minimum of 0.7;

[0155] Where, max( ) represents the maximum value operation, min( ) indicates the minimum value operation.

[0156] Therefore, when the slope is relatively small, When the slope is close to 1, and the slope is relatively large, This reduces the risk of overshoot artifacts on ridges, steep slope edges, and valley boundaries after enhancement.

[0157] 9) Apply shortwave infrared enhancement coefficient, visible light suppression and compensation coefficient, and vegetation structure preservation coefficient to the intermediate normalized result R according to the band category, and then multiply by the slope suppression coefficient. The enhanced multispectral sample E is obtained. The enhanced multispectral sample E represents the multispectral sample after terrain shadow suppression, background correction and fire point prior enhancement, with a size of H×W×C.

[0158] In practice, for ease of implementation, the method of "multiplying by coefficients first and then truncating quantiles" can be adopted. That is, first, perform pixel-by-pixel gain adjustment for each band, then truncate the values ​​below the 1st quantile of each band to the 1st quantile, and truncate the values ​​above the 99th quantile to the 99th quantile, and finally map them to a unified value range, such as [0,1].

[0159] It should be noted that gain adjustment refers to multiplying each pixel value of the intermediate normalized result by a scaling factor related to the region type (i.e., shortwave infrared enhancement factor, visible light suppression and compensation factor, or vegetation structure preservation factor), and then multiplying by the slope suppression factor. This allows for targeted enhancement or suppression of signal intensity based on the terrain shadow type and band attributes of the pixel, rather than uniformly stretching the entire image. It should also be noted that the normalized shadow intensity map S, the region type map Z, and the slope suppression coefficient... All images are single-channel images. When performing calculations with multispectral samples X or intermediate normalized results R, they are copied along the band direction to C channels before pixel-by-pixel and band-by-band calculations are performed. It should also be noted that this invention does not simply "brighten" or "standardize" the entire band, but rather performs differentiated processing according to the actual physical meaning of forest fire monitoring. Shortwave infrared prioritizes the enhancement of weak fire points, visible light prioritizes the suppression of shadows and the stabilization of smoke, and red edge and near-infrared prioritize the preservation of vegetation structure. In complex mountainous forest areas, both "fire point visibility" and "background stability" are taken into account.

[0160] In a specific implementation, S3 is as follows:

[0161] In real forest fire samples, small fire spots, thin fire lines, and weak fire spots on shady slopes are usually few in number. While conventional augmentation methods such as rotation, flipping, and random cropping can change the appearance of the sample, they cannot generate fire line morphologies that conform to the constraints of actual wind direction, slope aspect, and combustible material distribution, making it difficult to effectively expand high-value samples. This invention uses enhanced multispectral samples E and fire spot label maps M as a basis, generating spatially constrained fire line expansion areas based on wind direction, slope aspect, and vegetation conditions. Then, thermal radiation enhancement and plume obscuration modulation are applied to the expansion areas to obtain physically simulated augmented samples. The specific steps are as follows:

[0162] S3.1, Spread Direction Field Construction and Fireline Expansion

[0163] Real fire lines do not spread at a constant speed in all directions, but are more likely to spread along downwind, uphill, and in areas with high combustibility. Without directional constraints, the generated augmented fire lines would be too smooth and fail to reflect the true boundary morphology of forest fires. This invention generates a spread direction field based on fire point label maps, digital elevation models, and vegetation conditions, and expands the fire point area accordingly. The specific steps are as follows:

[0164] 1) Using the enhanced multispectral sample E, digital elevation model D, and fire point label map M as input, first perform a small noise removal on the fire point label map M (e.g., remove isolated connected regions with an area of ​​less than 2 pixels) to avoid mislabeling a single mislabeled pixel as an ignition source.

[0165] Furthermore, a connected component analysis is performed on the fire point label map M to extract the ignition source location from each fire point connected component. The ignition source location is used to represent the starting point of the spread.

[0166] In one implementation, when the area of ​​the fire point connected domain is less than 9 pixels, the centroid of the connected domain is directly taken as the ignition source; when the area of ​​the fire point connected domain is greater than or equal to 9 pixels, one ignition source is set every 3 to 5 pixels along the main axis of the connected domain in order to generate a more natural strip fire line extension.

[0167] 2) Calculate the combustible weight map from the enhanced multispectral sample E Combustible material weighting diagram It indicates the abundance of combustibles at different locations, with dimensions of H×W, and a value range from 0 to 1.

[0168] In practical implementation, segmented mappings can be constructed based on NDVI, specifically:

[0169] When NDVI < 0.2, set the combustible weight to 0.2 to 0.4;

[0170] When 0.2≤NDVI≤0.6, the combustible weight is increased linearly from 0.4 to 0.8;

[0171] When NDVI > 0.6, set the combustible weight to 0.8 to 1.0.

[0172] Based on this, the fire line can be made more inclined to extend along areas with higher vegetation cover, rather than easily crossing bare land and low vegetation areas.

[0173] 3) Calculate the slope aspect using the digital elevation model D, and read the wind speed and direction data corresponding to the time of sample imaging. The wind speed and direction data can come from ground meteorological station data or from reanalysis meteorological data.

[0174] In an easy-to-implement approach, hourly wind field data can be used and interpolated to the center of the sample, where wind direction is used to determine the downwind expansion direction and slope aspect is used to determine the uphill expansion direction.

[0175] 4) Calculate the spread priority in multiple discrete directions with each ignition source as the center. Specifically, divide 0° to 360° into 36 directions, with each direction being 10°. For each direction, first set the basic spread length, and then adjust it according to the consistency of wind direction, uphill slope and combustible weight.

[0176] In one embodiment, for example, when the Normalized Difference Vegetation Index (NDVI) at a certain location is 0.3, the combustible weight at that location is calculated from a preset segmented mapping rule. The rule stipulates that when 0.2 ≤ NDVI ≤ 0.6, The value increases linearly from 0.4 to 0.8, and the calculation uses linear interpolation, expressed as follows:

[0177] ;

[0178] Substituting NDVI=0.3, we get ,in, It is a weight value between 0 and 1. The higher the value, the more abundant the combustible material at that location, and the stronger the tendency of the fire line to spread in that direction.

[0179] In one implementation, the expansion coefficient is defined as a pre-set empirical constant, representing the multiple of the direction relative to the base expansion length. The expansion coefficient for the downwind direction can be 1.4 to 1.8, for the upwind direction it can be 0.6 to 0.9, for the uphill direction it can be 1.1 to 1.4, and for the downhill direction it can be 0.8 to 1.0. If the combustible weight at the current location is close to 1, the expansion length is further increased; if the combustible weight is low, the expansion length is shortened. For example, if the base expansion length is set to 4 pixels and the downwind expansion coefficient is 1.6, then the final expansion length in that direction = 4 × 1.6 = 6.4 (rounded to 6 pixels).

[0180] 5) Perform grid expansion according to the spread priority in each direction to obtain the expanded fire point label map. Expand fire point label map This represents the extended fire line area after being constrained by wind direction, slope, and combustibles. Its dimensions are H×W, and its value is 0 or 1.

[0181] In the specific implementation, a region growth method based on frontier advancement is adopted. Specifically, starting from the ignition source, the high-priority neighborhood direction is added to the expansion queue first. After each expansion of 1 pixel, the expansion priority of the surrounding 8 neighbors is re-evaluated. When the cumulative expansion length reaches the preset upper limit, it stops. The preset upper limit is used to control the intensity of a single augmentation, for example, it can be 2 to 6 pixels.

[0182] 6) Constrain the expansion results by combining low-combustible areas and significant boundaries. If a location to be expanded falls in a low-combustible area or is located on a significant land feature boundary characterized by the gradients of the red edge band and near-infrared band, its priority for inclusion in the expansion area is reduced. If the priority is below the threshold, expansion to that location will no longer be allowed. Based on this, it is possible to avoid the fire line from unreasonably crossing water bodies, roads, or bare rock boundaries.

[0183] In one embodiment, for example, the ignition source is located at (128, 128), with the downwind direction being 30° east of north, which has a high priority; starting from the source point, it expands by 1 pixel in the neighborhood in the 30° direction each time, and stops after a total of 5 expansions when the length reaches the preset upper limit.

[0184] 7) Expand the fire point label map Perform edge smoothing once to obtain the soft boundary expansion map. Soft boundary extension map This represents the smoothed probability distribution after the fireline expands, with a size of H×W and a value range of 0 to 1.

[0185] In the specific implementation, we can first... Perform Gaussian smoothing, then keep the value inside the fire point at a relatively high level, and convert the boundary transition region into a continuously decreasing value. This makes it easier to apply spectral modulation to the "core region, edge region, and influence region" separately.

[0186] It should be noted that "higher value" refers to the soft boundary extension map. The value in the extended fire point core region is typically maintained above 0.8. For example, before smoothing, the extended fire point label map... The internal value is 1, and the boundary value is 0; it is smoothed using Gaussian smoothing (e.g., using a standard deviation of 1.0). After the Gaussian kernel, the original internal region The value may drop to around 0.85, while the value near the original boundary gradually transitions from 0 to 0.3-0.6; if the original extended fire line width is 3 pixels, the smoothed value of the central column pixels... The value is 0.90, the adjacent columns are 0.65 and 0.40 respectively, and the outermost non-fire point area is 0.05, thus forming a continuous soft boundary distribution for subsequent partitioned and graded modulation.

[0187] It should also be noted that the useful training information for fire lines is not "the area increases", but "how the boundary expands along the terrain and wind field". By introducing directional and combustible constraints during the augmentation stage, the model's ability to learn about slender fire lines, asymmetric fire lines and locally expanding fire points in mountainous areas can be improved.

[0188] like Figure 5 As shown, the Normalized Difference Vegetation Index (NDVI) is a vegetation activity index calculated using near-infrared and red light, with green representing high-coverage vegetation.

[0189] S3.2, Enhanced Thermal Radiation and Modulated Smoke Plume Obscuring

[0190] Spatial expansion alone is insufficient because the responses of real firelines differ across shortwave infrared, near-infrared, and visible light. If only the label is expanded without simultaneous spectral adjustment, the generated sample will resemble a fireline in shape but not in spectrum. This invention constructs a fire core region, a fireline edge region, and a downwind plume region based on the expansion results, and modulates each band separately to obtain a physically simulated augmented sample. The specific steps are as follows:

[0191] 1) Based on the soft boundary extension diagram Construct the fire core zone, fire edge zone, and heat-affected zone, specifically...

[0192] The region with a value greater than or equal to 0.8 is defined as the fire core region. The area with values ​​between 0.4 and 0.8 is defined as the fire line edge zone. The region with values ​​between 0.1 and 0.4 is defined as the heat-affected zone;

[0193] Based on this, the extended area can be divided into a "high temperature center", a "combustion edge", and a "nearby transition zone".

[0194] 2) Construct a smoke plume region on the downwind side of the fire line according to the prevailing wind direction. The smoke plume region represents the background area that may be affected by smoke obscuration, with a size of H×W and a value range of 0 to 1.

[0195] In practice, a strip or fan-shaped area can be generated from the edge of the fire line along the prevailing wind direction, and the length of the plume area can be increased with the increase of wind speed. For example, the plume extension length can be 10 to 30 pixels, and the plume width can be 3 to 12 pixels. The closer to the fire line, the stronger the plume obscuration; the farther away from the fire line, the weaker the plume obscuration.

[0196] 3) Perform thermal radiation enhancement on the shortwave infrared band. Specifically, for the B12 band, the enhancement factor in the core area of ​​the fire point can be 1.8 to 2.8, the enhancement factor in the edge area of ​​the fire line can be 1.3 to 1.8, and the enhancement factor in the heat-affected zone can be 1.05 to 1.20. For the B11 band, the enhancement factor is slightly lower than that in the B12 band. Based on this, the significant increase in the response of the real fire point in the shortwave infrared can be simulated.

[0197] In one embodiment, as an example, the original pixel value of a core area in the B12 band is 0.25, the enhancement factor is 2.5, and the modulated value is 0.625 to simulate a high-temperature target.

[0198] 4) Perform weak enhancement or preservation processing on the red edge band and near-infrared band. Specifically, the enhancement factor of the red edge band and near-infrared band in the core area of ​​the fire point can be 1.02 to 1.10, and the enhancement factor of 1.00 to 1.06 in the edge area of ​​the fire line. Most background areas remain unchanged. Based on this, the vegetation boundary and fire-affected area transition information can be preserved, but these bands are not allowed to undertake the main high temperature expression task.

[0199] In one embodiment, for example, the original value of the core area pixel in the B8 band is 0.40, the enhancement factor is 1.05, and the value after modulation is 0.42, a slight change to preserve vegetation information.

[0200] 5) Perform plume transmittance attenuation on the visible light band. Specifically, within the plume region, the transmittance of the B2, B3, and B4 bands can be taken as 0.50 to 0.80; the denser the plume, the lower the transmittance; the farther away from the fire line, the transmittance gradually recovers to 1; for the shortwave infrared band, the plume transmittance attenuation is very weak, usually remaining between 0.90 and 1.00; based on this, the real phenomenon of "smoke significantly blocking visible light and having little impact on shortwave infrared" can be simulated.

[0201] In one embodiment, for example, the original value of the B3 band in the plume region is 0.18, the transmittance is 0.6, and after modulation it is 0.108, simulating the effect of smoke obscuring and darkening.

[0202] 6) The combined effect of thermal radiation enhancement and plume transmittance attenuation on the enhanced multispectral sample E is used to obtain the physical simulation augmented sample A. The physical simulation augmented sample A represents an augmented training sample with both realistic fireline morphology and reasonable spectral changes, and its size is H×W×C.

[0203] In practice, shortwave infrared enhancement is applied preferentially to the core and edge areas; visible light attenuation is applied preferentially to the plume area; when a location is simultaneously located in the heat-affected zone and the plume area, it is processed by superimposing the bands separately; after processing, a range truncation is performed on each band to prevent the enhancement value of a very small number of pixels from exceeding the dynamic range of the entire image.

[0204] In one embodiment, for example, a pixel is simultaneously subjected to thermal enhancement (multiplied by 1.8) and plume attenuation (multiplied by 0.7), with a net coefficient of 1.26 and an enhanced value of 3.2. If the value exceeds the upper limit of the dynamic range of the entire image of 3.0, it is truncated to 3.0.

[0205] 7) Define the augmented label corresponding to the augmented sample A in the physics simulation as the augmented fire point label map. Expand fire point label image A pixel-level binary annotation map representing the fire point region in the augmented sample, with dimensions H×W.

[0206] In one implementation, the extended fire point label map can be directly applied. After binarization, it is used as an augmented fire point label image. .

[0207] It should be noted that this invention does not simply "copy and expand" the fire point area, but rather expands it spatially according to the spread law and modulates it spectrally according to the combustion and smoke law. Based on this, the generated samples not only have a shape that is more like a real fire line, but also have a band response that is more like a real fire scene, which is conducive to improving the model's learning quality for complex fire line edges and weak fire points.

[0208] It should also be noted that the above-mentioned enhanced samples can be directly used to train the forest fire identification network. The following uses a dual-branch network as an example to demonstrate one application method.

[0209] In a specific implementation, S4 is as follows:

[0210] Step S4 is an exemplary application of the enhanced multispectral sample E and is an optional step used to illustrate how to combine the high-quality enhanced sample generated by the present invention with a specific fire identification network architecture to verify the improvement effect of the enhancement method on the actual fire detection performance. In addition, this network is not the only application of the present invention, and the enhanced sample can also be adapted to other semantic segmentation models.

[0211] In forest fire images, fire points are usually manifested as small-scale high-temperature anomalies in the shortwave infrared band, while smoke is usually manifested as large-scale diffuse obscuration in the visible light band. If all bands are learned using the same receptive field in the same branch, two types of problems are likely to occur: first, small fire points are submerged by large-scale textures; second, the smoke plume boundary is unstable due to the small receptive field.

[0212] This invention constructs a bi-branch dynamic receptive field network for decoupling fire point and smoke. The network consists of three parts:

[0213] a) Band splitting module: Based on band statistics and a learnable routing network, it generates fire point branch weights for each band. Weights of smoke branches ;

[0214] b) Firepoint Branch: Receives the weighted firepoint input tensor Small-scale high-temperature features are extracted using two 3×3 convolutional blocks and multi-scale dilated convolution, and a fire point feature map is output. Fire response diagram ;

[0215] c) Smoke Branch: Receives the weighted smoke input tensor First, preliminary features are extracted using two 5×5 convolutional blocks, followed by three parallel context branches: 3×3 convolution, 5×5 convolution, and 3×3 convolution with a dilation rate of 2; the feature maps output by the three branches are derived from the fire response map. Guided dynamic weighted fusion yields smoke feature maps. ;

[0216] The specific steps are as follows:

[0217] S4.1 Band splitting and dual-branch feature extraction

[0218] This invention first uses a lightweight routing module to assign "fire point branch weights" and "smoke branch weights" to each band, then performs feature extraction for the fire point branch and smoke branch respectively, and uses the fire point branch results to guide the dynamic selection of the receptive field for the smoke branch. The specific steps are as follows:

[0219] 1) The network input is the training input sample T, which represents the multispectral sample fed into the network with a size of H×W×C; when using real samples, the training input sample T=E; when using augmented samples, the training input sample T=A.

[0220] The training input sample T is input into the band splitting module. Specifically, for the c-th band, four global statistics are extracted as the basis for routing judgment for that band, including global mean, global standard deviation, 95th percentile value and center wavelength code.

[0221] The center wavelength code is used to represent the physical location of the current band. For example, the blue light band corresponds to a shorter wavelength code, and the short-wave infrared band corresponds to a longer wavelength code.

[0222] Based on this, the routing module can see not only "how bright and discrete this band is in the current sample", but also "whether this band originally belongs to visible light, near infrared or short-wave infrared".

[0223] 2) Input the 4D statistical vector of the c-th band into a two-layer fully connected routing network, and output the fire point branch weights. and smoke branch weight ,in,

[0224] This represents the weight assigned to the fire point branch by the c-th band. This represents the weight assigned to the smoke branch by the c-th band. Both weights are between 0 and 1, and their sum is 1.

[0225] In one implementation, the output dimension of the first fully connected layer can be 16, the output dimension of the second fully connected layer is 2, and finally the two weights are obtained through Softmax.

[0226] 3) Generate the fire point input tensor based on the fire point branch weights for all bands. Generate the smoke input tensor based on the smoke branch weights of all bands. Fire point input tensor The input features, which are more biased towards high-temperature anomalies, have dimensions of H×W×C; the smoke input tensor... This indicates an input feature that is more inclined towards smoke plume masking, and its size is also H×W×C.

[0227] In practice, each band of the training input sample T is multiplied by its corresponding weight and then stacked in the original band order. Due to the use of soft splitting, a certain band can participate in two branches at the same time, but the participation ratios are different.

[0228] In one embodiment, as an example, suppose the routing weight of the B12 band is... , Then 90% of the information in that band is sent to the fire point branch (multiplied by 0.9 and stacked to...). ), 10% of the information is sent to the smoke branch (multiplied by 0.1 and then stacked to The weighting of the B3 band may be reversed. , The information mainly flows to the smoke branch.

[0229] 4) Input the fire point into the tensor Input the fire point branch to obtain the fire point feature map. Fire response diagram Fire point branching is used to extract small-scale high-temperature anomaly features.

[0230] In one implementation, the fire point branch comprises two basic convolutional blocks and one multi-scale dilated convolutional module. Each basic convolutional block consists of a 3×3 convolution, batch normalization, and a linear rectified activation function, with the number of output channels being 32 and 64, respectively. The multi-scale dilated convolutional module consists of three parallel convolutional branches with dilation rates of 1, 2, and 3, respectively. The outputs of the three branches are concatenated along the channel dimension and then fused by a 1×1 convolution to obtain the fire point feature map. Fire response diagram Fire point feature map Represents the deep features of the fire point branch, with a size of Fire response diagram This indicates that the current location represents an intermediate response to a high-temperature anomaly, with dimensions H×W, where... This indicates the number of feature map channels output by the fire point branch, with a value of 32.

[0231] 5) Input the smoke into the tensor Input the smoke branch, which is used to extract large-scale diffuse smoke plume and occlusion boundary features.

[0232] In one implementation, the smoke branch first passes through two basic convolutional blocks. Each basic convolutional block consists of a 5×5 convolution, batch normalization, and a linear rectified activation function. The number of output channels can be 32 and 64 respectively. Using a larger basic convolutional kernel is beneficial for forming a relatively smooth context representation first.

[0233] Then, three parallel context branches are set up in the smoke branch, each corresponding to a different receptive field. Further, the first context branch uses a 3×3 convolution to focus on the finer edges of the smoke plume near the fire line; the second context branch uses a 5×5 convolution to focus on medium-scale smoke plume diffusion; and the third context branch uses a 3×3 convolution with a dilation rate of 2 to focus on a larger range of smoke plume structure. Each of the three context branches outputs a smoke feature map.

[0234] 6) Based on the fire response diagram Generate dynamic receptive field selection weights; specifically, generate the fire response map. The input is a shallow bootstrapping network, which consists of two 3×3 convolutional layers. The final output is a spatial weight map with three channels, and softmax normalization is performed on the channel dimensions to obtain the weight map. ,in, These represent the selection weights of the three receptive fields for the current position, each with a size of H×W;

[0235] When a location is close to the fire point boundary and the fire point response changes rapidly, the smaller receptive field has a higher weight; when a location is in a diffuse smoke plume region far from the fire point, the larger receptive field has a higher weight. Based on this, the smoke branch can no longer use a single convolutional field of view, but can dynamically select a suitable context range according to the fire point location.

[0236] 7) Weight the outputs of the three smoke context branches according to the graph. Perform pixel-by-pixel weighted fusion to obtain the smoke feature map. Smoke feature map This represents the deep features of smoke branching after dynamic receptive field adjustment, with a size of [size missing]. ,in, This represents the number of feature map channels output by the smoke branch, with a value of 32.

[0237] 8) Fire point feature map Smoke feature map Fire response diagram By merging the data, a pixel-level fire risk probability map is obtained. Pixel-level fire risk probability map This represents the predicted probability that each pixel belongs to a fire point region, with a size of H×W and a value range of 0 to 1.

[0238] In the specific implementation, first, the fire point feature map is... With smoke feature map After splicing the channels, the fire response diagram is then... After being expanded into one additional channel, the data is concatenated together, then compressed through a 1×1 convolution, and finally output as a pixel-level fire risk probability map using a sigmoid activation function. .

[0239] In one embodiment, as an example, when a location is situated in an area with a significant shortwave infrared anomaly and is adjacent to the edge of a smoke plume, the fire point response map... Taking a high value at this position will increase the proportion of small receptive field branches in the dynamic receptive field selection, making the network pay more attention to the fine changes near the fire line; when a position is far from the fire line and only appears as a large-scale gray-white diffuse area, the dynamic receptive field selection weight will increase the proportion of large receptive field branches, making the network pay more attention to the continuous structure of the plume; based on this, it avoids small fire points being submerged by a large background, and also avoids the plume boundary being incompletely learned due to the local field of view being too small.

[0240] It should be noted that, in order to reduce the competition for feature representation between fire points and smoke in the same convolution channel, this invention first splits the "high temperature anomaly representation" and "plume masking representation" into two branches by band splitting, and then guides the smoke branch to dynamically select the receptive field by the fire point branch result, thereby simultaneously improving the detection capability of weak fire points and the stability under complex smoke plume background.

[0241] S4.2, Joint Loss Training

[0242] Fire pixels are usually very few in the whole image. If only ordinary cross-entropy loss is used, the model is prone to bias towards large areas of background. If only the classification result is considered and the boundary is ignored, the model is prone to blurring and sticking at the edge of the fire line.

[0243] This invention employs a combination of focus loss and boundary consistency loss to jointly optimize network parameters. The specific steps are as follows:

[0244] 1) Define the training label map corresponding to the training input sample T as G. The training label map G represents a pixel-level binary label map with a size of H×W;

[0245] When the training input sample T=E, the training label map G=M; when the training input sample T=A, the training label map... .

[0246] It should be noted that this invention only calculates the loss for the effective area mask K=1, and the locations of clouds, water bodies and invalid boundaries are not included in the loss calculation.

[0247] 2) Based on the training label map G and the pixel-level fire risk probability map Calculate the focus loss The focus loss is used to improve the training weights of a few fire pixels and hard-to-classify boundary pixels.

[0248] In one implementation, the positive sample weight can be 0.25 and the focusing parameter can be 2.0; if the proportion of fire pixels in the current training batch is extremely low, the positive sample weight can also be appropriately increased.

[0249] It should be noted that focus loss is a loss function used to handle the problem of extreme class imbalance. In forest fire detection, fire point pixels account for a very small proportion of the entire image. If standard cross-entropy loss is used, the model will tend to predict all pixels as non-fire points to obtain a lower loss. Focus loss introduces a modulation factor to automatically reduce the contribution weight of correctly classified simple samples (such as large background areas) to the total loss, allowing the model to focus more on a few difficult-to-classify samples (such as fire point boundaries and weak fire points), thereby improving the recall rate of sparse fire point pixels.

[0250] 3) Construct the true boundary target based on the training label map G. Specifically, first perform Euclidean distance transformation on the training label map G to obtain the distance map, and then use the Sobel operator to extract the boundary gradient map from the distance map as the true boundary target.

[0251] Correspondingly, for pixel-level fire risk probability maps Similarly, the Sobel operator is used to extract the predicted boundary gradient map. The more consistent the actual boundary target and the predicted boundary gradient map are, the more accurate the fire line boundary prediction is.

[0252] In the specific implementation, a distance transformation is performed on the training label map G (binary 0 / 1), and the value of each non-fire pixel is the Euclidean distance (in pixels) to the boundary of the nearest fire point. Then, the Sobel operator is used to calculate the gradient magnitude of the distance map to obtain the true boundary target map (size H×W, with higher values ​​at the fire line). Then, the predicted probability map is... Similarly, calculate the Sobel gradient magnitude to obtain the predicted boundary gradient map, and finally calculate the mean square error of the two gradient maps. .

[0253] 4) Calculate boundary consistency loss In one implementation, the boundary consistency loss uses the mean square error between the predicted boundary gradient map and the true boundary target. The boundary consistency loss is used to constrain the position and direction of change of the fire line edge and reduce boundary blurring.

[0254] 5) Weight the focus loss and boundary consistency loss to form the total loss. , is represented as:

[0255] ;

[0256] Where μ represents the boundary consistency loss weight coefficient, and μ is preferably set to 0.1.

[0257] 6) Organize training batches by mixing real samples and augmented samples.

[0258] In one implementation, the number of real samples and augmented samples in each training batch is mixed in a 1:1 ratio. This preserves the real fire scene distribution while using physical simulation augmented samples to supplement complex boundary and weak fire point samples.

[0259] 7) Use an adaptive optimizer to update network parameters.

[0260] In one implementation, the AdamW optimizer can be used, with an initial learning rate of [value missing]. The batch size can be 8, and the total number of training rounds can be 100 to 150. During training, pixel-level recall, crossover ratio, and F1 score are monitored on the validation set, and the model parameters with the best overall performance are saved.

[0261] 8) After training is completed, the fire identification model is output. The fire identification model represents a dual-branch dynamic receptive field network that has completed parameter learning and can be used to identify fire points at the pixel level for newly acquired multispectral samples.

[0262] It should be noted that this invention not only optimizes "how many pixels are correctly classified", but also optimizes "how accurately the fire line boundary is drawn". Based on this, in complex mountain forest fire scenarios, the model can improve the recall rate of weak fire points and maintain the clarity and directional consistency of the fire line boundary.

[0263] In a specific implementation, S5 is as follows:

[0264] Acquire new temporal remote sensing images of the area to be monitored, perform atmospheric correction, band resampling and slicing according to method S1, read the corresponding digital elevation model and metadata, perform terrain shadow decoupling and differential enhancement as described in S2, and output enhanced multispectral sample E. This enhanced image can be directly used by the fire monitoring model or used as a visualization base map to assist manual interpretation.

[0265] Although the specific embodiments of the invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the invention. Based on the technical solutions of the invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the invention.

Claims

1. A remote sensing image enhancement method for forest fire monitoring, characterized in that, Includes the following steps: S1. Perform unified projection, unified resolution, and sample slicing on multispectral images, digital elevation models, and fire point labels in forest fire monitoring to obtain multi-source training samples for forest fires. Each training sample includes multispectral samples, digital elevation models, effective area masks, and fire point label maps. S2. Calculate the terrain shadow intensity based on the digital elevation model, and then combine the normalized combustion ratio and the local anomaly response of the shortwave infrared band to perform differential enhancement on the shortwave infrared band, visible light band and vegetation structure band to obtain enhanced multispectral samples. In step S2, the light intensity is estimated using a digital elevation model, and then combined with the normalized vegetation index and normalized burn ratio to generate a topographic shadow intensity map and a shadow area type map, as detailed below: The solar azimuth and solar altitude angles are read from the image metadata corresponding to the multispectral samples to determine whether each pixel is in the direct sunlight area or the shadow area; the Sobel operator is used to calculate the lateral elevation change and the longitudinal elevation change in the digital elevation model in a 3×3 neighborhood, and then the slope amplitude and aspect angle are synthesized from the lateral gradient and the longitudinal gradient; the light intensity of each pixel is calculated based on the slope, aspect, solar azimuth and solar altitude angle, and then the light intensity is mapped to a normalized shadow intensity map; Normalized Difference Vegetation Index (NDVI) and Normalized Burning Ratio (NBR) were calculated based on the near-infrared, red, and short-wave infrared reflectance in the multispectral samples. Based on the normalized shadow intensity map S, the normalized burn ratio NBR, and the local anomaly response of the shortwave infrared, each pixel is classified into region types, including ordinary shadow area, direct area, transition area, and shadow burn candidate area. The region type results are saved as a region type map. S3. Based on the enhanced multispectral samples and fire point label map, a spatially confined fire line expansion area is generated according to wind direction, slope aspect and vegetation conditions. Thermal radiation enhancement and smoke plume shading modulation are performed on the expansion area to obtain a physically simulated augmented sample. S4. Construct a dual-branch dynamic receptive field network for decoupling fire point and smoke, use training samples as input, verify the improvement effect of the enhanced samples on the actual fire point detection performance, and output a pixel-level fire risk probability map; use focus loss and boundary consistency loss to jointly optimize the network parameters and train and optimize the dual-branch dynamic receptive field network. S5. Acquire new temporal remote sensing images of the area to be monitored. After image enhancement, use them for the trained bi-branch dynamic receptive field network or as a visualization base map to assist manual interpretation.

2. The remote sensing image enhancement method for forest fire monitoring according to claim 1, characterized in that, S1 is as follows; Atmospheric correction is performed on multispectral images, the required bands are extracted and cropped according to spatial range, and the atmospherically corrected surface reflectance image is defined as a multispectral sample. The multispectral sample represents the multiband input data used for forest fire identification. The digital elevation model is reprojected to the same projection coordinate system as the multispectral sample, and then bilinear interpolation or cubic convolution interpolation is used to resample the digital elevation model to a uniform spatial resolution. Perform uniform resampling on all bands of the multispectral sample, and then extract cloud, cloud shadow, water body and invalid boundary information from the quality control information corresponding to the multispectral sample to generate an effective region mask; Fire point labels, either manually marked or corrected from existing fire records, are defined as fire point label maps. A fixed-size sliding window is used to crop training slices from the whole scene image, and each training slice is a training sample.

3. The remote sensing image enhancement method for forest fire monitoring according to claim 1, characterized in that, The calculation process for enhancing multispectral samples is as follows: Effective pixels are screened out from the effective area mask, and then healthy vegetation reference areas are selected from the effective pixels. The mean value of healthy vegetation background in each band is calculated and defined as the vegetation background reference reflectance. At each effective pixel location, a local neighborhood is established with the current location as the center, and the local background fluctuation intensity in each band is calculated. Background correction and local normalization are performed on the multispectral samples band by band to obtain intermediate normalization results. The intermediate normalization results represent the multispectral samples after removing vegetation background bias and unifying them according to the local fluctuation scale. For each region type in the regional type map, shortwave infrared enhancement coefficient and visible light suppression and compensation coefficient are constructed. Vegetation structure retention coefficient is constructed to adjust the enhancement intensity of the red edge band and near-infrared band. Slope suppression coefficient is constructed according to the terrain slope. The intermediate normalized results are applied to the shortwave infrared enhancement coefficient, visible light suppression and compensation coefficient, and vegetation structure retention coefficient according to the band category, and then multiplied by the slope suppression coefficient to obtain the enhanced multispectral sample.

4. The remote sensing image enhancement method for forest fire monitoring according to claim 1, characterized in that, Based on the fire point label map, digital elevation model, and vegetation conditions, a spread direction field is generated, and the fire point area is expanded accordingly. The specific operation is as follows: Using enhanced multispectral samples, digital elevation models, and fire point label maps as input, we first perform a small noise removal process on the fire point label maps, and then perform connected component analysis on the fire point label maps to extract the ignition source location from each fire point connected component. The ignition source location represents the spread initiation point. Then, the combustible weight map is calculated from the enhanced multispectral samples, the slope aspect is calculated from the digital elevation model, and the wind speed and wind direction data corresponding to the sample imaging time are read. With each ignition source as the center, discrete directions are divided, and the spread priority is calculated in each discrete direction. For each direction, the basic spread length is first set, and then adjusted according to the degree of wind direction consistency, uphill degree and combustible weight. Grid expansion is performed based on the spread priority of each direction to obtain an expanded fire point label map. The expanded fire point label map represents the expanded fire line area after being constrained by wind direction, slope aspect, and combustible material. The expansion results are then constrained by low combustible material areas and significant boundaries. Finally, edge smoothing is performed on the expanded fire point label map to obtain a soft boundary expansion map.

5. A remote sensing image enhancement method for forest fire monitoring according to claim 4, characterized in that, it extends... The specific constraints on the results are as follows: If a location to be expanded falls in a low-combustible area, or is located on a significant land feature boundary characterized by both red-edge and near-infrared band gradients, its priority for inclusion in the expansion area is reduced. If the priority is below a threshold, expansion to that location will no longer proceed.

6. A remote sensing image enhancement method for forest fire monitoring according to claim 4, characterized in that, The calculation process for augmented samples in physical simulation is as follows: Based on the soft boundary expansion map, the fire core area, fire line edge area, and heat-affected zone are constructed. A plume area is constructed on the downwind side of the fire line according to the prevailing wind direction. The plume area represents the background area affected by smoke obscuring. Then, thermal radiation enhancement is performed on the shortwave infrared band, weak enhancement or preservation processing is performed on the red edge band and near-infrared band, and plume transmittance attenuation is performed on the visible light band. Then, thermal radiation enhancement and plume transmittance attenuation are applied together to enhance the multispectral sample to obtain the physically simulated augmented sample. The augmented label corresponding to the physically simulated augmented sample is defined as the augmented fire point label map.

7. The remote sensing image enhancement method for forest fire monitoring according to claim 1, characterized in that, The dual-branch dynamic receptive field network includes: a band splitting module, a fire point branch, and a smoke branch; The band splitting module generates fire point branch weights and smoke branch weights for each band based on band statistics and a learnable routing network. The fire point branch receives the weighted fire point input tensor, extracts small-scale high-temperature features through two 3×3 convolutional blocks and multi-scale dilated convolution, and outputs a fire point feature map and a fire point response map. The smoke branch receives the weighted smoke input tensor, first extracts preliminary features through two 5×5 convolutional blocks, and then enters three parallel context branches: 3×3 convolution, 5×5 convolution, and 3×3 convolution with a dilation rate of 2. The feature maps output by the three branches are dynamically weighted and fused by the fire response map to obtain the smoke feature map.

8. A remote sensing image enhancement method for forest fire monitoring according to claim 7, characterized in that, The specific operations for band splitting and bi-branch feature extraction are as follows: Training samples include real samples and augmented samples. When using real samples, the corresponding input is an enhanced multispectral sample. When using augmented samples, the corresponding input is a physical simulation augmented sample. The training samples are input into the band splitting module. Four global statistics are extracted for each band as the basis for routing decisions, including global mean, global standard deviation, 95th percentile, and center wavelength encoding. The four-dimensional statistical vector of each band is then input into a two-layer fully connected routing network to output the fire branch weight and smoke branch weight. Each band of the training input sample is multiplied by its corresponding weight and then stacked in the original band order. A fire point input tensor is generated based on the fire point branch weights of all bands, and a smoke input tensor is generated based on the smoke branch weights of all bands. The fire point input tensor is then input into the fire point branch to obtain the fire point feature map and the fire point response map. Input the smoke tensor into the smoke branch, generate dynamic receptive field selection weights based on the fire response map, and perform pixel-by-pixel weighted fusion on the outputs of the three smoke context branches according to the receptive field selection weights to obtain the smoke feature map. By fusing the fire point feature map, smoke feature map, and fire point response map, a pixel-level fire risk probability map is obtained.

9. A remote sensing image enhancement method for forest fire monitoring according to claim 8, characterized in that, The calculation process of the network's loss function is as follows: The training label for the enhanced multispectral sample is the fire point label map, and the training label for the physical simulation augmented sample is the augmented fire point label map. The focal loss is calculated based on the training label map and the pixel-level fire risk probability map. The system constructs true boundary targets based on the training label map, performs Euclidean distance transformation on the training label map to obtain a distance map, and then uses the Sobel operator to extract the boundary gradient map as the true boundary target. The Sobel operator is also used to extract the predicted boundary gradient map on the pixel-level fire risk probability map. The mean square error of the boundary gradient map and the predicted boundary gradient map is calculated to obtain the boundary consistency loss. The total loss is obtained by weighted summation of the focus loss and the boundary consistency loss.

Images