A forest fire image reconstruction system based on satellite remote sensing

Abstract

This invention discloses a forest fire image reconstruction system based on satellite remote sensing, comprising: an image acquisition module, an image feature extraction module, a multi-scale fusion module, and an image reconstruction module. The image acquisition module acquires multi-temporal satellite remote sensing images of a target area before, during, and after a fire, forming a time-series image sequence. The image feature extraction module extracts shallow features from the images during the fire period within the time-series image sequence. The multi-scale fusion module processes and fuses the shallow features of the images to obtain multi-scale fused features. The image reconstruction module extracts prior information from the multi-scale fused features and fuses the prior information to obtain a reconstructed fire image. This invention improves image clarity through super-resolution reconstruction of satellite remote sensing images, thereby enabling accurate identification and monitoring of forest fires and providing data support for fire early warning.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a forest fire image reconstruction system based on satellite remote sensing. Background Technology

[0002] Utilizing satellite remote sensing technology for monitoring and assessing forest fires has become the mainstream approach. However, existing satellite remote sensing-based forest fire image processing systems still suffer from the following technical shortcomings in practical applications: First, the resolution of current satellite images is insufficient, making it difficult to accurately capture the key physical characteristics of fires. Second, existing image reconstruction methods have significant limitations when processing remote sensing images of forest fires, which possess complex textures and dynamic changes. Third, existing satellite remote sensing-based forest fire monitoring systems rarely detect and warn of potential reignition risks, highlighting the urgent need to improve forest fire monitoring and response capabilities. Summary of the Invention

[0003] To address at least one of the aforementioned technical problems, the present invention aims to provide a forest fire image reconstruction system based on satellite remote sensing, which can accurately identify and monitor forest fires, providing data support for fire early warning.

[0004] This invention provides a forest fire image reconstruction system based on satellite remote sensing, comprising: an image acquisition module, an image feature extraction module, a multi-scale fusion module, and an image reconstruction module; The image acquisition module is used to acquire multi-temporal satellite remote sensing images of the target area before, during and after the fire, forming a time-series image sequence. The image feature extraction module is used to extract shallow features of images during the fire occurrence period in a time-series image sequence; The multi-scale fusion module is used to process and fuse the shallow features of the image to obtain multi-scale fused features. The image reconstruction module is used to extract prior information from multi-scale fusion features and fuse the prior information to obtain a reconstructed fire image.

[0005] In this scheme, the step of extracting shallow features from images during the fire occurrence period in the time-series image sequence specifically includes: The images during the fire are decomposed into a low-frequency approximate subband LL and a high-frequency detail subband (LH, HL, HH), with the following expressions: , , , ;in Represented as a low-pass filter, , , These are represented as horizontal, vertical, and diagonal high-pass filters, respectively. The size of the decomposed subband is , , , ; Shallow feature extraction is performed on each sub-band, and the expressions are as follows: , ; , ;in , , , These are the shallow features of the four subbands: LL, LH, HL, and HH. for Convolution operation, LeakyReLU is the linearly corrected unit activation function.

[0006] In this scheme, the multi-scale fusion module includes: a hierarchical dense unit, a pixel-by-pixel addition unit, an input compression and excitation unit, a linear correction unit, and an enhanced high-frequency unit; The hierarchical dense unit is used to iteratively process the shallow features of each sub-band to obtain features. ; The pixel-by-pixel addition unit is used to process features through different convolutional layers. The features are processed and added pixel by pixel to obtain the features. ; The input compression and excitation unit is used for features Processing is performed to obtain features. ; The linear correction unit is used for features Linear correction is performed to obtain multi-scale fused features. Its expression is: , where P is Convolution adjusts the number of channels; The enhanced high-frequency unit is used for multi-scale fusion features. The process is performed to obtain multi-scale fusion features that enhance high-frequency information. .

[0007] In this scheme, the shallow features of each sub-band are iteratively processed to obtain features. The steps specifically include: The specific formula for inputting the shallow features of each sub-band in the image into the hierarchical dense unit for iterative processing is as follows: Where: g = 1, 2, ... G; G is the total number of layers of the hierarchical dense unit. When g = 1, the shallow features of each sub-band are output. After each iteration, the features output from the previous iteration are... and the features of the current iteration output Perform difference calculation to obtain the first feature difference; If the first feature difference is less than or equal to the preset feature threshold, then the feature output of the current iteration will be... Set as feature ; If the difference in the first feature is greater than the preset feature threshold, the iteration continues until all dense units in all layers have completed iteration, and the output features are determined. and the features Set as feature .

[0008] In this scheme, the features are processed through different convolutional layers. The features are processed and added pixel by pixel to obtain the features. The steps specifically include: Features Send to Convolutional layers yield features Its expression is ; Features Send to Convolutional layers yield features Its expression is ; Features Send to Convolutional layers yield features Its expression is ; Features , and Perform pixel-by-pixel summation to obtain features .

[0009] In this scheme, the multi-scale fusion features The process is performed to obtain multi-scale fusion features that enhance high-frequency information. The expression is: ,in for convolution, ;in , , where BN represents batch normalization.

[0010] This solution also includes: a fire warning module, which is used to extract pixels from the reconstructed fire image; The pixel is compared and analyzed with a preset number of pixels to obtain the pixel similarity. If the similarity of the pixels is greater than a preset similarity threshold, then the location of the pixel is marked as a suspected fire source location; Based on a preset time period, the location of suspected fire sources in the reconstructed fire images at different time points is obtained; If the location of the suspected fire source in the reconstructed fire image moves between different time points and does not return to the corresponding location at the previous time point, a fire warning message will be triggered. The fire warning information is sent to a preset management terminal for notification.

[0011] In this solution, after triggering the fire alarm information, the following is also included: The suspected fire source locations were marked to determine the fire area and designated as key monitoring areas; Once the fire is extinguished, the risk factor for reignition is extracted from the time-series images of the fire after it occurred. Based on the reignition risk factors, predict the probability value of reignition risk within the key observation area; If the probability value of reignition risk is greater than the preset probability threshold, a reignition warning message is generated and sent to the preset management terminal for notification.

[0012] One or more technical solutions proposed in this application have at least the following technical effects: 1. By introducing a super-resolution adversarial network with multi-scale feature enhancement based on wavelet transform, the image is decomposed into low-frequency approximate sub-bands and high-frequency detail sub-bands. Then, shallow feature extraction and fusion processing are performed on each sub-band to reconstruct high-resolution images and improve image clarity. 2. Mark the fire area and conduct focused observation after the disaster to prevent the fire from reigniting. Attached Figure Description

[0013] Figure 1 A block diagram of a forest fire image reconstruction system based on satellite remote sensing according to the present invention is shown. Figure 2 A comparison image of the present invention is shown. Detailed Implementation

[0014] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0015] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0016] Figure 1 A block diagram of a forest fire image reconstruction system based on satellite remote sensing according to the present invention is shown.

[0017] like Figure 1 As shown, this invention discloses a forest fire image reconstruction system based on satellite remote sensing, comprising: an image acquisition module, an image feature extraction module, a multi-scale fusion module, and an image reconstruction module; The image acquisition module is used to acquire multi-temporal satellite remote sensing images of the target area before, during and after the fire, forming a time-series image sequence. The image feature extraction module is used to extract shallow features of images during the fire occurrence period in a time-series image sequence; The multi-scale fusion module is used to process and fuse the shallow features of the image to obtain multi-scale fused features. The image reconstruction module is used to extract prior information from multi-scale fusion features and fuse the prior information to obtain a reconstructed fire image.

[0018] Furthermore, the step of extracting shallow features from images during the fire occurrence period in the time-series image sequence specifically includes: The images during the fire are decomposed into a low-frequency approximate subband LL and a high-frequency detail subband (LH, HL, HH), with the following expressions: , , , ;in Represented as a low-pass filter, , , These are represented as horizontal, vertical, and diagonal high-pass filters, respectively. The size of the decomposed subband is , , , Where H represents the image height, W represents the image width, and C represents the number of image channels; Shallow feature extraction is performed on each sub-band, and the expressions are as follows: , ; , ;in , , , These are the shallow features of the four subbands: LL, LH, HL, and HH. for Convolution operation, LeakyReLU is the linearly corrected unit activation function.

[0019] Furthermore, the multi-scale fusion module includes: a hierarchical dense unit, a pixel-by-pixel addition unit, an input compression and excitation unit, a linear correction unit, and a high-frequency enhancement unit; The hierarchical dense unit is used to iteratively process the shallow features of each sub-band to obtain features. ; The pixel-by-pixel addition unit is used to process features through different convolutional layers. The features are processed and added pixel by pixel to obtain the features. ; The input compression and excitation unit is used for features Processing is performed to obtain features. ; The linear correction unit is used for features Linear correction is performed to obtain multi-scale fused features. Its expression is: , where P is Convolution adjusts the number of channels; The enhanced high-frequency unit is used for multi-scale fusion features. The process is performed to obtain multi-scale fusion features that enhance high-frequency information. .

[0020] According to an embodiment of the present invention, the method for iteratively processing the shallow features of each sub-band to obtain features is described. The steps specifically include: The specific formula for inputting the shallow features of each sub-band in the image into the hierarchical dense unit for iterative processing is as follows: Where: g = 1, 2, ... G; G is the total number of layers of the hierarchical dense unit. When g = 1, the shallow features of each sub-band are output. After each iteration, the features output from the previous iteration are... and the features of the current iteration output Perform difference calculation to obtain the first feature difference; If the first feature difference is less than or equal to the preset feature threshold, then the feature output of the current iteration will be... Set as feature ; If the difference in the first feature is greater than the preset feature threshold, the iteration continues until all dense units in all layers have completed iteration, and the output features are determined. and the features Set as feature .

[0021] Furthermore, the feature processing through different convolutional layers... The features are processed and added pixel by pixel to obtain the features. The steps specifically include: Features Send to Convolutional layers yield features Its expression is ; Features Send to Convolutional layers yield features Its expression is ; Features Send to Convolutional layers yield features Its expression is ; Features , and Perform pixel-by-pixel summation to obtain features .

[0022] Furthermore, the multi-scale fusion features The process is performed to obtain multi-scale fusion features that enhance high-frequency information. The expression is: ,in for convolution, ;in , , where BN represents batch normalization.

[0023] It should be noted that real-time monitoring of forests is achieved through satellite remote sensing, with periodic acquisition of forest images. These images are categorized based on their appearance before and after a fire: pre-fire, during-fire, and post-fire. A time-series image sequence is then constructed. For the images during the fire, wavelet transform is introduced for multi-scale decomposition, resulting in low-frequency approximate sub-bands (LL) and high-frequency detail sub-bands (LH, HL, HH). Then, shallow feature extraction is performed on each wavelet sub-band to capture its unique local details. , , , Then, the shallow features of each sub-band are iteratively optimized using hierarchical dense units to output features. ,in This can be represented as a concatenation operation of four shallow features, resulting in... For example, if G=4, then when and If the feature difference is still greater than the preset feature threshold, then... Set as output feature And terminate the iteration; then pass , and Three convolutional layers output features The process is performed, and the pixels are added one by one to determine the features. Its expression is Then the features Input compression and excitation units to obtain features Its expression is Then, the features are processed using the LeakyReLU linearly corrected unit activation function in the multi-scale fusion module. After correction, multi-scale fusion features are obtained. .

[0024] Furthermore, regarding the multi-scale fusion features... Processing is performed to determine the multi-scale fusion features that enhance high-frequency information. .

[0025] Furthermore, the reconstructed fire images are set as... Its expression is: ;in ; This is an upsampling operation, where r is the upsampling factor. This is for operations handled by the spatial global attention module. This is for processing operations by the channel attention module.

[0026] According to an embodiment of the present invention, it further includes: a fire warning module, which is used to extract pixels from the reconstructed fire image; The pixel is compared and analyzed with a preset number of pixels to obtain the pixel similarity. If the similarity of the pixels is greater than a preset similarity threshold, then the location of the pixel is marked as a suspected fire source location; Based on a preset time period, the location of suspected fire sources in the reconstructed fire images at different time points is obtained; If the location of the suspected fire source in the reconstructed fire image moves between different time points and does not return to the corresponding location at the previous time point, a fire warning message will be triggered. The fire warning information is sent to a preset management terminal for notification.

[0027] It should be noted that, based on the characteristics of the continuity and spread of fire, if the suspected fire source location moves and does not return to its previous position, it indicates that the current suspected fire source location is real, triggering a fire warning message.

[0028] According to an embodiment of the present invention, after triggering the fire alarm information, the method further includes: The suspected fire source locations were marked to determine the fire area and designated as key monitoring areas; Once the fire is extinguished, the risk factor for reignition is extracted from the time-series images of the fire after it occurred. Based on the reignition risk factors, predict the probability value of reignition risk within the key observation area; If the probability value of reignition risk is greater than the preset probability threshold, a reignition warning message is generated and sent to the preset management terminal for notification.

[0029] It should be noted that the reignition risk factors include the rate of surface temperature decay and the temporal change in the moisture content of combustibles, such as the temporal change rate of the fuel moisture content (FMC). The expression is: ,in and Reflectance in the shortwave infrared and near-infrared bands, respectively. Let be the empirical coefficient for vegetation volume density, then ,in For the j-th observation time point after the fire occurred, Let P be the observation time interval; and let P be the probability value of reignition risk, with the following formula: ,in , ,..., As a risk factor for resurgence, For the intercept term, , ,..., The regression coefficients for each factor are obtained by evaluating a sample of historical reignition events.

[0030] Figure 2 A comparison image of the present invention is shown.

[0031] like Figure 2 As shown, the clarity of the reconstructed forest fire images is significantly higher than that of the original images.

[0032] This invention discloses a forest fire image reconstruction system based on satellite remote sensing, comprising: an image acquisition module, an image feature extraction module, a multi-scale fusion module, and an image reconstruction module. The image acquisition module acquires multi-temporal satellite remote sensing images of a target area before, during, and after a fire, forming a time-series image sequence. The image feature extraction module extracts shallow features from the images during the fire period within the time-series image sequence. The multi-scale fusion module processes and fuses the shallow features of the images to obtain multi-scale fused features. The image reconstruction module extracts prior information from the multi-scale fused features and fuses the prior information to obtain a reconstructed fire image. This invention improves image clarity through super-resolution reconstruction of satellite remote sensing images, thereby enabling accurate identification and monitoring of forest fires and providing data support for fire early warning.

[0033] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0034] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0035] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0036] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0037] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A forest fire image reconstruction system based on satellite remote sensing, characterized in that, include: Image acquisition module, image feature extraction module, multi-scale fusion module, and image reconstruction module; The image acquisition module is used to acquire multi-temporal satellite remote sensing images of the target area before, during and after the fire, forming a time-series image sequence. The image feature extraction module is used to extract shallow features of images during the fire occurrence period in a time-series image sequence; The multi-scale fusion module is used to process and fuse the shallow features of the image to obtain multi-scale fused features. The image reconstruction module is used to extract prior information from multi-scale fusion features and fuse the prior information to obtain a reconstructed fire image.

2. The forest fire image reconstruction system based on satellite remote sensing according to claim 1, characterized in that, The step of extracting shallow features from images during the fire occurrence period in the time-series image sequence specifically includes: The images during the fire are decomposed into a low-frequency approximate subband LL and a high-frequency detail subband (LH, HL, HH), with the following expressions: , , , ;in Represented as a low-pass filter, , , These are represented as horizontal, vertical, and diagonal high-pass filters, respectively. The size of the decomposed subband is , , , ; Shallow feature extraction is performed on each sub-band, and the expressions are as follows: , ; , ;in , , , These are the shallow features of the four subbands: LL, LH, HL, and HH. for Convolution operation, LeakyReLU is the linearly corrected unit activation function.

3. A forest fire image reconstruction system based on satellite remote sensing according to claim 1, characterized in that, The multi-scale fusion module includes: a hierarchical dense unit, a pixel-by-pixel addition unit, an input compression and excitation unit, a linear correction unit, and an enhanced high-frequency unit; The hierarchical dense unit is used to iteratively process the shallow features of each sub-band to obtain features. ; The pixel-by-pixel addition unit is used to process features through different convolutional layers. The features are processed and added pixel by pixel to obtain the features. ; The input compression and excitation unit is used for features Processing is performed to obtain features. ; The linear correction unit is used for features Linear correction is performed to obtain multi-scale fused features. Its expression is: , where P is Convolution adjusts the number of channels; The enhanced high-frequency unit is used for multi-scale fusion features. The process is performed to obtain multi-scale fusion features that enhance high-frequency information. .

4. A forest fire image reconstruction system based on satellite remote sensing according to claim 3, characterized in that, The method involves iteratively processing the shallow features of each sub-band to obtain features. The steps specifically include: The specific formula for inputting the shallow features of each sub-band in the image into the hierarchical dense unit for iterative processing is as follows: Where: g = 1, 2, ... G; G is the total number of layers of the hierarchical dense unit. When g = 1, the shallow features of each sub-band are output. After each iteration, the features output from the previous iteration are... and the features of the current iteration output Perform difference calculation to obtain the first feature difference; If the first feature difference is less than or equal to the preset feature threshold, then the feature output of the current iteration will be... Set as feature ; If the difference in the first feature is greater than the preset feature threshold, the iteration continues until all dense units in all layers have completed iteration, and the output features are determined. and the features Set as feature .

5. A forest fire image reconstruction system based on satellite remote sensing according to claim 3, characterized in that, The features are processed through different convolutional layers. The features are processed and added pixel by pixel to obtain the features. The steps specifically include: Features Send to Convolutional layers yield features Its expression is ; Features Send to Convolutional layers yield features Its expression is ; Features Send to Convolutional layers yield features Its expression is ; Features , and Perform pixel-by-pixel summation to obtain features .

6. A forest fire image reconstruction system based on satellite remote sensing according to claim 3, characterized in that, The multi-scale fusion features The process is performed to obtain multi-scale fusion features that enhance high-frequency information. The expression is: ,in for convolution, ;in , , where BN represents batch normalization.

7. A forest fire image reconstruction system based on satellite remote sensing according to claim 1, characterized in that, Also includes: A fire warning module, which is used to extract pixels from reconstructed fire images; The pixel is compared and analyzed with a preset number of pixels to obtain the pixel similarity. If the similarity of the pixels is greater than a preset similarity threshold, then the location of the pixel is marked as a suspected fire source location; Based on a preset time period, the location of suspected fire sources in the reconstructed fire images at different time points is obtained; If the location of the suspected fire source in the reconstructed fire image moves between different time points and does not return to the corresponding location at the previous time point, a fire warning message will be triggered. The fire warning information is sent to a preset management terminal for notification.

8. A forest fire image reconstruction system based on satellite remote sensing according to claim 7, characterized in that, After the fire alarm message is triggered, the following is also included: The suspected fire source locations were marked to determine the fire area and designated as key monitoring areas; Once the fire is extinguished, the risk factor for reignition is extracted from the time-series images of the fire after it occurred. Based on the reignition risk factors, predict the probability value of reignition risk within the key observation area; If the probability value of reignition risk is greater than the preset probability threshold, a reignition warning message is generated and sent to the preset management terminal for notification.

Images