A method for detecting changes in flooded infrastructure based on feature fusion deep supervision networks

Abstract

This invention provides a method for detecting changes in flooded infrastructure based on a feature fusion deep supervised network. The method includes: acquiring pre-disaster and post-disaster images; inputting the pre-disaster images into a pre-disaster encoder to obtain multiple pre-disaster image features at different scales; inputting the post-disaster images into a post-disaster encoder to obtain multiple post-disaster image features at different scales; inputting the multiple pre-disaster and post-disaster image features at different scales into a multi-attention-constrained deep supervised multi-scale feature fusion module to obtain multiple pre-disaster channel spatial correction feature maps and multiple post-disaster channel spatial correction feature maps at different scales; and inputting the pre-disaster and post-disaster channel spatial correction feature maps into a decoder to obtain a change detection map. This invention can improve the monitoring capability of changes in flooded buildings and roads in complex flood scenarios.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method for detecting changes in flooded infrastructure based on feature fusion deep supervision networks. Background Technology

[0002] To mitigate the direct impact of flooding, first responders need to react quickly and assess the damage when or shortly after a flood occurs. The most important aspect is assessing the damage to buildings and roads to determine which structures have been destroyed and which roads are blocked by floodwaters.

[0003] Currently, in practical applications, the solution for identifying flooded buildings and roads is to directly overlay land cover data onto post-disaster flood vector maps. However, this approach has two drawbacks: firstly, outdated land cover data leads to misjudgments of disaster damage; secondly, floodwaters obscure buildings, preventing the detection of flooded structures.

[0004] This shows that the flood disaster assessment methods in related technologies have technical problems, such as low accuracy in monitoring changes in flooded buildings and roads. Summary of the Invention

[0005] This invention provides a method for detecting changes in flooded infrastructure based on feature fusion deep supervision networks, which addresses the shortcomings of existing flood disaster assessment methods, such as low accuracy in monitoring changes in flooded buildings and roads, and improves the ability to monitor changes in flooded buildings and roads in complex flood scenarios.

[0006] This invention provides a method for detecting changes in flooded infrastructure based on a feature fusion deep supervised network, comprising the following steps: acquiring pre-disaster and post-disaster images, wherein the pre-disaster images represent images of the target object before it suffers from flooding, and the post-disaster images represent images of the target object after it suffers from flooding; inputting the pre-disaster images into a pre-disaster encoder to obtain multiple pre-disaster image features at different scales output by the pre-disaster encoder; inputting the post-disaster images into a post-disaster encoder to obtain multiple post-disaster image features at different scales output by the post-disaster encoder; inputting the multiple pre-disaster image features at different scales into a multi-attention-constrained deep supervised multi-scale feature fusion module to obtain multiple pre-disaster channel spaces at different scales output by the multi-attention-constrained deep supervised multi-scale feature fusion module. Correction feature maps; inputting the multiple post-disaster image features at different scales into the multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple post-disaster channel spatial correction feature maps at different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module; wherein, the multi-attention constraint includes channel attention mechanism and spatial attention mechanism; inputting the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map into the decoder to obtain the change detection map output by the decoder, wherein, the pre-disaster channel spatial correction feature map and the pre-disaster channel spatial correction feature map have the same scale, and the change detection map is used to represent the change monitoring comparison results of the target object.

[0007] According to the present invention, a method for detecting changes in flooded infrastructure based on a feature fusion deep supervised network is provided. The pre-disaster encoder includes: a first pre-disaster encoder, a second pre-disaster encoder, a third pre-disaster encoder, and a fourth pre-disaster encoder. The step of inputting the pre-disaster image into the pre-disaster encoder to obtain multiple pre-disaster image features of different scales output by the pre-disaster encoder includes: inputting the pre-disaster image into the first pre-disaster encoder to obtain a first pre-disaster image feature output by the first pre-disaster encoder, wherein the scale of the first pre-disaster image feature is one-quarter of the scale of the pre-disaster image, and the number of channels of the first pre-disaster image feature is 1; inputting the first pre-disaster image feature into the second pre-disaster encoder to obtain a second pre-disaster image feature. The second pre-disaster image feature output by the first encoder has a scale of one-eighth of the pre-disaster image and a channel count of 2. The second pre-disaster image feature is then input to the third pre-disaster encoder to obtain a third pre-disaster image feature output by the third encoder. The third pre-disaster image feature has a scale of one-sixteenth of the pre-disaster image and a channel count of 4. The third pre-disaster image feature is then input to the fourth pre-disaster encoder to obtain a fourth pre-disaster image feature output by the fourth encoder. The fourth pre-disaster image feature has a scale of one-thirty-second of the pre-disaster image and a channel count of 8.

[0008] According to the present invention, a method for detecting changes in flooded infrastructure based on feature fusion deep supervised networks is provided. The number of channels of the pre-disaster encoder is 192, 384, 768, and 1536, respectively; the number of modules of the pre-disaster encoder is 3, 3, 9, and 3, respectively; the structure of the pre-disaster encoder includes: depthwise convolution, layer normalization, first pointwise convolution, Gaussian error linear unit, and second pointwise convolution.

[0009] According to the present invention, a method for detecting changes in flooded infrastructure based on a feature fusion deep supervised network is provided. The method involves inputting multiple pre-disaster image features at different scales into a multi-attention-constrained deep supervised multi-scale feature fusion module to obtain multiple pre-disaster channel spatial correction feature maps at different scales output by the multi-attention-constrained deep supervised multi-scale feature fusion module. This includes: taking each of the multiple different scales as a target scale and performing the following operations to obtain a pre-disaster channel spatial correction feature map at the target scale: upsampling the multiple pre-disaster image features at different scales to obtain multiple pre-disaster image features at the target scale with different numbers of channels; and upsampling the multiple pre-disaster image features at different scales... The target-scale pre-disaster image features are stitched together to obtain a target-scale feature map. This target-scale feature map is then input into a channel attention module to obtain a channel attention feature map output by the module. The channel attention feature map is multiplied by the target-scale feature map to obtain a channel correction feature map. The target-scale feature map is then input into a spatial attention module to obtain a spatial attention feature map output by the module. This spatial attention feature map is multiplied by the target-scale feature map to obtain a spatial correction feature map. Finally, the channel correction feature map is multiplied by the spatial correction feature map to obtain a target-scale pre-disaster channel spatial correction feature map.

[0010] According to the present invention, a method for detecting changes in flooded infrastructure based on a feature fusion deep supervision network is provided. The decoder includes a first decoder, a second decoder, a third decoder, and a fourth decoder. The step of inputting the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map into the decoder to obtain the change detection map output by the decoder includes: performing channel stitching on the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map at the fourth scale to obtain a fourth stitched feature map; inputting the fourth stitched feature map into the fourth decoder to obtain a third-scale feature map output by the fourth decoder; and processing the pre-disaster channel spatial correction feature map, the post-disaster channel spatial correction feature map, and the third-scale feature map at the third scale. The image is stitched together to obtain a third stitched feature map; the third stitched feature map is input to a third decoder to obtain a second-scale feature map output by the third decoder; the pre-disaster channel spatial correction feature map, the post-disaster channel spatial correction feature map, and the second-scale feature map at the second scale are stitched together to obtain a second stitched feature map; the second stitched feature map is input to a second decoder to obtain a first-scale feature map output by the second decoder; the pre-disaster channel spatial correction feature map, the post-disaster channel spatial correction feature map, and the first-scale feature map at the first scale are stitched together to obtain a first stitched feature map; the first stitched feature map is input to a first decoder to obtain a change detection map output by the first decoder.

[0011] According to the present invention, a method for detecting changes in flooded infrastructure based on a feature fusion deep supervised network is provided. The decoder structure includes: convolution, first batch normalization, first corrected linear unit, inverse convolution, second batch normalization, and second corrected linear unit. The present invention also provides a device for detecting changes in flooded infrastructure based on a feature fusion deep supervised network, comprising the following modules: an acquisition module for acquiring pre-disaster images and post-disaster images, wherein the pre-disaster images represent images of the target object before it suffers from flooding, and the post-disaster images represent images of the target object after it suffers from flooding; a pre-disaster encoding module for inputting the pre-disaster images into a pre-disaster encoder to obtain multiple pre-disaster image features of different scales output by the pre-disaster encoder; a post-disaster encoding module for inputting the post-disaster images into a post-disaster encoder to obtain multiple post-disaster image features of different scales output by the post-disaster encoder; and a multi-scale fusion module for inputting the multiple pre-disaster image features of different scales into a multi-attention-constrained deep supervised multi-scale feature fusion module to obtain the multi-attention-constrained... The deep-supervised multi-scale feature fusion module outputs multiple pre-disaster channel spatial correction feature maps at different scales; the multi-scale fusion module is further used to input the multiple post-disaster image features at different scales into the multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple post-disaster channel spatial correction feature maps at different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module; wherein, the multi-attention constraint includes channel attention mechanism and spatial attention mechanism; the decoding module is used to input the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map into the decoder to obtain the change detection map output by the decoder, wherein, the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map have the same scale, and the change detection map is used to represent the change monitoring comparison result of the target object.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the flooded infrastructure change detection method based on feature fusion deep supervision network as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the flooded infrastructure change detection method based on feature fusion deep supervision network as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the flooded infrastructure change detection method based on feature fusion deep supervision network as described above.

[0015] This invention provides a method for detecting changes in flooded infrastructure based on a feature fusion deep-supervised network. By acquiring pre-flood images and post-flood images of the target object, and inputting these images into a pre-flood encoder and a post-flood encoder respectively, the method can extract features from multiple pre-flood and post-flood images at different scales. This multi-scale feature extraction enhances the model's ability to recognize target objects of different sizes and shapes, improving the accuracy of change detection. The method employs a deep-supervised multi-scale feature fusion module with multi-attention constraints to fuse features from pre-flood and post-flood images at different scales. Simultaneously, the deep-supervised strategy ensures that features at each level are effectively utilized, helping to capture change information at different scales. This improves the robustness and generalization ability of the model; it introduces channel attention and spatial attention mechanisms to focus on important channels and spatial locations in the image, thereby enhancing the ability to extract key information; it inputs the channel spatial correction feature map before the disaster and the channel spatial correction feature map after the disaster into the decoder to obtain the change detection map. Due to the combined effect of multi-scale feature extraction, multi-attention constraints and deep supervised feature fusion in the previous steps, the change detection map can accurately reflect the changes of the target object before and after the flood disaster, improve the ability to monitor changes of flooded buildings and roads in complex flood scenes, and thus solve the technical problem of low accuracy in monitoring changes of flooded buildings and roads in flood disaster assessment methods in related technologies. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the method for detecting changes in flooded infrastructure based on feature fusion deep supervision networks provided by the present invention.

[0018] Figure 2 This is a simplified change detection structure diagram of the flooded infrastructure change detection method provided by the present invention.

[0019] Figure 3 This is the overall network structure diagram of the flooded infrastructure change detection method based on feature fusion deep supervision network provided by the present invention.

[0020] Figure 4This is a detailed network structure diagram of the flooded infrastructure change detection method based on feature fusion deep supervision network provided by the present invention.

[0021] Figure 5 This is a diagram of the deep supervision high-dimensional-low-dimensional multi-scale feature fusion module with multi-attention constraints provided by the present invention.

[0022] Figure 6 This is a comparison chart of monitoring results for changes in flooded buildings and roads, provided by the present invention, using common change detection structures.

[0023] Figure 7 This is a comparison chart of ablation experiment results provided by the present invention.

[0024] Figure 8 This is a comparison chart of monitoring results of changes in flooded buildings and roads using MSFF embedded with different deep learning models, provided by this invention.

[0025] Figure 9 This is a schematic diagram of the structure of the flooded infrastructure change detection device based on feature fusion deep supervision network provided by the present invention.

[0026] Figure 10 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] Floods are a major natural disaster, and with global climate change, they are becoming more frequent, sudden, and have significant chain effects. Floods cause enormous loss of life, infrastructure damage, and property loss, as well as a series of other environmental problems such as post-disaster cleanup and reconstruction, and healthcare. To mitigate the direct impact of floods, emergency responders need to react quickly and assess the damage when or shortly after a flood occurs. The most important aspect is assessing the damage to buildings and roads to determine which areas require assistance and which roads are blocked by floodwaters. Remote sensing imagery is a primary tool for quickly mapping flood-affected areas, but detailed emergency monitoring and damage assessment of the disaster situation are time-consuming. In flood disasters, most submerged buildings and roads remain in normal condition; buildings block floodwaters, rendering conventional flood detection methods unable to detect submerged buildings, and the elongated shape of roads also reduces the model's extraction capabilities. Therefore, using deep learning models to quickly and accurately quantify the damage to buildings and roads is of great significance.

[0029] Currently, in practical applications, the solution for identifying flooded buildings and roads is to directly overlay land cover data onto post-flood vector maps. However, this approach has limitations. Firstly, outdated land cover data leads to misjudgments of disaster damage. Secondly, buildings often obscure floodwaters, preventing the detection of flooded structures. Dual-temporal image change detection technology can accurately determine the damage status of infrastructure before and after flooding. Through analysis and comparison of geographic information and remote sensing imagery from different periods, remote sensing image change detection plays a crucial role in geographic data updates and disaster damage assessment. Detecting changes in damaged infrastructure is more challenging than detecting changes in buildings and roads in their normal state. This is because disaster scenarios present additional interference features: such as damage to non-building structures in earthquakes (the detection of damaged infrastructure in earthquakes has been extensively studied), and the mixture of various land features in floods. However, detecting changes in flooded buildings and roads remains a difficult task. Unlike earthquakes that cause complete destruction and severe collapse, most flooded buildings and roads appear normal from an aerial view during floods. Therefore, there is an urgent need for feature extractors with stronger context awareness capabilities, as well as change detection structures capable of fully mining the semantic information of both pre- and post-flood temporal phases. With the introduction of the ultra-high resolution flooded building and road dataset xBD, change detection structures have been initially applied to the detection of flooded buildings and roads in flood scenarios. However, different change detection structures vary in their ability to characterize changes in flooded buildings and roads, and it remains unclear which change detection structure is the most effective.

[0030] Deep learning models excel at discovering high-dimensional structures in complex data, leveraging convolutional structures to capture spatial contextual information, playing a crucial role in disaster emergency monitoring and damage assessment. Pioneering applications of convolutional neural networks for image semantic segmentation, with innovations in the encoder-decoder structure and improved upsampling performance by preserving pooling indices, have further enhanced the model's understanding of complex scenes through U-shaped structures, pyramid pooling modules, dilated convolutions, and multi-branch parallel convolutions, effectively fusing multi-scale contextual information. However, these studies still face limitations in reducing complexity while generating more powerful multi-scale features, even when fully considering the use of scale-related contextual information. A new approach involves constructing extremely deep networks using residual structures and balancing network width, depth, and resolution using NAS search techniques. Introduced from the natural language domain, this approach models sequences based on attention mechanisms, borrowing design principles from Transformers and implementing their modules entirely using deep learning models (CNNs). These encoder design approaches significantly enhance the feature extraction capabilities of the network structure.

[0031] Multi-scale feature fusion and layer-based feature fusion strategies are designed to fuse high-dimensional and low-dimensional features, introducing a large number of channels into the fusion strategy. Multi-channel convolutional kernels exhibit significant spatial similarity and channel redundancy. To alleviate channel redundancy, channel attention mechanisms (e.g., Squeeze-and-Excitation (SE) and Efficient Channel Attention (ECA)) focus on channel-dimensional attention, assigning appropriate weights to feature channels at different levels. Since convolutional kernels produce similar responses to similar patterns in local space, Convolutional Block Attention Module (CBAM) and Concurrent Spatial and Channel Squeeze & Excitation (scSE) consider both spatial and channel-dimensional attention, further addressing the spatial independence problem. Some studies embed attention mechanisms in the encoder or decoder of the network structure, neglecting the independence of high- and low-dimensional multi-scale features, resulting in limited utilization of multi-scale features in change detection tasks. Dynamic multi-scale feature fusion or multi-attention constraint methods are designed to learn fusion weights from features, guiding the organic fusion of shallow and deep semantic features. These attention-based multi-scale feature fusions rely on specific deep learning models and cannot be directly applied to deep learning models with different structures.

[0032] In summary, in flood emergency monitoring and disaster assessment tasks, current professional researchers in operational applications use flood vector maps overlaid with land cover data. However, due to outdated land cover data or buildings obscuring floodwaters, the accuracy of monitoring changes in submerged buildings and roads is low. This invention designs a floodCD structure utilizing pre- and post-flood dual-temporal image change detection. Different feature extractors and network structures are combined and their ability to detect changes in submerged buildings and roads in ultra-high resolution remote sensing images is tested. Regarding the different capabilities of deep learning models in complex flood scenarios, this invention explores the effectiveness of different network structures in detecting changes in submerged buildings and roads. Facing the problem that high-dimensional and low-dimensional feature maps of different sizes extracted by deep learning models at each stage are not effectively fused and utilized, this invention constructs a plug-and-play multi-attention-constrained multi-scale feature extraction module and employs a deep supervised learning strategy to fully utilize global-local multi-scale features of the flood scenario to improve change detection accuracy. The flood infrastructure change detection method based on feature fusion and deep supervised networks proposed in this invention improves the ability to monitor changes in submerged buildings and roads in complex flood scenarios, confirming its superiority in flood scenarios.

[0033] Optionally, the flooded infrastructure change detection method based on feature fusion deep supervision network in this embodiment of the invention can be executed by a server, by a terminal device, or by both a server and a terminal device. Taking the execution of the flooded infrastructure change detection method based on feature fusion deep supervision network in this embodiment by a server as an example.

[0034] Figure 1 This is a flowchart illustrating the method for detecting changes in flooded infrastructure based on feature fusion deep supervision networks provided by this invention. Figure 1 As shown, the method includes the following steps.

[0035] Step 101: Obtain pre-disaster and post-disaster images.

[0036] Among them, pre-disaster images are used to represent images of the target object before it was affected by floods, and post-disaster images are used to represent images of the target object after it was affected by floods.

[0037] In the task of emergency monitoring and disaster assessment of flood disasters, professional researchers in the field of business application currently use flood vector maps overlaid with land cover data. However, due to the failure to update the land cover data in a timely manner or the obstruction of floodwaters by buildings, the accuracy of monitoring changes in flooded buildings and roads is not high.

[0038] In this embodiment of the invention, pre-disaster images of the target object before it suffers from flooding and post-disaster images of it after it suffers from flooding are acquired. Here, the pre-disaster images and post-disaster images can be ultra-high resolution remote sensing images of the target object; the target object can be a building or road that has been flooded, or a building object affected by flooding.

[0039] Step 102: Input the pre-disaster images into the pre-disaster encoder to obtain multiple pre-disaster image features of different scales output by the pre-disaster encoder.

[0040] In this embodiment of the invention, a pre-disaster encoder is used to encode pre-disaster images and extract features of multiple pre-disaster images at different scales, which can capture information at different levels in the images.

[0041] Here, the pre-disaster encoder may include a first pre-disaster encoder, a second pre-disaster encoder, a third pre-disaster encoder, and a fourth pre-disaster encoder, which are used to convert the input pre-disaster images into pre-disaster image features of different scales.

[0042] For example, pre-disaster image features at different scales include: pre-disaster image features at one-quarter scale, pre-disaster image features at one-eighth scale, pre-disaster image features at one-sixteenth scale, and pre-disaster image features at one-thirty-second scale.

[0043] It should be noted that the number of channels for pre-disaster image features varies at different scales. For example, the number of channels for pre-disaster image features at a quarter scale is 1, the number of channels for pre-disaster image features at a quarter scale is 2, the number of channels for pre-disaster image features at a sixteenth scale is 4, and the number of channels for pre-disaster image features at a thirty-second scale is 8.

[0044] Through the embodiments of the present invention, the pre-disaster encoder can extract pre-disaster image features at multiple different scales, covering different levels of information from global to local, which helps to understand the content of pre-disaster images more comprehensively.

[0045] Step 103: Input the post-disaster images into the post-disaster encoder to obtain multiple post-disaster image features of different scales output by the post-disaster encoder.

[0046] In this embodiment of the invention, a post-disaster encoder is used to encode post-disaster images and extract post-disaster image features at multiple different scales, which can capture information at different levels in the images.

[0047] Here, the post-disaster encoder may include a first post-disaster encoder, a second post-disaster encoder, a third post-disaster encoder, and a fourth post-disaster encoder, which are used to convert the input post-disaster images into post-disaster image features of different scales.

[0048] For example, post-disaster image features at different scales include: one-quarter scale, one-eighth scale, one-sixteenth scale, and one-thirty-second scale.

[0049] It should be noted that the number of channels for post-disaster image features varies at different scales. For example, the number of channels for post-disaster image features at a quarter scale is 1, the number of channels for post-disaster image features at a quarter scale is 2, the number of channels for post-disaster image features at a sixteenth scale is 4, and the number of channels for post-disaster image features at a thirty-second scale is 8.

[0050] Through the embodiments of the present invention, the post-disaster encoder can extract post-disaster image features at multiple different scales, covering different levels of information from global to local, which helps to understand the content of post-disaster images more comprehensively.

[0051] Step 104: Input multiple pre-disaster image features at different scales into the multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple pre-disaster channel spatial correction feature maps at different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module.

[0052] In this embodiment of the invention, a multi-attention mechanism is introduced during the feature fusion process, including a channel attention mechanism and a spatial attention mechanism; deep supervision refers to applying supervision at different scales of feature fusion to ensure that features at each scale are fully utilized.

[0053] Here, deep supervision is used to add an auxiliary classifier as a branch of the network in the hidden layer of the target in a deep neural network to supervise the backbone network, in order to solve problems such as vanishing gradients and slow convergence speed in deep neural network training.

[0054] Under multiple attention constraints and deep supervision, multi-scale feature fusion is performed on pre-disaster image features at multiple different scales according to the target scale to obtain a pre-disaster channel spatial correction feature map at the target scale. The target scale is then changed, and the multi-scale feature fusion is repeated until all different scales are traversed to obtain a pre-disaster channel spatial correction feature map at each of the multiple different scales.

[0055] Through the embodiments of the present invention, after processing by the deep-supervised multi-scale feature fusion module with multi-attention constraints, multiple pre-disaster channel spatial correction feature maps of different scales will be output. These maps not only contain information from the original image (i.e., pre-disaster image), but also have been enhanced by multi-scale, multi-attention mechanisms and deep supervision, resulting in higher information density and accuracy.

[0056] Step 105: Input multiple post-disaster image features at different scales into the multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple post-disaster channel spatial correction feature maps at different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module.

[0057] Among them, multi-attention constraints include channel attention mechanisms and spatial attention mechanisms;

[0058] Under multiple attention constraints and deep supervision, multi-scale feature fusion is performed on post-disaster image features at multiple different scales according to the target scale to obtain a post-disaster channel spatial correction feature map at the target scale. The target scale is then changed, and the multi-scale feature fusion is repeated until all different scales are traversed to obtain a post-disaster channel spatial correction feature map at each of the multiple different scales.

[0059] Through the embodiments of the present invention, after processing by the deep-supervised multi-scale feature fusion module with multi-attention constraints, multiple post-disaster channel spatial correction feature maps of different scales will be output. These maps not only contain information from the original images (i.e., post-disaster images), but also have been enhanced by multi-scale, multi-attention mechanisms and deep supervision, resulting in higher information density and accuracy.

[0060] Step 106: Input the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map into the decoder to obtain the change detection map output by the decoder.

[0061] Among them, the pre-disaster channel spatial correction feature map has the same scale as the pre-disaster channel spatial correction feature map, and the change detection map is used to represent the comparison results of change monitoring of the target object.

[0062] In this embodiment of the invention, the decoder includes a first decoder, a second decoder, a third decoder, and a fourth decoder, each decoder being used to decode pre-disaster channel spatial correction feature maps and post-disaster channel spatial correction feature maps at different scales.

[0063] In this embodiment of the invention, inputting the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map into the decoder can generate a change detection map, which is used to show the comparison results of the changes of the target object before and after the flood disaster, providing an important basis for disaster assessment, emergency response and post-disaster reconstruction.

[0064] refer to Figure 2 , Figure 2 This is a simplified change detection structure diagram of the flooded infrastructure change detection method provided by the present invention.

[0065] like Figure 2As shown, pre-disaster and post-disaster images are input into the encoder to obtain pre-disaster image features at multiple scales and post-disaster image features at multiple scales. These features are then input into a deep-supervised multi-scale feature fusion module with multi-attention constraints to obtain pre-disaster channel spatial correction feature maps at multiple scales and post-disaster channel spatial correction feature maps at multiple scales. Finally, these pre-disaster and post-disaster channel spatial correction feature maps are stitched together and input into the decoder to obtain a change detection map.

[0066] Through the steps described in this embodiment of the invention, pre-disaster images and post-disaster images are obtained. The pre-disaster images represent images of the target object before it suffers from flooding, and the post-disaster images represent images of the target object after it suffers from flooding. The pre-disaster images are input to a pre-disaster encoder to obtain multiple pre-disaster image features at different scales output by the encoder. The post-disaster images are input to a post-disaster encoder to obtain multiple post-disaster image features at different scales output by the encoder. These multiple pre-disaster image features at different scales are then input to a multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple features output by the multi-attention-constrained deep-supervised multi-scale feature fusion module. Pre-disaster spatial correction feature maps of passageways at different scales are generated. Multiple post-disaster image features at different scales are input into a deep-supervised multi-scale feature fusion module with multi-attention constraints, resulting in multiple post-disaster spatial correction feature maps of passageways at different scales output by the module. The multi-attention constraints include both passageway attention and spatial attention mechanisms. The pre-disaster and post-disaster spatial correction feature maps are input into a decoder to obtain a change detection map output by the decoder. The pre-disaster and post-disaster spatial correction feature maps are at the same scale, and the change detection map is used to represent the comparison results of change monitoring of target objects. This improves the ability to monitor changes in flooded buildings and roads in complex flood scenarios, thereby solving the technical problem of low accuracy in monitoring changes in flooded buildings and roads in related flood disaster assessment methods.

[0067] According to the method for detecting changes in flooded infrastructure based on feature fusion deep supervised networks provided by the present invention, the pre-disaster encoder includes: a first pre-disaster encoder, a second pre-disaster encoder, a third pre-disaster encoder, and a fourth pre-disaster encoder; pre-disaster images are input into the pre-disaster encoder to obtain multiple pre-disaster image features of different scales output by the pre-disaster encoder, including:

[0068] The pre-disaster image is input into the first pre-disaster encoder to obtain the first pre-disaster image feature output by the first pre-disaster encoder. The scale of the first pre-disaster image feature is one-quarter of the pre-disaster image, and the number of channels of the first pre-disaster image feature is 1.

[0069] The first pre-disaster image features are input into the second pre-disaster encoder to obtain the second pre-disaster image features output by the second pre-disaster encoder. The scale of the second pre-disaster image features is one-eighth of the pre-disaster image, and the number of channels of the second pre-disaster image features is 2.

[0070] The second pre-disaster image features are input into the third pre-disaster encoder to obtain the third pre-disaster image features output by the third pre-disaster encoder. The scale of the third pre-disaster image features is one-sixteenth of the pre-disaster image, and the number of channels of the third pre-disaster image features is 4.

[0071] The third pre-disaster image feature is input into the fourth pre-disaster encoder to obtain the fourth pre-disaster image feature output by the fourth pre-disaster encoder. The scale of the fourth pre-disaster image feature is one thirty-second of the pre-disaster image, and the number of channels of the fourth pre-disaster image feature is 8.

[0072] Here, the scale of the feature is the height (H) and width (W) of the feature map.

[0073] refer to Figure 3 , Figure 3 This is the overall network structure diagram of the flooded infrastructure change detection method based on feature fusion deep supervision network provided by the present invention.

[0074] The pre-disaster image is input into the first encoder (encoder_1) to obtain the first pre-disaster image features. ,in, Indicates features, Indicates encoder, Indicating before the disaster, Represents the set of real numbers. and These represent the height and width of the original image (pre-disaster imagery), respectively. The number of channels representing the features); the first pre-disaster image features are input into the second encoder (encoder_2) to obtain the second pre-disaster image features ( The second pre-disaster image features are input into the third encoder (encoder_3) to obtain the third post-disaster image features. The third post-disaster image features are input into the fourth encoder (encoder_4) to obtain the fourth post-disaster image features. ).

[0075] The encoding process for post-disaster images is the same as that for pre-disaster images (wherein, (This refers to the aftermath of a disaster), and the invention will not be described in detail here.

[0076] In some embodiments, the post-disaster encoder includes: a first post-disaster encoder, a second post-disaster encoder, a third post-disaster encoder, and a fourth post-disaster encoder.

[0077] The post-disaster image is input into the first post-disaster encoder to obtain the first post-disaster image feature output by the first post-disaster encoder. The scale of the first post-disaster image feature is one-quarter of the post-disaster image, and the number of channels of the first post-disaster image feature is 1.

[0078] The first post-disaster image feature is input into the second post-disaster encoder to obtain the second post-disaster image feature output by the second post-disaster encoder. The scale of the second post-disaster image feature is one-eighth of the post-disaster image, and the number of channels of the second post-disaster image feature is 2.

[0079] The second post-disaster image features are input into the third post-disaster encoder to obtain the third post-disaster image features output by the third post-disaster encoder. The scale of the third post-disaster image features is one-sixteenth of the post-disaster image, and the number of channels of the second post-disaster image features is 4.

[0080] The third post-disaster image feature is input into the fourth post-disaster encoder to obtain the fourth post-disaster image feature output by the fourth post-disaster encoder. The scale of the fourth post-disaster image feature is one thirty-second of the post-disaster image. The number of channels of the second post-disaster image feature is 8.

[0081] Through the embodiments of the present invention, pre-disaster images and post-disaster images can be encoded into features of different scales, thereby obtaining information at different levels from global to local in the images.

[0082] According to the feature fusion-based deep supervised network-based flooded infrastructure change detection method provided by the present invention, the number of channels of the pre-disaster encoder is 192, 384, 768, and 1536, respectively; the number of modules of the pre-disaster encoder is 3, 3, 9, and 3, respectively; the structure of the pre-disaster encoder includes: depthwise convolution, layer normalization, first pointwise convolution, Gaussian error linear unit, and second pointwise convolution.

[0083] refer to Figure 4 , Figure 4 This is a detailed network structure diagram of the flooded infrastructure change detection method based on feature fusion deep supervised network provided by the present invention, which includes multiple encoders, a deep supervised multi-scale feature fusion module with multiple attention constraints, multiple channel splicing, and multiple decoders.

[0084] It should be noted that encoders include pre-disaster encoders and post-disaster encoders, and the structure and characteristics of pre-disaster encoders and post-disaster encoders are the same.

[0085] In this embodiment of the invention, to balance the number of model parameters and the performance of feature extraction, ConvNeXt-Tiny is introduced as the backbone of the encoder, and pre-trained weights are added on top of it to construct a system as follows: Figure 4The encoder, decoder, and high-dimensional-low-dimensional multi-scale feature fusion module (i.e., a deep-supervised multi-scale feature fusion module with multiple attention constraints) shown are illustrated. The encoder has four stages with channels of (192, 384, 768, 1536), and the number of modules is (3, 3, 9, 3), respectively. The encoder structure includes depthwise convolution, layer normalization (LayerNorm), pointwise convolution, Gaussian error linear unit (GELU), and pointwise convolution. This encoder structure has stronger context awareness and extracts more accurate semantic information.

[0086] Deep convolution can capture local features of input data, and the range of features captured expands as the kernel size increases. In encoder architectures, deep convolutional layers are typically located at the beginning of the module to initially extract features from the input data.

[0087] LayerNorm is a normalization technique used to normalize each sample of the input data. It can help accelerate the model training process and improve model stability. In the encoder architecture, LayerNorm is typically located after the convolutional layers and is used to normalize the output of the convolutional layers.

[0088] Pointwise convolution (also known as 1x1 convolution) is a special type of convolution operation with a 1x1 kernel. It is primarily used to change the number of channels in the input data while maintaining the spatial resolution of the data. In encoder architectures, pointwise convolution is commonly used to connect modules at different stages and to perform dimensionality reduction or enhancement operations on features.

[0089] GELU (Gaussian Error Linear Unit) is a non-linear activation function that combines the advantages of ReLU activation with the randomness of a Gaussian distribution. In the encoder structure, the GELU activation function is used to increase the non-linear expressive power of the model, thereby capturing more complex features.

[0090] Through the embodiments of this application, the deep convolutional layers and pointwise convolutional layers in the encoder structure gradually extract deeper features by continuously convolving and transforming the input data. These features not only contain local information of the input data but also contextual information. Therefore, the encoder structure has strong context-aware capabilities; at the same time, components such as LayerNorm and GELU also help improve the model's ability to extract semantic information.

[0091] According to the feature fusion-based deep-supervised network-based method for detecting changes in flooded infrastructure provided by this invention, multiple pre-disaster image features at different scales are input into a multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple pre-disaster channel spatial correction feature maps at different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module, including:

[0092] Perform the following operations on each of the multiple different scales as the target scale to obtain the pre-disaster channel spatial correction feature map at the target scale:

[0093] Upsampling was performed on pre-disaster image features at multiple different scales to obtain pre-disaster image features at multiple target scales with different numbers of channels.

[0094] Channel stitching is performed on the features of pre-disaster images at multiple target scales with different numbers of channels to obtain a target scale feature map;

[0095] The target scale feature map is input into the channel attention module to obtain the channel attention feature map output by the channel attention module;

[0096] Multiply the channel attention feature map with the target scale feature map to obtain the channel correction feature map;

[0097] The target scale feature map is input into the spatial attention module to obtain the spatial attention feature map output by the spatial attention module;

[0098] Multiply the spatial attention feature map with the target scale feature map to obtain the spatial correction feature map;

[0099] Multiply the channel correction feature map with the spatial correction feature map to obtain the pre-disaster channel spatial correction feature map at the target scale.

[0100] refer to Figure 5 , Figure 5 This is a diagram of the deep supervision high-dimensional-low-dimensional multi-scale feature fusion module with multi-attention constraints provided by the present invention.

[0101] The target scale is determined by setting one-quarter scale of the pre-disaster imagery. Features of the pre-disaster imagery at one-quarter scale with one channel are then analyzed. Upsampling was performed to obtain pre-disaster image features at a quarter-scale with 1 channel. ).

[0102] Features of pre-disaster images at an 1 / 8 scale with 2 channels ( Upsampling was performed to obtain pre-disaster image features at a quarter-scale with 2 channels. ).

[0103] Features of pre-disaster images at a scale of 1 / 16 and with 4 channels ( Upsampling was performed to obtain pre-disaster image features at a quarter-scale with 4 channels. ).

[0104] Features of pre-disaster images at a scale of 1 / 32 with 8 channels ( Upsampling was performed to obtain pre-disaster image features at a quarter-scale with 8 channels. ).

[0105] Channel stitching is performed on the pre-disaster image features obtained by upsampling to a target scale of one-quarter scale to obtain a target scale feature map.

[0106] The target-scale feature map is input into the spatial-channel compression activation module.

[0107] Adaptive average pooling is performed on the target-scale feature map to obtain the first spatial feature ( ); perform convolution and modified linear transformation on the first spatial features to obtain the second spatial features ( ); Convolution and activation are performed on the second spatial features to obtain the third spatial features. The spatial attention feature map is obtained by multiplying the third spatial feature map with the target scale feature map. The spatial compression-activation feature map (i.e., the spatial correction feature map) is obtained by multiplying the third spatial feature map with the target scale feature map.

[0108] The Modified Linear Unit (ReLU) is a commonly used activation function, typically used to increase the non-linearity of a network. ReLU layers usually follow convolutional layers, performing a non-linear transformation on the convolutional layer's output, setting all negative values ​​to 0 while leaving positive values ​​unchanged.

[0109] For example, first, the convolutional layer uses a set of convolutional filters to convolve the input image, generating a series of feature maps. Then, the ReLU layer (i.e., the rectified linear unit) performs a non-linear transformation on the output of the convolutional layer. Specifically, each element in the feature map is processed by the ReLU function, setting all negative values ​​to 0 and leaving positive values ​​unchanged.

[0110] Convolution is performed on the target scale feature map to obtain the first channel feature ( The channel attention feature map is obtained by multiplying the channel attention feature map with the target scale feature map. The channel compression-activation feature map (i.e., the channel correction feature map) is obtained by multiplying the channel attention feature map with the target scale feature map.

[0111] Multiplying the spatial compressed-activation feature map and the channel compressed-activation feature map yields the spatial channel compressed-activation feature map. ), which is the channel space correction feature map.

[0112] Here, the target scales of one-quarter, one-eighth, one-sixteenth, and one-thirty-second are used respectively. Upsampling, channel stitching, and spatial-channel compression activation operations are repeated to obtain pre-disaster channel spatial correction feature maps at each different scale.

[0113] It is understandable that the specific process of inputting multiple post-disaster image features of different scales into a multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple post-disaster channel spatial correction feature maps of different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module is the same as the above process, only the input is replaced with post-disaster images, and will not be described again in this invention.

[0114] According to the feature fusion-based deep supervision network-based method for detecting changes in flooded infrastructure provided by the present invention, the decoder includes a first decoder, a second decoder, a third decoder, and a fourth decoder. The pre-disaster spatial correction feature map and the post-disaster spatial correction feature map are input into the decoder to obtain the change detection map output by the decoder, including:

[0115] The pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map at the fourth scale are stitched together to obtain the fourth stitched feature map.

[0116] The fourth stitched feature map is input into the fourth decoder to obtain the third scale feature map output by the fourth decoder;

[0117] The pre-disaster channel spatial correction feature map, the post-disaster channel spatial correction feature map, and the third-scale feature map are spliced ​​together to obtain the third spliced ​​feature map.

[0118] The third stitched feature map is input into the third decoder to obtain the second scale feature map output by the third decoder;

[0119] The pre-disaster channel spatial correction feature map, the post-disaster channel spatial correction feature map, and the second-scale feature map are spliced ​​together to obtain the second spliced ​​feature map.

[0120] The second concatenated feature map is input into the second decoder to obtain the first scale feature map output by the second decoder;

[0121] The first-scale pre-disaster channel spatial correction feature map, the first-scale post-disaster channel spatial correction feature map, and the first-scale feature map are spliced ​​together to obtain the first spliced ​​feature map.

[0122] The first spliced ​​feature map is input into the first decoder to obtain the change detection map output by the first decoder.

[0123] like Figure 3As shown, the deep-supervised multi-scale feature fusion module with multi-attention constraints outputs the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map at the fourth scale, the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map at the third scale, the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map at the second scale, and the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map at the first scale.

[0124] The fourth-scale pre-disaster channel spatial correction feature map ( ) and the spatial correction feature map of the post-disaster passage at the fourth scale ( Channel splicing is performed to obtain the fourth spliced ​​feature map; the fourth spliced ​​feature map is input into the fourth decoder (decoder_4) to obtain the third scale feature map.

[0125] The third-scale feature map and the third-scale pre-disaster channel spatial correction feature map ( ) and the spatial correction feature map of the post-disaster passage at the third scale ( Channel splicing is performed to obtain the third spliced ​​feature map; the third spliced ​​feature map is input into the third decoder (decoder_3) to obtain the second scale feature map.

[0126] The second-scale feature map and the second-scale pre-disaster channel spatial correction feature map ( ) and the second-scale post-disaster corridor spatial correction feature map ( Channel splicing is performed to obtain the second spliced ​​feature map; the second spliced ​​feature map is input into the third decoder (decoder_2) to obtain the first scale feature map.

[0127] The first-scale feature map and the first-scale pre-disaster channel spatial correction feature map ( ) and the first-scale post-disaster channel spatial correction feature map ( The channels are spliced ​​to obtain the first spliced ​​feature map; the second spliced ​​feature map is input to the third decoder (decoder_1) to obtain the change detection map.

[0128] Input the change detection map into the change detection head to obtain the flood label output by the change detection head, where 0 represents the background, 1 represents normal buildings, 2 represents flooded buildings, 3 represents normal roads, and 4 represents flooded roads.

[0129] In this embodiment of the invention, image features of different scales (including pre-disaster image features and post-disaster image features) are resampled and channel stitched, and then subjected to convolution, batch normalization, corrected linear units, spatial-channel compression activation, convolution, and batch normalization to obtain channel spatial correction feature maps of different scales (including pre-disaster channel spatial correction feature maps and post-disaster channel spatial correction feature maps).

[0130] According to the feature fusion-based deep supervised network-based method for detecting changes in flooded infrastructure provided by the present invention, the decoder structure includes: convolution, first batch normalization, first corrected linear unit, inverse convolution, second batch normalization, and second corrected linear unit.

[0131] like Figure 4 As shown, the decoder structure consists of convolution, inverse convolution, BN (batch normalization), and ReLU (corrected linear unit), which realizes the fusion and supplementation of spatial and semantic features of high-dimensional and low-dimensional multi-scale features.

[0132] The following describes an example of the flooded infrastructure change detection method based on feature fusion deep supervision network provided by the present invention in a practical application scenario.

[0133] Deep learning, with its superior feature extraction capabilities, is widely used in change detection. There are various change detection structures based on deep learning. DL-based change detection frameworks can be categorized into single-branch and dual-branch structures based on their feature extraction strategies. Single-branch structures utilize early fusion to merge temporal images into multispectral images. However, due to the lack of independent original input images, single-branch structures struggle to extract fine temporal features from each bitemporal image, hindering subsequent change detection. Dual-branch structures are conjoined networks consisting of two subnetworks with shared or non-shared weights, employing two bitemporal fusion methods: mid-term fusion and late-term fusion. Each branch independently learns high-level feature representations of a single-temporal image through multiple convolutions. This powerful representation capability enables dual-branch networks to effectively capture change information in complex RS images, thereby improving the accuracy of change detection. To enhance the model's context awareness of single-temporal images, this invention employs a mid-term fusion strategy and embeds a Multi-Scale Feature Fusion (MSFF) module to improve the mid-term fusion strategy, designing the FloodCD change detection structure. Figure 4 As shown, a simplified change detection structure (i.e., a deep supervised multi-scale feature fusion module with multiple attention constraints), encoder-decoder, and fusion method are displayed.

[0134] To balance the number of model parameters and feature extraction performance, this invention introduces ConvNeXt-Tiny as the encoder backbone and adds pre-trained weights to it, constructing a system as follows: Figure 4 The encoder, decoder, and high-dimensional-low-dimensional multi-scale feature fusion module (i.e., a deep-supervised multi-scale feature fusion module with multiple attention constraints) are shown. The encoder has four stages with channels of (192, 384, 768, 1536), and the number of modules is (3, 3, 9, 3). The encoder structure includes depthwise convolution, layernorm, pointwise convolution, GELU, and pointwise convolution. This encoder structure has stronger context awareness and extracts more accurate semantic information. The decoder structure consists of convolution, inverse convolution, BN, and ReLU, realizing the fusion and supplementation of spatial and semantic features of high-dimensional-low-dimensional multi-scale features.

[0135] PSPNet pioneered the pyramid-structured feature fusion approach. In subsequent research, feature fusion has been widely applied to image segmentation, leading to the introduction of Self-Attention Feature Fusion Networks (SA-FFNet) to improve semantic segmentation performance. For example, Multi-Scale Cross-Attention Hierarchical Network (MSCCA-Net) feature fusion, while emphasizing inter-scale fusion, still suffers from the problem of insufficient utilization of high-dimensional and low-dimensional multi-scale features. Even with full-scale jump connections merging all fine-grained spatial and coarse-grained semantic information into the decoder, it cannot be readily applied to various promising deep learning models, heavily relying on the specific structure of the module. To overcome the transfer problem of multi-scale feature fusion modules and fully utilize multi-scale features, this invention constructs a plug-and-play, multi-attention-constrained, deeply supervised high-dimensional and low-dimensional multi-scale feature fusion module (i.e., a multi-attention-constrained, deeply supervised multi-scale feature fusion module), which will significantly improve the detection accuracy of changes in flooded buildings and roads in flood scenes.

[0136] like Figure 5 As shown, scSE is a joint attention mechanism that combines the channel attention and spatial attention mechanisms of the convolutional module. The target-scale feature map is input into the channel attention module to obtain the channel attention feature map output by the channel attention module; the channel attention feature map is multiplied with the target-scale feature map to obtain the channel correction feature map; the target-scale feature map is input into the spatial attention module to obtain the spatial attention feature map output by the spatial attention module; the spatial attention feature map is multiplied with the target-scale feature map to obtain the spatial correction feature map; the channel correction feature map is multiplied with the spatial correction feature map to obtain the pre-disaster channel spatial correction feature map at the target scale.

[0137] Unlike cross-entropy loss, Dice loss, and focal loss, which are used as the main loss function of the network structure, this invention utilizes cross-entropy loss as the main loss function and focal loss as an auxiliary loss function to optimize the hidden layer feature extraction capability. This invention adds deep supervised learning to the hidden layers of the network, directly optimizing the gradient backpropagation process of the hidden layer weights, making them biased towards high-discrimination feature maps. This enhances the flood inundation feature extraction capability of the hidden layers in the network. (Pre-flood and post-flood images and water body masks are also included.) In this invention, the second decoder of FloodCDNet is denoted as... And a deep supervised classifier was added. This introduces deep supervised optimization into the network's encoder and the first two decoders, enabling the model to learn features at different levels and improve the detection results of road changes in flooded buildings. The specific deep supervised classifier can be found in the following formula (1):

[0138] (1)

[0139] in, This represents a deep supervised classifier. For the model In the training set The heatmap generated by the above reasoning, For the second decoder The generated feature map, It is the batch size of the samples in the training set, each These are one-hot tags used to mark data. Upsampling of the feature map, Indicates a pre-disaster image mask. This refers to a post-disaster image mask.

[0140] refer to Figure 3 , Figure 3This paper proposes FloodCDNet as a workflow for detecting changes in flooded buildings and roads from ultra-high resolution remote sensing imagery. FloodCDNet employs a weight-sharing structure that fuses change detection features from two temporal images, learning five categories of change information through an encoder-decoder architecture. This method improves the identification capability of flooded buildings and roads by learning semantic features of buildings and roads in normal state from pre-disaster images and flood coverage information around flooded buildings and roads after the disaster. For the feature extractor, a Siamese network based on ConvNeXt is used. It can extract semantic features of buildings, roads, and floods from two temporal images, obtaining semantic features of flooded buildings and roads with strong context awareness. To enhance the extraction and fusion of semantic features at different levels from two temporal images, this invention constructs a deep-supervised high-dimensional-low-dimensional multi-scale feature fusion module with on-demand multi-attention constraints, thereby obtaining robust global-local semantic representations of infrastructure and floods, significantly improving the accuracy of rapid extraction of flooded buildings and roads. Subsequently, data augmentation strategies and a deep-supervised module for supervising hidden layers are gradually introduced to improve detection accuracy.

[0141] To verify the superiority of the change detection structure adopted in this invention, the early fusion (FC-EF), mid-term fusion (Fc-siam-conc), and subtract (Fc-siam-diff) fusion strategies were reimplemented as F1, F2, and F3, respectively; and a late fusion (Fc-siam-last-conc) strategy F4 was designed. Comparison of MIOU accuracy results for change detection of flooded buildings and roads on the SpaceNet8 dataset shows that Fc-siam-conc (60.56%) outperforms all other structures, exceeding FC-EF and Fc-siam-diff, while Fc-siam-last-conc achieves 14.32%, 4.78%, and 1.31%, respectively. The mid-term conc fusion strategy demonstrates outstanding performance in change detection of flooded buildings and roads.

[0142] refer to Figure 6 , Figure 6 This is a comparison chart of monitoring results for changes in flooded buildings and roads, provided by the present invention, using common change detection structures.

[0143] like Figure 6 As shown, the experimental results for detecting different structural changes are presented. (Buildings and roads are considered simultaneously.) The mid-stage fusion method performs better than the early and late fusion methods in terms of missed detections (red circles) and false detections (orange circles) of flooded buildings. Figure 6 (a) Figure 6(d) shows that mid-stage fusion had fewer missed detections of normal buildings compared to early-stage fusion. Although normal buildings were not as good as late-stage fusion in terms of indicators, the number of missed detections of normal buildings was significantly lower. Figure 6 (c)). In areas where both flooded and normal buildings (roads) coexist, all three fusion methods show high false detection rates (orange circles), and a large number of pixels are missed on normal roads. Figure 6 (b)

[0144] Referring to Table 1, which is a comparison table of the accuracy of monitoring changes in flooded buildings and roads using different change detection structures provided by this invention.

[0145] Table 1

[0146]

[0147] To evaluate the FloodCDNet model for detecting changes in flooded buildings and roads, we constructed models with the same dual-temporal fusion structure, denoted as M2 and M3, using EfficientNet-b5 and ResNeSt50 as backbones. In this study, we implemented FC-siam-conc, ChangeFormer, and SNUNet, denoted as M1, M4, and M5. All models used the same parameters to ensure fair comparison. The table lists the IOU accuracy of different models on the SpaceNet8 dataset. FloodCDNet outperforms other methods in IOU, exceeding FC-siam-conc, EfficientNet-b5, ResNeSt50, ChangeFormer, and SNUNet by 7.32%, 6.73%, 6.16%, 10.02%, and 16.15%, respectively. The models M2, M3, and M6 constructed in this invention all outperform other change detection models. The performance of the transformer-based ChangeFormer is inferior to similar CNN products.

[0148] refer to Figure 2 Table 2 is a comparison table of the monitoring accuracy of different deep learning models for monitoring changes in flooded buildings and roads provided by this invention.

[0149] Table 2

[0150]

[0151] This invention conducted ablation experiments on the SpaceNet8 dataset, including the proposed modular data augmentation, multi-scale feature fusion, and deep supervised learning. Their performance was compared with baseline methods lacking these features. The table lists the accuracy of the flooded building and road change detection results, showing that integrating these modules improves the accuracy of the evaluation metric IoU. Notably, the baseline method reduced the MIOU value by 6.11%. This means that these modules significantly improve the accuracy of flooded building and road change detection. In particular, the addition of the data augmentation strategy had a significant impact on the accuracy of change detection, specifically increasing the MIOU by 4.78%. Furthermore, adding MSFF improved the accuracy of flooded building and road change detection by 1.33%.

[0152] refer to Figure 7 , Figure 7 This is a comparison chart of ablation experiment results provided by the present invention.

[0153] Figure 7 The ablation results are displayed, with B4 identifying the most regular boundary patterns of the changed areas, showing the highest consistency with the actual ground conditions. In particular, the addition of the MSFF module enhances the identification of both normal and flooded buildings. Figure 7 (The orange circles in (a), (b), and (d)). Figure 7 (b) and Figure 7 As shown by the orange and red circles in (c), after adding deep supervision, the flooded roads and roads become more regular, showing fine boundary details.

[0154] Referring to Table 3, which presents the ablation experiment provided by this invention, the accuracy of monitoring changes in flooded buildings and roads by different modules is compared.

[0155] Table 3

[0156]

[0157] To evaluate the plug-and-play simplicity of MSFF and deep supervised learning, and their impact on change detection results, we conducted ablation experiments on M1, M2, and M3. The table lists the evaluation results of Fc-siam-conc, Resnest50, Efficientnet-b5, and Convnext-tiny after adding MSFF and deep supervised learning, showing that detection accuracy is improved when using them.

[0158] refer to Figure 8 , Figure 8 This is a comparison chart of monitoring results of changes in flooded buildings and roads using MSFF embedded with different deep learning models, provided by this invention.

[0159] Figure 8The results of embedding the MFLL module are shown. Compared with other methods, the results obtained using the ConvNeXt feature extractor are more accurate for flooded buildings, flooded roads, and normal buildings and roads. Figure 8 (a) Figure 8 (b) and Figure 8 In the middle (d) section, the false positive and false negative rates are low, and the shapes and contours are clearly defined. This indicates that ConvNeXt, as a feature extractor, exhibits stronger robustness in recognizing various objects. Furthermore, the false negative problem is more prominent on normal roads (d). Figure 8 (c) and Figure 8 (d)

[0160] Referring to Table 4, which is a comparison table of the monitoring accuracy of road changes in flooded buildings by embedding MSFF with different deep learning models provided by this invention.

[0161] Table 4

[0162]

[0163] The following describes the flooded infrastructure change detection device based on feature fusion deep supervision network provided by the present invention. The flooded infrastructure change detection device based on feature fusion deep supervision network described below can be referred to in correspondence with the flooded infrastructure change detection method based on feature fusion deep supervision network described above.

[0164] refer to Figure 9 , Figure 9 This is a schematic diagram of the structure of the flooded infrastructure change detection device based on feature fusion deep supervision network provided by the present invention, which includes: acquisition module 901, pre-disaster coding module 902, post-disaster coding module 903, multi-scale fusion module 904, and decoding module 905.

[0165] The acquisition module 901 is used to acquire pre-disaster images and post-disaster images. The pre-disaster images represent images of the target object before it was affected by the flood, and the post-disaster images represent images of the target object after it was affected by the flood.

[0166] The pre-disaster coding module 902 is used to input pre-disaster images into the pre-disaster encoder to obtain multiple pre-disaster image features of different scales output by the pre-disaster encoder.

[0167] The post-disaster coding module 903 is used to input post-disaster images into the post-disaster encoder to obtain multiple post-disaster image features of different scales output by the post-disaster encoder.

[0168] The multi-scale fusion module 904 is used to input multiple pre-disaster image features of different scales into the multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple pre-disaster channel spatial correction feature maps of different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module.

[0169] The multi-scale fusion module 904 is also used to input multiple post-disaster image features of different scales into the multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple post-disaster channel spatial correction feature maps of different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module.

[0170] Among them, multi-attention constraints include channel attention mechanisms and spatial attention mechanisms;

[0171] The decoding module 905 is used to input the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map into the decoder to obtain the change detection map output by the decoder. The pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map have the same scale, and the change detection map is used to represent the change monitoring comparison results of the target object.

[0172] Specifically, the flooded infrastructure change detection device based on feature fusion deep supervision network provided by the present invention can realize all the method steps implemented in the above-mentioned flooded infrastructure change detection method embodiment based on feature fusion deep supervision network, and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0173] refer to Figure 10 , Figure 10 This is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as... Figure 10As shown, the electronic device may include: a processor 1010, a communication interface 1020, a memory 1030, and a communication bus 1040. The processor 1010, communication interface 1020, and memory 1030 communicate with each other via the communication bus 1040. The processor 1010 can call logical instructions in the memory 1030 to execute a method for detecting changes in flooded infrastructure based on a feature fusion deep supervised network. This method includes: acquiring pre-disaster and post-disaster images, wherein the pre-disaster images represent images of the target object before it suffers from flooding, and the post-disaster images represent images of the target object after it suffers from flooding; inputting the pre-disaster images into a pre-disaster encoder to obtain multiple pre-disaster image features at different scales output by the pre-disaster encoder; inputting the post-disaster images into a post-disaster encoder to obtain multiple post-disaster image features at different scales output by the post-disaster encoder; and inputting the multiple pre-disaster image features at different scales into a multi-attention constrained deep supervised multi-scale feature fusion module to obtain multi-attention constrained deep supervised multi-scale feature fusion. The pre-disaster channel spatial correction feature maps at different scales are output by the force-constrained deep-supervised multi-scale feature fusion module. The post-disaster image features at different scales are then input into the multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain the post-disaster channel spatial correction feature maps at different scales. The multi-attention constraint includes both channel attention and spatial attention mechanisms. The pre-disaster and post-disaster channel spatial correction feature maps are input into the decoder to obtain the change detection map output by the decoder. The pre-disaster and post-disaster channel spatial correction feature maps have the same scale, and the change detection map is used to represent the comparison results of change monitoring of the target object.

[0174] Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0175] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the flooded infrastructure change detection method based on feature fusion deep supervision network provided by the above methods. The method includes: acquiring pre-disaster images and post-disaster images, wherein the pre-disaster images represent images of the target object before it suffers from flooding, and the post-disaster images represent images of the target object after it suffers from flooding; inputting the pre-disaster images into a pre-disaster encoder to obtain multiple pre-disaster image features of different scales output by the pre-disaster encoder; inputting the post-disaster images into a post-disaster encoder to obtain multiple post-disaster image features of different scales output by the post-disaster encoder; and inputting the multiple pre-disaster images of different scales into the post-disaster encoder to obtain multiple post-disaster image features of different scales output by the post-disaster encoder. Image features are input to a multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple pre-disaster channel spatial correction feature maps at different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module. Post-disaster image features at multiple scales are then input to the same module to obtain multiple post-disaster channel spatial correction feature maps at different scales. The multi-attention constraint includes both channel attention and spatial attention mechanisms. The pre-disaster and post-disaster channel spatial correction feature maps are input to a decoder to obtain a change detection map output by the decoder. The pre-disaster and post-disaster channel spatial correction feature maps have the same scale, and the change detection map is used to represent the comparison results of change monitoring of the target object.

[0176] Furthermore, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method for detecting changes in flooded infrastructure based on feature fusion deep supervised networks provided by the methods described above. This method includes: acquiring pre-disaster images and post-disaster images, wherein the pre-disaster images represent images of the target object before it suffers from flooding, and the post-disaster images represent images of the target object after it suffers from flooding; inputting the pre-disaster images into a pre-disaster encoder to obtain multiple pre-disaster image features at different scales output by the pre-disaster encoder; inputting the post-disaster images into a post-disaster encoder to obtain multiple post-disaster image features at different scales output by the post-disaster encoder; and inputting the multiple pre-disaster image features at different scales into a deep supervised network with multiple attention constraints. A supervised multi-scale feature fusion module is used to obtain multiple pre-disaster channel spatial correction feature maps at different scales, output by a deep supervised multi-scale feature fusion module with multi-attention constraints. Post-disaster image features at multiple different scales are then input into the deep supervised multi-scale feature fusion module with multi-attention constraints to obtain multiple post-disaster channel spatial correction feature maps at different scales, output by the deep supervised multi-scale feature fusion module with multi-attention constraints. The multi-attention constraints include channel attention mechanisms and spatial attention mechanisms. The pre-disaster and post-disaster channel spatial correction feature maps are input into a decoder to obtain a change detection map output by the decoder. The pre-disaster and post-disaster channel spatial correction feature maps have the same scale, and the change detection map is used to represent the comparison results of change monitoring of the target object.

[0177] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting changes in flooded infrastructure based on feature fusion deep supervised networks, characterized in that, include: Acquire pre-disaster images and post-disaster images, wherein the pre-disaster images represent images of the target object before it suffers from flooding, and the post-disaster images represent images of the target object after it suffers from flooding; The pre-disaster images are input into the pre-disaster encoder to obtain multiple pre-disaster image features of different scales output by the pre-disaster encoder. The post-disaster images are input into the post-disaster encoder to obtain multiple post-disaster image features of different scales output by the post-disaster encoder. The pre-disaster image features of multiple different scales are input into the deep-supervised multi-scale feature fusion module with multi-attention constraints to obtain the spatial correction feature maps of multiple different scales of the pre-disaster channel output by the deep-supervised multi-scale feature fusion module with multi-attention constraints. The multiple post-disaster image features at different scales are input into the multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple post-disaster channel spatial correction feature maps at different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module. The multi-attention constraint includes channel attention mechanism and spatial attention mechanism; The pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map are input into the decoder to obtain the change detection map output by the decoder. The pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map have the same scale. The change detection map is used to represent the change monitoring comparison results of the target object. The process involves inputting the pre-disaster image features at multiple different scales into a multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple pre-disaster channel spatial correction feature maps at different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module, including: Perform the following operations on each of the multiple different scales as the target scale to obtain the pre-disaster channel spatial correction feature map at the target scale: Upsampling is performed on the pre-disaster image features at multiple different scales to obtain pre-disaster image features at multiple target scales with different numbers of channels. Channel stitching is performed on the pre-disaster image features of multiple target scales with different numbers of channels to obtain a target scale feature map; The target scale feature map is input into the channel attention module to obtain the channel attention feature map output by the channel attention module; Multiply the channel attention feature map with the target scale feature map to obtain the channel correction feature map; The target scale feature map is input into the spatial attention module to obtain the spatial attention feature map output by the spatial attention module; Multiply the spatial attention feature map with the target scale feature map to obtain the spatial correction feature map; Multiply the channel correction feature map with the spatial correction feature map to obtain the pre-disaster channel spatial correction feature map at the target scale.

2. The method for detecting changes in flooded infrastructure based on feature fusion deep supervision networks according to claim 1, characterized in that, The pre-disaster encoder includes: a first pre-disaster encoder, a second pre-disaster encoder, a third pre-disaster encoder, and a fourth pre-disaster encoder; the process of inputting the pre-disaster image into the pre-disaster encoder to obtain multiple pre-disaster image features of different scales output by the pre-disaster encoder includes: The pre-disaster image is input into the first pre-disaster encoder to obtain the first pre-disaster image feature output by the first pre-disaster encoder. The scale of the first pre-disaster image feature is one-quarter of the pre-disaster image, and the number of channels of the first pre-disaster image feature is 1. The first pre-disaster image feature is input into the second pre-disaster encoder to obtain the second pre-disaster image feature output by the second pre-disaster encoder. The scale of the second pre-disaster image feature is one-eighth of the pre-disaster image, and the number of channels of the second pre-disaster image feature is 2. The second pre-disaster image feature is input into the third pre-disaster encoder to obtain the third pre-disaster image feature output by the third pre-disaster encoder. The scale of the third pre-disaster image feature is one-sixteenth of the pre-disaster image, and the number of channels of the third pre-disaster image feature is 4. The third pre-disaster image feature is input into the fourth pre-disaster encoder to obtain the fourth pre-disaster image feature output by the fourth pre-disaster encoder. The scale of the fourth pre-disaster image feature is one thirty-second of the pre-disaster image, and the number of channels of the fourth pre-disaster image feature is 8.

3. The method for detecting changes in flooded infrastructure based on feature fusion deep supervision networks according to claim 2, characterized in that, The number of channels in the pre-disaster encoder is 192, 384, 768, and 1536, respectively; the number of modules in the pre-disaster encoder is 3, 3, 9, and 3, respectively; the structure of the pre-disaster encoder includes: depthwise convolution, layer normalization, first pointwise convolution, Gaussian error linear unit, and second pointwise convolution.

4. The method for detecting changes in flooded infrastructure based on feature fusion deep supervision networks according to claim 1, characterized in that, The decoder includes a first decoder, a second decoder, a third decoder, and a fourth decoder. The step of inputting the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map into the decoder to obtain the change detection map output by the decoder includes: The pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map at the fourth scale are stitched together to obtain the fourth stitched feature map. The fourth stitched feature map is input into the fourth decoder to obtain the third scale feature map output by the fourth decoder; The pre-disaster channel spatial correction feature map, the post-disaster channel spatial correction feature map, and the third-scale feature map are spliced ​​together to obtain the third spliced ​​feature map. The third stitched feature map is input into the third decoder to obtain the second scale feature map output by the third decoder; The pre-disaster channel spatial correction feature map, the post-disaster channel spatial correction feature map, and the second-scale feature map are spliced ​​together to obtain the second spliced ​​feature map. The second stitched feature map is input into the second decoder to obtain the first scale feature map output by the second decoder; The pre-disaster channel spatial correction feature map, the post-disaster channel spatial correction feature map, and the first-scale feature map are spliced ​​together to obtain the first spliced ​​feature map. The first spliced ​​feature map is input into the first decoder to obtain the change detection map output by the first decoder.

5. The method for detecting changes in flooded infrastructure based on feature fusion deep supervision networks according to claim 4, characterized in that, The decoder structure includes: convolution, first batch normalization, first modified linear unit, inverse convolution, second batch normalization, and second modified linear unit.

6. A device for detecting changes in flooded infrastructure based on feature fusion deep supervision networks, characterized in that, include: The acquisition module is used to acquire pre-disaster images and post-disaster images, wherein the pre-disaster images represent images of the target object before it suffers from flooding, and the post-disaster images represent images of the target object after it suffers from flooding. The pre-disaster encoding module is used to input the pre-disaster image into the pre-disaster encoder to obtain multiple pre-disaster image features of different scales output by the pre-disaster encoder. The post-disaster encoding module is used to input the post-disaster images into the post-disaster encoder to obtain multiple post-disaster image features of different scales output by the post-disaster encoder. A multi-scale fusion module is used to input the pre-disaster image features of multiple different scales into a multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple pre-disaster channel spatial correction feature maps of different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module. The multi-scale fusion module is also used to input the multiple post-disaster image features of different scales into the multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple post-disaster channel spatial correction feature maps of different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module. The multi-attention constraint includes channel attention mechanism and spatial attention mechanism; The decoding module is used to input the pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map into the decoder to obtain the change detection map output by the decoder. The pre-disaster channel spatial correction feature map and the post-disaster channel spatial correction feature map have the same scale. The change detection map is used to represent the change monitoring comparison results of the target object. The process involves inputting the pre-disaster image features at multiple different scales into a multi-attention-constrained deep-supervised multi-scale feature fusion module to obtain multiple pre-disaster channel spatial correction feature maps at different scales output by the multi-attention-constrained deep-supervised multi-scale feature fusion module, including: Perform the following operations on each of the multiple different scales as the target scale to obtain the pre-disaster channel spatial correction feature map at the target scale: Upsampling is performed on the pre-disaster image features at multiple different scales to obtain pre-disaster image features at multiple target scales with different numbers of channels. Channel stitching is performed on the pre-disaster image features of multiple target scales with different numbers of channels to obtain a target scale feature map; The target scale feature map is input into the channel attention module to obtain the channel attention feature map output by the channel attention module; Multiply the channel attention feature map with the target scale feature map to obtain the channel correction feature map; The target scale feature map is input into the spatial attention module to obtain the spatial attention feature map output by the spatial attention module; Multiply the spatial attention feature map with the target scale feature map to obtain the spatial correction feature map; Multiply the channel correction feature map with the spatial correction feature map to obtain the pre-disaster channel spatial correction feature map at the target scale.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the flooded infrastructure change detection method based on feature fusion deep supervision network as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the flooded infrastructure change detection method based on feature fusion deep supervision network as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the flooded infrastructure change detection method based on feature fusion deep supervision network as described in any one of claims 1 to 5.

Images