Method and device for extracting flood inundated area based on double-time-phase remote sensing image

Abstract

The application provides a flood inundated area extraction method based on double-time-phase remote sensing images, comprising the following steps: obtaining a pre-time-phase remote sensing image and a post-time-phase remote sensing image of a known flood inundated area to construct a model training data set; constructing an extraction model comprising a feature alignment model and an inundated area extraction model, and training the extraction model by using the training set; and using the trained extraction model to extract a flood inundated area of a target area. The application is aimed at the problem that the traditional flood inundated area extraction method is easily affected by sensors, light and water body silt, and the extraction accuracy is poor. The application uses a generative adversarial network to reduce the image features affected by sensors and light, adds a gate feature fusion module GCFM at a flood decoding end to adaptively identify complex water body features, and eliminates the missing detection problem caused by water body silt, thereby ensuring the extraction accuracy and reliability of the flood inundated area, and significantly improving the automation level of the flood inundated area extraction.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a method and apparatus for extracting flood-inundated areas based on dual-temporal satellite remote sensing images. Background Technology

[0002] Quickly ascertaining the extent of flood damage is of great practical significance for carrying out emergency rescue, loss assessment, post-disaster reconstruction, and comprehensive watershed management.

[0003] With the rapid development of science and technology, especially the advancement of remote sensing technology, new perspectives have been provided for the monitoring and assessment of flood disasters. Remote sensing technology, with its advantages of wide coverage, short monitoring cycles, and high image resolution, plays a crucial role in natural disaster monitoring, early warning, and assessment. High-resolution optical remote sensing imagery is an important data source for flood disaster monitoring and assessment. However, since floods are often followed by overcast and rainy weather, it is often necessary to use images from different miniature sensors to collect data on the affected areas. Due to differences in satellite sensors and the time of image acquisition, there are significant differences in image indicators such as brightness, color, and saturation between pre-disaster and post-disaster images, posing a significant challenge to the automated extraction of flood-inundated areas. Furthermore, suspended sediment in the water after a disaster can alter the spectral characteristics of the water body, thus affecting the effectiveness of traditional pixel-feature-based water body information extraction methods.

[0004] Traditional methods for extracting flood inundation areas use water body indices, machine learning, or deep learning algorithms to extract water body areas from pre-disaster and post-disaster images, and then use GIS analysis technology to extract the extent of the inundation area. However, differences in image quality caused by sensors and weather, as well as differences in water body characteristics caused by sediment in the post-disaster water, pose significant challenges to the extraction of the flood inundation area. Summary of the Invention

[0005] To address the aforementioned problems, this invention proposes a flood inundation area extraction method based on dual-temporal remote sensing imagery. The method includes: acquiring pre-temporal and post-temporal remote sensing images of a known flood inundation area as training data and constructing a training dataset; constructing an extraction model and training it using the training dataset; the extraction model includes a feature alignment model and an inundation area extraction model. The feature alignment model is a generative adversarial network model with a two-branch residual convolutional neural network structure, possessing two identical backbone networks to process the pre-temporal and post-temporal remote sensing images in parallel; the feature alignment model and the inundation area extraction model share backbone network weights, and the backbone network parameters are updated synchronously during model training; and using the trained extraction model to extract the flood inundation area of ​​the target region.

[0006] Furthermore, the feature alignment model also includes a decoder, a generator, and a discriminator; the decoder includes parallel DUA and DUB modules, each consisting of a three-layer upsampling convolutional module; the generator includes GAN_A, composed of a backbone network BackboneA and a DUA module, and GAN_B, composed of a backbone network BackboneB and a DUB module; the discriminator includes discriminator modules Dis_A and Dis_B, each comprising two downsampling modules and a two-dimensional convolutional network; the steps for training this extraction model include:

[0007] Image features FA of the previous time-phase remote sensing image TA are extracted through the backbone network BackboneA, and image features FB of the subsequent time-phase remote sensing image TB are extracted through the backbone network BackboneB.

[0008] The DUA module restores the original image size and number of channels from the image features FA to generate the reconstructed image SameA, and the DUB module restores the original image size and number of channels from the image features FB to generate the reconstructed image SameB.

[0009] Using the subsequent time-phase remote sensing image TB as input, the generator GAN_A generates the pseudo image FakeA; using the previous time-phase remote sensing image TA as input, the generator GAN_B generates the pseudo image FakeB.

[0010] The generator GAN_A takes the pseudo image FakeA as input to generate the cyclic reconstructed image RecoverA, and the generator GAN_B takes the pseudo image FakeB as input to generate the cyclic reconstructed image RecoverB. The weight sharing network is used to align the features of the cyclic reconstructed images RecoverA and RecoverB.

[0011] Using loss function L GAN总 The generator, L, is used to train this feature alignment model. GAN总 = L1 A + L1 B + L GAN_A +L GAN_B + L consistent_A + L consistent_B ;

[0012] , representing the loss between Real_A and the reconstructed image SameA, where Real_A is the real image input to the generator GAN_A;

[0013] , representing the loss between Real_B and the reconstructed image SameB, where Real_B is the real image input to the generator GAN_B;

[0014] , represents the loss of the generator GAN_A attempting to deceive the discrimination module Dis_A, where N is the number of images involved in the calculation;

[0015] This indicates the loss of the generator GAN_B attempting to deceive the authentication module Dis_B;

[0016] This represents the consistency loss between the cyclically reconstructed images RecoverA and Real_A;

[0017] This represents the consistency loss between the cyclically reconstructed images RecoverB and Real_B;

[0018] The flooded area extraction model is constructed using backbone networks BackboneA and BackboneB, as well as the gated feature fusion module GCFM and the gated convolution module GCM.

[0019] Furthermore, the joint loss function L is used to train and optimize the extraction model. ; , This extracts the two-channel features from the final output of the flood extraction model. , This indicates that the auxiliary classifier affects the generated samples. The predicted probability; Tags indicating auxiliary tasks; Indicates the number of samples

[0020] Preferably, the accuracy of the trained extraction model is evaluated using the test set of the training dataset; wherein, the harmonic index F1-score of the extraction model is obtained, and the extraction model with the largest harmonic index F1-score is the final extraction model.

[0021]

[0022] Wherein, Precision represents the proportion of pixels that the model predicts to be flooded areas but are actually flooded areas. Recall represents the proportion of pixels in the actual flooded area that were correctly predicted. TP represents correctly predicted flooded area pixels, FP represents incorrectly predicted flooded area pixels, and FN represents missed flooded area pixels.

[0023] This invention also proposes a flood inundation area extraction device based on dual-temporal remote sensing imagery, comprising: a dataset construction module for acquiring pre-temporal and post-temporal remote sensing images of a known flood inundation area as training data and constructing a training dataset; a model training module for constructing an extraction model and training the extraction model using the training dataset; the extraction model includes a parallel feature alignment model and an inundation area extraction model, wherein the feature alignment model is a generative adversarial network model with a two-branch residual convolutional neural network structure, having two identical backbone networks to process the pre-temporal and post-temporal remote sensing images in parallel; the feature alignment model and the inundation area extraction model share backbone network weights, and the backbone network parameters are updated synchronously during model training; and an extraction module for extracting the flood inundation area of ​​the target area using the trained extraction model.

[0024] Furthermore, the model training module includes a feature alignment model construction module, used to construct the feature alignment model using a generative adversarial network model with a dual-branch residual convolutional neural network structure. This module includes a decoder, generator, and discriminator for constructing the feature alignment model. The decoder comprises parallel DUA and DUB modules, each consisting of three upsampling convolutional layers. The generator includes GAN_A, composed of a backbone network BackboneA and a DUA module, and GAN_B, composed of a backbone network BackboneB and a DUB module. The discriminator includes discriminator modules Dis_A and Dis_B, each comprising two downsampling modules and a two-dimensional convolutional network.

[0025] The feature alignment model training module is used to train the model with a loss function L. GAN总 Training the feature alignment model includes: extracting image features FA from the preceding time-phase remote sensing image TA through the backbone network BackboneA, and extracting image features FB from the following time-phase remote sensing image TB through the backbone network BackboneB; restoring the original image size and number of channels from the image features FA using the DUA module to generate the reconstructed image SameA, and restoring the original image size and number of channels from the image features FB using the DUB module to generate the reconstructed image SameB; generating a pseudo-image FakeA using the following time-phase remote sensing image TB as input from the generator GAN_A, and generating a pseudo-image FakeB using the preceding time-phase remote sensing image TA as input from the generator GAN_B; generating a cyclic reconstructed image RecoverA using the pseudo-image FakeA as input from the generator GAN_A, and generating a cyclic reconstructed image RecoverB using the pseudo-image FakeB as input from the generator GAN_B; and performing feature alignment between the cyclic reconstructed images RecoverA and RecoverB using a weight sharing network.

[0026] Using loss function L GAN总 The generator, L, is used to train this feature alignment model. GAN总 = L1 A + L1 B + L GAN_A +L GAN_B + L consistent_A + L consistent_B ;

[0027] , representing the loss between Real_A and the reconstructed image SameA, where Real_A is the real image input to the generator GAN_A;

[0028] , representing the loss between Real_B and the reconstructed image SameB, where Real_B is the real image input to the generator GAN_B;

[0029] , represents the loss of the generator GAN_A attempting to deceive the discrimination module Dis_A, where N is the number of images involved in the calculation;

[0030] This indicates the loss of the generator GAN_B attempting to deceive the authentication module Dis_B;

[0031] This represents the consistency loss between the cyclically reconstructed images RecoverA and Real_A;

[0032] This represents the consistency loss between the cyclically reconstructed images RecoverB and Real_B;

[0033] The flooded area extraction model building module is used to construct the flooded area extraction model using backbone networks BackboneA and BackboneB, as well as the gated feature fusion module GCFM and the gated convolution module GCM.

[0034] Furthermore, the model training module also includes an optimization module, used to train and optimize the extraction model using the joint loss function L. Includes: a flooded area extraction model optimization module, used to employ the cross-entropy loss function CE. W Train and optimize the flooded area extraction model. , This extracts two-channel features from the final output of the flood model; the feature alignment model optimization module is used to apply the cross-entropy loss function CE. G Train and optimize the generator of this feature alignment model. , This indicates that the auxiliary classifier affects the generated samples. The predicted probability; Tags indicating auxiliary tasks; Indicates the number of samples.

[0035] Preferably, it also includes: a model evaluation module, used to evaluate the accuracy of the trained extraction model using the test set of the training dataset; wherein, the harmonic index F1-score of the extraction model is obtained, and the extraction model with the largest harmonic index F1-score is the final extraction model.

[0036] Precision represents the proportion of pixels that the model predicts to be flooded areas, but which are actually flooded areas. Recall represents the proportion of pixels in the actual flooded area that were correctly predicted. TP represents correctly predicted flooded area pixels, FP represents incorrectly predicted flooded area pixels, and FN represents missed flooded area pixels.

[0037] The present invention also proposes a computer-readable storage medium storing computer-executable instructions, characterized in that, when the computer-executable instructions are executed, the flood inundation area extraction method based on dual-temporal remote sensing images described above is implemented.

[0038] The present invention also proposes a computer program product, including a computer program that, when executed by a processor, implements the flood inundation area extraction method based on dual-temporal remote sensing imagery as described above.

[0039] This invention addresses the problem that traditional flood inundation area extraction methods are easily affected by sensor interference, lighting, and water sediment, resulting in poor extraction accuracy. It utilizes generative adversarial networks to reduce the influence of sensor and lighting on image features, and adds a gated feature fusion module (GCFM) to the flood decoding end to adaptively identify complex water features, eliminating the problem of missed detection caused by water sediment. While ensuring the extraction accuracy and reliability of flood inundation areas, it significantly improves the automation level of flood inundation area extraction. Attached Figure Description

[0040] Figure 1 This is a flowchart of the flood inundation area extraction method based on dual-temporal remote sensing imagery of the present invention.

[0041] Figure 2 This is a schematic diagram of the training network architecture for the flood inundation area extraction model of the present invention.

[0042] Figure 3 This is a flowchart of the extraction model training process of the present invention.

[0043] Figure 4 This is a schematic diagram of the GCFM module of the present invention.

[0044] Figure 5 This is a schematic diagram of the GCM module of the present invention.

[0045] Figure 6 This is a schematic diagram of the flood inundation area extraction process of the present invention.

[0046] Figure 7 This is a schematic diagram of the flood inundation area extraction device based on dual-temporal remote sensing images of the present invention.

[0047] Figure 8 This is a schematic diagram of the model training module of the present invention.

[0048] Figure 9 This is a schematic diagram of an electronic device according to the present invention.

[0049] Figure 10 This is a schematic diagram of the hardware structure of an electronic device according to the present invention.

[0050] The attached figures are labeled as follows:

[0051] 10: Flood-Inundated Area Extraction Device 11: Dataset Construction Module

[0052] 12: Model Training Module 121: Feature Alignment Model Building Module

[0053] 122: Feature Alignment Model Training Module; 123: Flooded Area Extraction Model Construction Module

[0054] 124: Optimization Module 1241: Optimization Module for Flooded Area Extraction Model

[0055] 1242: Feature Alignment Model Optimization Module; 125: Model Evaluation Module

[0056] 13: Extraction Module

[0057] S1, S2, S21, S22, S23, S24, S25, S26, S3: Steps Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0059] It should be noted that, in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0060] In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0061] This invention addresses the problem that traditional flood inundation area extraction methods are easily affected by sensor interference, lighting, and water sediment, leading to poor extraction accuracy. It proposes an independent dual-branch feature alignment method for extracting flood inundation areas from dual-temporal remote sensing images. By utilizing generative adversarial networks to reduce the influence of sensor and lighting on image features, a gated feature fusion module (GCFM) is added to the flood decoding end to adaptively identify complex water features and eliminate the problem of missed detection caused by water sediment. While ensuring the accuracy and reliability of flood inundation area extraction, this method significantly improves the automation level of flood inundation area extraction.

[0062] This invention designs a flood inundation area extraction model based on independent bi-branch change detection technology. It adopts a generative adversarial network and a change detection network to share backbone network model parameters to achieve feature alignment between pre-disaster and post-disaster two-temporal remote sensing images. It proposes a method for extracting flood inundation areas from two-temporal remote sensing images with independent bi-branch feature alignment.

[0063] Figure 1 This is a flowchart of the flood inundation area extraction method based on dual-temporal remote sensing imagery of the present invention. Figure 1 As shown, in the first embodiment of the present invention, a method for extracting flood-inundated areas is proposed, comprising:

[0064] Step S1: Create the training sample dataset

[0065] We acquired the pre-flood remote sensing images (TA) and post-flood remote sensing images (TB) of known flood-affected areas. We then manually interpreted the two sets of remote sensing images before and after the flood disaster. We used vectors to delineate the flood-inundated areas, converted the vector labels to raster labels, and used the open-source Python module GDAL to segment the images and labels. Finally, we randomly divided a dataset into training, validation, and test sets in a 7:1:2 ratio.

[0066] The preceding time-phase remote sensing image (TA) and the following time-phase remote sensing image (TB) are the raw data input into the model. They represent remote sensing images of the same area at different time points and are used to extract flood-inundated areas.

[0067] Step S2: Construct the extraction model and train the model based on the training sample dataset.

[0068] Figure 2 This is a schematic diagram of the training network architecture for the flood inundation area extraction model of the present invention. Figure 2 As shown, the extraction model of this invention includes two parallel network models: a feature alignment model based on generative adversarial model and a flood-inundated area extraction model based on change detection. During training, the two models share the backbone network weights and simultaneously update the backbone network parameters. The training process of the extraction model is as follows: Figure 3 As shown, it includes:

[0069] Step S21, Generate network module

[0070] The generative adversarial network (GAN) model is a dual-branch residual convolutional neural network (RBN) architecture, using ResNet50 as the backbone network. The backbone networks are named BackboneA and BackboneB, each containing four stages. The GAN model only uses the output features from Stage 4 as input to the decoder. The decoder network consists of DUA and DUB modules, each composed of three upsampling convolutional layers. Each upsampling convolutional layer consists of a transposed convolution with a kernel size of 3 and a stride of 2, and a ReLU activation function. The generator GAN_A is composed of BackboneA and DUA, and the generator GAN_B is composed of BackboneB and DUB. The discriminator modules Dis_A and Dis_B consist of two downsampling modules and a two-dimensional convolution. The downsampling module consists of a two-dimensional convolution with a kernel size of 3*3 and a stride of 2, and a ReLU activation function. The two-dimensional convolution module has 128 input channels and 1 output channel, with a kernel size of 3*3.

[0071] Step S22, Training the generator

[0072] The loss function uses L1 loss to train the generator of the model. Real_A is input into GAN_A to get sameA, and RealB is input into GAN_B to get sameB. The L1 loss of GAN_A and GAN_B is calculated by formulas (1) and (2) respectively. A L1 B Among them, L1 A Used to calculate the loss between Real_A and SameA, L1 B Used to calculate the loss between Real_B and SameB.

[0073] (1)

[0074] (2)

[0075] in, _A、 _B represents the real image input to the generator. A, B is the pseudo-image generated by the generator, and n is the number of pixels in the image.

[0076] The adversarial loss function is calculated using the Generative Network (GANB) to generate Feak_B from the TB temporal image based on Real_A, which is used to deceive the discriminator Dis_B. A binary cross-entropy loss function is used to measure the difference between the discriminator and the true label. Here, the adversarial loss L is calculated using 1 and the output of Dis_B. GAN_A To minimize the generator loss, the discriminator Dis_B is made to misclassify its generated fake image as a real image; the reverse process causes dis_A to misclassify the fake image Feak_A generated by the generator GANA based on RealB as a real image. The loss function is expressed as L. GAN_B The results are obtained by calculation using formulas (3) and (4):

[0077] (3)

[0078] (4)

[0079] The consistency loss function uses GenA to generate RecoverA based on FeakB, and uses GenB to generate RecoverB based on FeakA. The L1 loss function is then used to calculate the consistent row loss between RecoverA and Real_A, and between RecoverB and Real_B. , .

[0080] (5)

[0081] (6)

[0082] The generator loss is shown in equation (7):

[0083] L GAN总 = L1 A + L1 B + L GAN_A + L GAN_B + L consistent_A + L consistent_B (7)

[0084] Step S23, train the discriminator

[0085] During training, the discriminator uses discriminators Dis_A and Dis_B to process images from time TA and time TB, respectively. First, the discriminators calculate the output of the real images (real_A and real_B) respectively, and then use the cross-entropy loss function to calculate the loss, so that the discriminator classifies the real images as real, that is, the output of the discriminator is close to 1. Then, the discriminators calculate the output of the fake images FeakA and FeakB, so that the discriminator classifies these images as fake, that is, the output of the discriminator is close to 0.

[0086] The above steps train the feature alignment model. In the specific implementation, firstly, the 3*512*512 time-phase remote sensing image of the previous time phase (TA) is input into the GANA network. It passes through BackboneA to obtain a 1024*64*64 output feature, which is then input into the DUA module to recover the original 3*512*512 image, outputting the SameA image. The TB time-phase image is input into the GANB module for processing, outputting the SameB image. Then, the TB image is input into GANA to generate a fake image FakeA based on the TB time-phase image, which is then identified as genuine using the Dis_A discrimination module. The TB time-phase fake image FakeB, generated by GANB based on the TA time-phase image, is also identified as genuine using the Dis_B discrimination module. FakeA is input into BackboneA to obtain RecoverA, and FakeB is input into BackboneB to obtain RecoverB. The dual-time-phase image features are then optimized using a weight-sharing network. The aforementioned generative adversarial network is used to align the TA and TB image features, improving the network's ability to extract high-quality semantic features.

[0087] Step S24: Train the flood-inundated area extraction model

[0088] The flood-inundated area extraction model uses the ResNet50 branches of the generative model GAN, BankboneA and BackboneB, to output Stage4 features FA and FB from the previous time-phase image A and the subsequent time-phase image B, respectively. The features FA and FB are used as a gated feature fusion module GCFM to fuse the features and output the fused features. Then, convolution and upsampling are used to restore the original image size step by step to obtain the network's output feature Final_Feat.

[0089] like Figure 4 , 5As shown, the gated feature fusion module GCFM is a three-branch network structure. Branch 1 merges the FA and FB features and performs convolution and upsampling, reducing the number of channels in the output feature to half of the original. Branches 2 and 3 process the FA and FB features respectively, through two stacked modules of 3×3 convolution and ReLU activation function, and then perform upsampling. After processing, the processed features are input into the gated convolution module GCM, and the processed features are output. Finally, the output features of the three branches are merged, and the original image size is restored step by step to output the network feature Final_Feat.

[0090] Finally, the output features of the flood-inundated extraction model, Final_Feat, are input into a convolutional layer with 64 input channels and 2 output channels to classify the image pixels.

[0091] Step S25: Use the joint loss function L to train and optimize the extraction model.

[0092] (8)

[0093] Among them, the flood inundation extraction model adopts the cross-entropy loss function (CE).

[0094] (9)

[0095] This refers to the two-channel features that are the final output of the flood extraction model.

[0096] It is an auxiliary cross-entropy loss for the generator, used to help train the generator so that the images it generates can better fool the discriminator.

[0097] (10)

[0098] It is an auxiliary classifier for generating samples The predicted probability; It is the label for auxiliary tasks (1 or 0); It refers to the number of samples.

[0099] Step S26, Model Accuracy Evaluation

[0100] The model accuracy was verified on the test set, and the optimal model was selected for flood inundation area extraction. The F1-score was used as the accuracy evaluation metric for the model. The F1-score is a harmonic metric combining precision and recall. Precision represents the proportion of pixels predicted as flooded areas that are actually flooded areas, while recall represents the proportion of pixels in actual flooded areas that were correctly predicted. The formula for calculating the F1-score is shown below.

[0101] (11)

[0102] (12)

[0103] (13)

[0104] TP represents correctly predicted pixels, FP represents incorrectly predicted pixels, and FN represents undetected flooded pixels.

[0105] The extraction model that maximizes the harmonic index F1-score is taken as the final extraction model.

[0106] Step S3: Extraction of the flooded area

[0107] Figure 6 This is a schematic diagram of the flood-inundated area extraction process of the present invention. (See diagram below.) Figure 6 As shown, during flood extraction, dual-temporal high-resolution remote sensing images are input into the change detection network in the dual-temporal remote sensing image flood inundation area extraction method with independent dual-branch feature alignment to extract the flood inundation area of ​​the high-resolution image.

[0108] It should be noted that, in various embodiments of the present invention, the order of the steps does not imply the order of execution. The execution order of each step should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0109] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0110] In a second embodiment of the present invention, a device for extracting flood-inundated areas based on dual-temporal remote sensing imagery is proposed, such as... Figure 7 As shown, the flood-inundated area extraction device 10 of the present invention includes:

[0111] The dataset construction module 11 is used to acquire the pre-flood remote sensing image TA and post-flood remote sensing image TB of the known flood-inundated area as training data and construct the training dataset. The pre-flood remote sensing image TA and post-flood remote sensing image TB are the original data input into the model, which represent remote sensing images of the same area at different time points, and are used to extract the flood-inundated area.

[0112] We acquired the pre-flood remote sensing images (TA) and post-flood remote sensing images (TB) of known flood-affected areas. We then manually interpreted the two sets of remote sensing images before and after the flood disaster. We used vectors to delineate the flood-inundated areas, converted the vector labels to raster labels, and used the open-source Python GDAL module to segment the images and labels. Finally, we randomly divided a training dataset into training, validation, and test sets in a 7:1:2 ratio.

[0113] Model training module 12 is used to construct the extraction model and train it using a training dataset. The extraction model includes a parallel feature alignment model and a flooded area extraction model. The feature alignment model is a generative adversarial network (GAN) model with a two-branch residual convolutional neural network structure, featuring two identical backbone networks to process pre- and post-temporal remote sensing images in parallel. The feature alignment model and the flooded area extraction model share backbone network weights, and the backbone network parameters are updated synchronously during model training. Figure 8 As shown, it includes:

[0114] The feature alignment model construction module 121 is used to construct a feature alignment model using a generative adversarial network model with a dual-branch residual convolutional neural network structure. It includes a decoder, a generator, and a discriminator for constructing the feature alignment model. The decoder comprises parallel DUA and DUB modules, each consisting of three upsampling convolutional layers. The generator includes a generator GAN_A composed of a backbone network BackboneA and a DUA module, and a generator GAN_B composed of a backbone network BackboneB and a DUB module. The discriminator includes a discriminator module Dis_A and a discriminator module Dis_B, each comprising two downsampling modules and a two-dimensional convolutional network.

[0115] Feature alignment model training module 122, used to train the model with loss function L GAN总The training of the feature alignment model includes: extracting image features FA from the previous time-phase remote sensing image TA through the backbone network BackboneA, and extracting image features FB from the subsequent time-phase remote sensing image TB through the backbone network BackboneB; restoring the original image size and number of channels from the image features FA using the DUA module to generate the reconstructed image SameA, and restoring the original image size and number of channels from the image features FB using the DUB module to generate the reconstructed image SameB; generating a pseudo-image FakeA using the subsequent time-phase remote sensing image TB as input from the generator GAN_A, and generating a pseudo-image FakeB using the previous time-phase remote sensing image TA as input from the generator GAN_B; generating a cyclic reconstructed image RecoverA using the pseudo-image FakeA as input from the generator GAN_A, and generating a cyclic reconstructed image RecoverB using the pseudo-image FakeB as input from the generator GAN_B; and performing feature alignment between the cyclic reconstructed images RecoverA and RecoverB using a weight sharing network.

[0116] L GAN总 = L1 A + L1 B + L GAN_A + L GAN_B + L consistent_A + L consistent_B ;

[0117] , representing the loss between Real_A and the reconstructed image SameA, where Real_A is the real image input to the generator GAN_A;

[0118] , representing the loss between Real_B and the reconstructed image SameB, where Real_B is the real image input to the generator GAN_B;

[0119] , represents the loss of the generator GAN_A attempting to deceive the discrimination module Dis_A, where N is the number of images involved in the calculation;

[0120] This indicates the loss of the generator GAN_B attempting to deceive the authentication module Dis_B;

[0121] This represents the consistency loss between the cyclically reconstructed images RecoverA and Real_A;

[0122] This represents the consistency loss between the cyclically reconstructed images RecoverB and Real_B;

[0123] The flooded area extraction model construction module 123 is used to construct the flooded area extraction model using the backbone network BackboneA and BackboneB, as well as the gated feature fusion module GCFM and the gated convolution module GCM.

[0124] Optimization module 124 is used to train and optimize the extraction model using the joint loss function L. ;include:

[0125] The flooded area extraction model optimization module 1241 is used to employ the cross-entropy loss function CE. W Train and optimize the flooded area extraction model. , This extracts the two-channel features from the final output of the flood extraction model.

[0126] Feature alignment model optimization module 1242 is used to employ the cross-entropy loss function CE G Train and optimize the generator of the feature alignment model. , This indicates that the auxiliary classifier affects the generated samples. The predicted probability; Tags indicating auxiliary tasks; Indicates the number of samples.

[0127] The model evaluation module 125 is used to evaluate the accuracy of the trained extraction model using the test set of the training dataset; wherein, the harmonic index F1-score of the extraction model is obtained, and the extraction model with the largest harmonic index F1-score is the final extraction model.

[0128] Precision represents the proportion of pixels that the model predicts to be flooded areas, but which are actually flooded areas. Recall represents the proportion of pixels in the actual flooded area that were correctly predicted. TP represents correctly predicted flooded area pixels, FP represents incorrectly predicted flooded area pixels, and FN represents missed flooded area pixels.

[0129] Extraction module 13 is used to extract the flood-inundated area of ​​the target area using the trained extraction model.

[0130] In a third embodiment of the present invention, a computer-readable storage medium is provided. The flood inundation area extraction device based on dual-temporal remote sensing imagery of the present invention, if its function is implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. Therefore, in the third embodiment of the present invention, a computer-readable storage medium is provided for storing a computer program for a flood inundation area extraction method based on dual-temporal remote sensing imagery. It should be understood that the computer-readable storage medium in the embodiments of the present invention can be volatile memory and / or non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0131] Figure 9 This is a schematic diagram of an electronic device according to the present invention. Figure 9As shown, in the fourth embodiment of the present invention, an electronic device 100 is proposed, including the flood-inundated area extraction device 10 based on dual-temporal remote sensing imagery as described above. Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a program instructing related hardware (e.g., processor, FPGA, ASIC, etc.). All or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module in the above embodiments can be implemented in hardware, for example, by implementing its corresponding function through integrated circuits, or it can be implemented as a software functional module, for example, by a processor executing a program / instruction stored in memory to implement its corresponding function. The embodiments of the present invention are not limited to any particular combination of hardware and software.

[0132] It should be noted that the structure of the electronic device shown in the accompanying drawings of this invention does not constitute a limitation thereof. The actual knowledge structure recognition device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0133] The electronic device of the present invention can be any device with data processing capabilities, such as a computer or other similar device. The device embodiment can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. Figure 10 This is a schematic diagram of the hardware structure of an electronic device according to the present invention. Figure 10 As shown, from a hardware perspective, this is a hardware structure diagram of any device with data processing capabilities, including the flood inundation area extraction device based on dual-temporal remote sensing imagery of the present invention. Except... Figure 10 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0134] When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0135] The present invention provides a flood inundation area extraction method based on dual-temporal remote sensing images. The generative adversarial network (GAN) is trained using an independent dual-branch backbone network to mitigate the interference of image features on the system and imaging conditions. The independent dual-branch change detection network of the flood inundation area extraction model extracts the flood inundation area. This model shares the backbone network model weights with the GAN. The GAN is used to optimize the backbone feature extraction parameters, reducing systematic and random errors between the two images, thereby obtaining high-precision flood inundation area extraction results.

[0136] The above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the present invention, and the patent protection scope of the present invention should be defined by the claims.

Claims

1. A method for extracting flood-inundated areas based on dual-temporal remote sensing imagery, characterized in that, include: Obtain pre- and post-contemporary remote sensing images of known flood-inundated areas as training data and construct a training dataset. An extraction model is constructed and trained using the training dataset. The extraction model includes a feature alignment model and a flooded area extraction model. The feature alignment model is a generative adversarial network model with a dual-branch residual convolutional neural structure, which has two identical backbone networks to process the pre-temporal remote sensing image and the post-temporal remote sensing image in parallel. The feature alignment model also includes a decoder, a generator, and a discriminator. The decoder comprises parallel DUA and DUB modules, each consisting of three upsampling convolutional layers. The generator includes GAN_A, composed of a backbone network BackboneA and the DUA module, and GAN_B, composed of a backbone network BackboneB and the DUB module. The discriminator includes discriminator modules Dis_A and Dis_B, each comprising two downsampling modules and a two-dimensional convolutional network. This feature alignment model shares backbone network weights with the flooded region extraction model, and backbone network parameters are updated synchronously during model training. The training steps for this extraction model include: Image features FA of the previous time-phase remote sensing image TA are extracted through the backbone network BackboneA, and image features FB of the subsequent time-phase remote sensing image TB are extracted through the backbone network BackboneB. The DUA module restores the original image size and number of channels from the image features FA to generate the reconstructed image SameA, and the DUB module restores the original image size and number of channels from the image features FB to generate the reconstructed image SameB. Using the subsequent time-phase remote sensing image TB as input, the generator GAN_A generates the pseudo image FakeA; using the previous time-phase remote sensing image TA as input, the generator GAN_B generates the pseudo image FakeB. The generator GAN_A takes the pseudo image FakeA as input to generate the cyclic reconstructed image RecoverA, and the generator GAN_B takes the pseudo image FakeB as input to generate the cyclic reconstructed image RecoverB. The weight sharing network is used to align the features of the cyclic reconstructed images RecoverA and RecoverB. Using loss function L GAN总 Train this feature alignment model, L GAN总 = L1 A + L1 B + L GAN_A + L GAN_B +L consistent_A + L consistent_B ; , representing the loss between Real_A and the reconstructed image SameA, where Real_A is the real image input to the generator GAN_A; , representing the loss between Real_B and the reconstructed image SameB, where Real_B is the real image input to the generator GAN_B; , represents the loss of the generator GAN_A attempting to deceive the discrimination module Dis_A, where N is the number of images involved in the calculation; This indicates the loss of the generator GAN_B attempting to deceive the authentication module Dis_B; This represents the consistency loss between the cyclically reconstructed images RecoverA and Real_A; This represents the consistency loss between the cyclically reconstructed images RecoverB and Real_B; The flooded area extraction model is constructed using backbone networks BackboneA and BackboneB, as well as gated feature fusion module GCFM and gated convolution module GCM. The trained extraction model is used to extract the flood-inundated area of ​​the target region.

2. The method for extracting flood-inundated areas as described in claim 1, characterized in that, The extraction model is trained and optimized using a joint loss function L. ; , The two-channel features are the final output of the extraction model; , This indicates that the auxiliary classifier affects the generated samples. The predicted probability; Tags indicating auxiliary tasks; Indicates the number of samples.

3. The method for extracting flood-inundated areas as described in claim 1, characterized in that, The process of training this extraction model also includes: The accuracy of the trained extraction model is evaluated using the test set of the training dataset; the harmonic index F1-score of the extraction model is obtained, and the extraction model with the largest harmonic index F1-score is the final extraction model. Wherein, Precision represents the proportion of pixels that the model predicts to be flooded areas but are actually flooded areas. Recall represents the proportion of pixels in the actual flooded area that were correctly predicted. TP represents correctly predicted flooded area pixels, FP represents incorrectly predicted flooded area pixels, and FN represents missed flooded area pixels.

4. A device for extracting flood-inundated areas based on dual-temporal remote sensing imagery, characterized in that, include: The dataset construction module is used to acquire pre- and post-contemporary remote sensing images of known flood-inundated areas as training data and construct a training dataset. The model training module is used to build an extraction model and train the extraction model with the training dataset. The extraction model includes a parallel feature alignment model and a flooded area extraction model. The feature alignment model is a generative adversarial network model with a two-branch residual convolutional neural network structure, which has two identical backbone networks to process the previous and subsequent time-phase remote sensing images in parallel. The feature alignment model shares backbone network weights with the flooded area extraction model, and the backbone network parameters are updated synchronously during model training; including: The feature alignment model construction module is used to construct the feature alignment model using a generative adversarial network model with a dual-branch residual convolutional neural network structure. It includes a decoder, a generator, and a discriminator for constructing the feature alignment model. The decoder comprises parallel DUA and DUB modules, each consisting of three upsampling convolutional layers. The generator includes GAN_A, composed of a backbone network BackboneA and a DUA module, and GAN_B, composed of a backbone network BackboneB and a DUB module. The discriminator includes discriminator modules Dis_A and Dis_B, each comprising two downsampling modules and a two-dimensional convolutional network. The feature alignment model training module is used to train the model with a loss function L. GAN总 Training the feature alignment model includes: extracting image features FA from the preceding temporal remote sensing image TA through the backbone network BackboneA, and extracting image features FB from the following temporal remote sensing image TB through the backbone network BackboneB; restoring the original image size and number of channels from the image features FA using the DUA module to generate the reconstructed image SameA, and restoring the original image size and number of channels from the image features FB using the DUB module to generate the reconstructed image SameB; generating a pseudo-image FakeA using the following temporal remote sensing image TB as input from the generator GAN_A, and generating a pseudo-image FakeB using the preceding temporal remote sensing image TA as input from the generator GAN_B; generating a cyclic reconstructed image RecoverA using the pseudo-image FakeA as input from the generator GAN_A, and generating a cyclic reconstructed image RecoverB using the pseudo-image FakeB as input from the generator GAN_B; and performing feature alignment between the cyclic reconstructed images RecoverA and RecoverB using a weight sharing network. GAN总 = L1 A + L1 B + L GAN_A + L GAN_B + L consistent_A + L consistent_B ; , representing the loss between Real_A and the reconstructed image SameA, where Real_A is the real image input to the generator GAN_A; , representing the loss between Real_B and the reconstructed image SameB, where Real_B is the real image input to the generator GAN_B; , represents the loss of the generator GAN_A attempting to deceive the discrimination module Dis_A, where N is the number of images involved in the calculation; This indicates the loss of the generator GAN_B attempting to deceive the authentication module Dis_B; This represents the consistency loss between the cyclically reconstructed images RecoverA and Real_A; This represents the consistency loss between the cyclically reconstructed images RecoverB and Real_B; The flooded area extraction model building module is used to build the flooded area extraction model using backbone networks BackboneA and BackboneB, as well as the gated feature fusion module GCFM and the gated convolution module GCM. The extraction module is used to extract flood-inundated areas from the target area using a trained extraction model.

5. The flood-inundated area extraction device as described in claim 4, characterized in that, The model training module also includes: The optimization module is used to train and optimize the extraction model using the joint loss function L. ;include: The flooded area extraction model optimization module is used to apply the cross-entropy loss function CE. W Train and optimize the flooded area extraction model. , The two-channel features are the final output of the extraction model; The feature alignment model optimization module is used to apply the cross-entropy loss function CE. G Train and optimize the generator of this feature alignment model. , This indicates that the auxiliary classifier affects the generated samples. The predicted probability; Tags indicating auxiliary tasks; Indicates the number of samples.

6. The flood-inundated area extraction device as described in claim 4, characterized in that, The model training module also includes a model evaluation module, which is used to evaluate the accuracy of the trained extraction model using the test set of the model training dataset; wherein, the harmonic index F1-score of the extraction model is obtained, and the extraction model with the largest harmonic index F1-score is the final extraction model. Precision represents the proportion of pixels that the model predicts to be flooded areas, but which are actually flooded areas. Recall represents the proportion of pixels in the actual flooded area that were correctly predicted. TP represents correctly predicted flooded area pixels, FP represents incorrectly predicted flooded area pixels, and FN represents missed flooded area pixels.

7. An electronic device comprising a flood inundation area extraction device based on dual-temporal remote sensing imagery as described in any one of claims 4 to 6.

8. A computer-readable storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, the flood inundation area extraction method based on dual-temporal remote sensing imagery as described in any one of claims 1 to 3 is implemented.

Images