Urban impervious surface all-weather extraction method and system based on stacked generative adversarial network, storage medium and electronic device

Abstract

The application discloses a city impervious surface all-weather extraction method and system based on a stacked generative adversarial network, a storage medium and an electronic device, and comprises the following steps: respectively pre-processing optical remote sensing images and SAR remote sensing images into a training set and a test set; constructing an impervious surface dataset: constructing a generative adversarial network for SAR image domain to optical image domain transformation, and constructing an urban impervious surface extraction model based on optical and SAR image feature fusion; training the model and testing the model to obtain a trained training model. The application generates simulated optical images from SAR images by using a generative adversarial network, then fuses the generated simulated optical images and SAR images and inputs them into an impervious surface extraction model based on the generative adversarial network for impervious surface extraction, improves the impervious surface extraction precision based on SAR images, realizes all-weather extraction of urban impervious surfaces in a tropical / subtropical region with much cloud and rain, and fills the application gap of optical remote sensing.

Description

Technical Field

[0001] This invention relates to the field of remote sensing technology, and in particular to a method, system, storage medium, and electronic device for all-weather extraction of urban impermeable surfaces based on stacked adversarial generative networks. Background Technology

[0002] Currently, the world has been impacted by rapid urbanization over the past few decades, leading to the replacement of vast amounts of natural surfaces with impermeable surfaces. Impermeable surfaces refer to urban artificial surfaces that prevent water infiltration into the soil, primarily including buildings, roads, parking lots, and rooftops. The replacement of permeable natural surfaces with impermeable ones alters the material cycling processes of global ecosystems, increasing ecological risks and threatening human health. Urban impermeability has become a key indicator for measuring the degree of urbanization and the quality of the urban ecological environment. Building inclusive, safe, resilient, and sustainable cities and human settlements is one of the 17 Sustainable Development Goals of the United Nations. Therefore, regular monitoring of impermeable surfaces is crucial in rapidly urbanizing areas.

[0003] This patent primarily studies how to accurately extract impermeable surfaces in urban areas of tropical / subtropical regions with frequent cloud cover and rainfall using only SAR imagery, thus filling a gap in the application of optical remote sensing data. Optical remote sensing data possesses rich spectral information, high spatial resolution, and good image quality, enabling high-precision extraction of urban impermeable surfaces. However, optical remote sensing is a passive method, and its advantages are limited in cloudy and rainy areas, resulting in a significant application gap and preventing accurate all-weather extraction of urban impermeable surfaces. Synthetic aperture radar (SAR) imagery is an active remote sensing method with longer wavelengths and independence from environmental conditions such as lighting, providing all-weather Earth observation capabilities. However, SAR is a side-looking imaging method, resulting in significant geometric distortion, a lack of spectral information about ground features, and severe speckle noise that significantly impacts image quality. Furthermore, the world's fastest-urbanizing region is currently located in Southeast Asia, a typical tropical or subtropical region characterized by frequent cloud cover and rainfall. According to statistics, for cloudy and rainy tropical and subtropical regions, the effective days for a single point in optical remote sensing data is less than 40%, and the effective days for a region are less than 20%. This makes it difficult for optical satellites to effectively monitor impermeable surfaces, and the extraction accuracy of urban impermeable surfaces based solely on SAR imagery is low and cannot meet application requirements.

[0004] Given its powerful ability to represent nonlinear relationships, deep learning technology has been widely applied in various fields. With the development of multi-platform and multi-modal remote sensing sensors, the field of remote sensing is now in the era of big data, enabling deep learning technology to be successfully applied to land use / cover classification, ground object recognition, and remote sensing image retrieval, gradually becoming a mainstream method. In impervious surface extraction based on SAR imagery, deep learning extraction methods mainly improve the model structure and loss function, without considering improving the SAR image quality at the data source to enhance the accuracy of impervious surface extraction. Although these deep learning-based impervious surface extraction methods have achieved significantly better results than traditional machine learning methods, their extraction accuracy is significantly lower than that of optical images, insufficient to meet application requirements. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, storage medium, and electronic device for all-weather extraction of urban impervious surfaces based on stacked adversarial generative networks, which can improve the accuracy of extracting urban impervious surfaces using SAR images alone.

[0006] The technical solution adopted in this invention is as follows:

[0007] A method for all-weather extraction of impermeable surfaces in cities based on stacked generative adversarial networks includes the following steps.

[0008] Step a: By performing corresponding preprocessing on optical remote sensing images and SAR remote sensing images respectively, a dataset of optical and SAR image domain transformation is constructed, and the dataset is randomly divided into training set and test set according to a certain ratio.

[0009] Step b: Construct an impermeable surface dataset: With the assistance of visual interpretation of optical images, interpret SAR images and label the land cover types pixel by pixel. The labeled land cover types include two categories: impermeable surfaces and permeable surfaces, thereby constructing an impermeable surface dataset. The dataset is then randomly divided into a training set and a test set according to a certain ratio.

[0010] Step c, the construction of the generative adversarial network for the transformation from SAR image domain to optical image domain, i.e. the construction of the first generative adversarial network model, specifically includes: using the powerful nonlinear relationship modeling capability of deep neural networks, using the optical and SAR image domain transformation datasets constructed in step a, constructing the relationship between the backscattering intensity of SAR images and the spectral reflectance of optical remote sensing images based on the generative adversarial network, generating simulated optical images, i.e., quasi-optical images, thereby realizing the transformation of SAR images from the SAR image domain to the optical domain, obtaining the spectral information of simulated ground features, and supplementing the SAR images;

[0011] Step d: Construct an urban impervious surface extraction model based on the fusion of optical and SAR image features, i.e., the construction of the second generative adversarial network model. Specifically, this includes: fusing the optical-like image generated in step c with the SAR image to obtain the fused image X, and inputting it into the second generative adversarial network to extract optical and SAR image features at different levels of ground features, thereby constructing an urban impervious surface extraction model based on the fusion of optical and SAR image features.

[0012] Step e: With the support of the optical and SAR image domain transformation dataset constructed in step a and the impermeable surface dataset constructed in step b, and under the constraint of the target loss function, the first generative adversarial network model and the second generative adversarial network model constructed in steps c and d are trained, and the first generative adversarial network model and the second generative adversarial network model are tested using the test dataset to obtain the trained model.

[0013] Step f: Input the SAR image of the impermeable surface area to be extracted into the trained model obtained in step e to extract the impermeable surface.

[0014] In step c, the generative adversarial network for the transformation from the SAR image domain to the optical image domain is a conditional generative adversarial network, which includes a generator and a discriminator. The generator is used to generate simulated optical images from SAR images, and the discriminator is used to judge the authenticity of the generated optical images under the condition of real optical images. The two are adversarial to each other.

[0015] Step c employs a pix2pix model, whose generator is a UNet model. The encoder consists of multiple 3×3 convolutional layers, batch normalization layers, and ReLU activation layers, and uses max pooling to downsample and extract multi-scale features from the image. The decoder also consists of multiple 3×3 convolutional layers, batch normalization layers, and ReLU activation layers, and uses upsampling to gradually restore the size of the features extracted by the encoder to the size of the input image. Furthermore, skip connections are used to connect the encoder and decoder features to achieve the fusion of high and low-level features, reducing the spatial detail information lost during downsampling. The discriminator is a conditional patchGAN, which determines whether the generated optical image is real or fake under the condition of the corresponding real optical image.

[0016] Step c generates the target loss function of the adversarial network, including the generator and discriminator loss functions:

[0017]

[0018] Where D(·) is the discriminator, G(·) is the generator, x is the input SAR image, and y is the actual optical image; E x～Pdata(x) It represents expectations.

[0019] To generate more realistic optical images, the generator loss function includes: total variational loss, gradient loss, L1 loss, and adversarial loss, as shown in the following expression:

[0020] L Total =L1+αL TV +βL gradient +δL adversarial (2)

[0021] Among them, L Total Let L1 be the overall generator loss function, and L2 be the L1 loss function. TV Let L be the total variational loss function. gradient Let L be the gradient loss function. adversarial The adversarial loss function is defined by α, β, and δ as weight parameters.

[0022] The generator of the second generative adversarial network is UNet, and the discriminator consists of a block composed of four convolutional layers, ReLU, and batch normalization layers. The generator's loss function includes adversarial loss, Dice loss, and cross-entropy loss, with the specific expressions as follows:

[0023] L Total =L BCE +αL Dice +βL adversarial (3).

[0024] In step f, the model test results are evaluated using four metrics: overall accuracy based on the confusion matrix, Kappa coefficient, average intersection-union ratio, and F1 score.

[0025] A system for all-weather extraction of impermeable surfaces in cities based on stacked adversarial generative networks.

[0026] Includes the following modules,

[0027] The preprocessing module is used to preprocess optical and SAR images, construct an optical and SAR image domain transformation dataset, and randomly divide the dataset into training and test sets according to a certain ratio.

[0028] The impermeable surface dataset construction module is used to interpret SAR images based on visual interpretation of optical images and label each pixel as impermeable or permeable, constructing an impermeable surface dataset, and randomly dividing the dataset into training and testing sets according to a certain ratio; the SAR image to optical image generation module is used to construct a SAR image to optical image model, i.e., the first generative adversarial network model, inputting the optical and SAR image domain transformation sample data constructed in step a into the first generative adversarial network and constructing the model's target loss function; the urban impermeable surface extraction model construction module is used to construct an impermeable surface extraction model based on generative adversarial network, i.e., the second generative adversarial network model, fusing the preprocessed SAR image obtained in step a and the optical-like image obtained in step c as sample data into the second generative adversarial network model, and constructing the model's target loss function;

[0029] The model training and testing module is used to train the first and second generative adversarial network models under the constraints of the constructed target loss function and with the support of the dataset, and to test the trained models using test data, thereby obtaining the trained model.

[0030] The urban impervious surface extraction module is used to extract impervious surfaces in the city. It inputs the SAR image of the area where impervious surface extraction is required into the trained model to extract the impervious surface and evaluates its extraction accuracy.

[0031] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the device on which the computer-readable storage medium is located performs the all-weather extraction method for urban impermeable surfaces based on stacked adversarial generative networks as described above.

[0032] An electronic device includes a memory and a processor, wherein the memory stores a program executable on the processor, and the processor executes the program to implement the all-weather extraction method for urban impermeable surfaces based on stacked adversarial generative networks as described above.

[0033] This invention utilizes generative adversarial networks (GANs) to generate simulated optical images from SAR images. The generated simulated optical images and SAR images are then fused and input into a GAN-based impermeable surface extraction model for impermeable surface extraction. This improves the accuracy of impermeable surface extraction based on SAR images, enabling all-weather extraction of impermeable surfaces in cloudy and rainy tropical / subtropical cities. It fills a gap in the application of optical remote sensing. By employing the concept of domain transformation, it synthesizes spectral information that facilitates urban impermeable surface extraction from a single SAR data source, thus addressing the problem of low accuracy when using SAR images alone for urban impermeable surface extraction. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart of the present invention;

[0036] Figure 2 This is a schematic diagram of the network structure of the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] like Figure 1 and 2 As shown, this invention first generates simulated optical images from SAR images using a generative adversarial network (GAN). Subsequently, the generated simulated optical images and SAR images are fused and input into a constructed impermeable surface extraction model based on a GAN. Under the joint constraints of total variation, gradient, Laplacian, and binary cross-entropy loss, the accuracy of impermeable surface extraction based solely on SAR images is improved, enabling all-weather extraction of impermeable surfaces in cloudy and rainy tropical / subtropical urban areas, filling the application gap of optical remote sensing.

[0039] Through the above method, the present invention can synthesize simulated optical images from a single SAR data source, fuse the synthesized simulated optical images with SAR images and use them for impermeable surface extraction, make up for the lack of ground object spectral information in SAR images, and improve the accuracy of impermeable surface extraction based on SAR images.

[0040] See Figure 1 The present invention provides a method for all-weather extraction of urban impermeable surfaces based on stacked adversarial generative networks, comprising the following steps:

[0041] a. Construction of optical and SAR image domain transformation sample library: By preprocessing optical and SAR images, an optical and SAR image domain transformation sample library is constructed.

[0042] In practice, the acquired optical remote sensing images are first preprocessed with atmospheric correction, radiometric calibration, and geometric correction. The acquired SAR remote sensing images are then preprocessed with radiometric calibration, multi-view processing, speckle denoising, and geocoding. Next, the optical and SAR images are registered using polynomials, with the root mean square error (RMSE) of the geometric registration being less than one pixel. This constructs a domain transformation dataset for both optical and SAR images. The dataset is then randomly divided into training and test sets according to a certain ratio. This step can be performed in advance; an existing optical and SAR image domain transformation sample library can be directly input when entering the workflow.

[0043] b. Construction of Impermeable Surface Dataset: With the aid of optical imagery, SAR images are visually interpreted to label land cover types pixel by pixel. The labeled land cover types include impermeable and permeable surfaces, thereby constructing an impermeable surface dataset. The dataset is then randomly divided into training and test sets according to a certain ratio. This step can be performed in advance; when entering the workflow, an existing SAR image impermeable surface dataset can be directly input.

[0044] In practice, with the assistance of optical images already registered with SAR images, and combined with other data such as the national geographic census, OpenStreetMap data, and the Third National Land Survey, the SAR images are visually interpreted and land cover types are labeled. The land cover label types include impermeable surfaces and permeable surfaces, where impermeable surfaces can be labeled as "1" and permeable surfaces as "0". After all pixels are labeled, the entire image can be cropped into image blocks of 512×512, 256×256, or 128×128 pixels according to computing resources, thereby constructing an urban impermeable surface dataset based on SAR images.

[0045] c. Construction of Generative Adversarial Network for Transformation from SAR Image Domain to Optical Image Domain: Utilizing the powerful nonlinear relationship modeling capabilities of deep neural networks, a relationship between the scattering intensity of SAR images and the spectral reflectance of optical images is constructed based on the generative adversarial network. This generates simulated optical images, thereby realizing the transformation of SAR images from the SAR image domain to the optical domain, obtaining spectral information of simulated ground features, and supplementing the SAR images.

[0046] In practical implementation, to control the output of the generative model, the generative adversarial network for the transformation from the SAR image domain to the optical image domain uses a conditional generative adversarial network (GAN). This network comprises a generator and a discriminator. The generator generates simulated optical images from SAR images, while the discriminator judges the authenticity of the generated optical images under the condition of real optical images. The two are adversarial. This patent uses a pix2pix model, with a UNet generator. The encoder consists of multiple 3×3 convolutional layers, batch normalization layers, and ReLU activation layers, and uses max pooling to downsample and extract multi-scale features from the image. The decoder also consists of multiple 3×3 convolutional layers, batch normalization layers, and ReLU activation layers, and uses upsampling to gradually restore the size of the features extracted by the encoder to the size of the input image. Furthermore, skip connections are used to connect the encoder and decoder features, achieving the fusion of high and low-level features and reducing the spatial detail lost during downsampling. The discriminator is a conditional patchGAN, which determines whether the generated optical image is real or fake under the condition of the corresponding real optical image.

[0047] The objective loss function for the first generative adversarial network model includes the generator and discriminator loss functions:

[0048]

[0049] Where D(·) is the discriminator, G(·) is the generator, x is the input SAR image, and y is the actual optical image.

[0050] To generate more realistic optical images, the generator loss function includes: total variational loss, gradient loss, L1 loss, and adversarial loss, as shown in the following expression:

[0051] L Total =L1+αL TV +βL gradient +δL adversarial (2)

[0052] Among them, L Total Let L1 be the overall generator loss function, and L2 be the L1 loss function. TV Let L be the total variational loss function. gradient Let L be the gradient loss function. adversarial The adversarial loss function is defined by α, β, and δ as weight parameters.

[0053] Total variation loss: The total variation of an image, also known as the L1 norm of the image gradient, can remove noise from the image while preserving important details such as edges. Therefore, to reduce the speckle noise from SAR images in simulated optical images generated from SAR images, this patent uses the total variation loss as a regularization term to constrain the generator's learning, based on the principle of total variation, guiding the model to output optical images with a smoother spatial structure. The definition of the total variation loss function is as follows:

[0054]

[0055] Where G(x) represents the optical image generated by the first generative adversarial network model, (i,j) represents the pixel position of the image, and W and H are the width and height of the generated optical image, respectively.

[0056] Gradient Loss: In image processing, gradients are typically used to measure spatial detail in an image. To ensure that the optical image output by the model contains rich spatial detail, this patent constrains the generated optical image to have similar gradient information to the real optical image. Specifically, the designed gradient loss function is the mean square error between the gradients of the generated and real optical images, calculated as follows:

[0057]

[0058] in, and , where are the gradients of the generated optical image and the real optical image, respectively; (i,j) represents the image pixel position; and W and H are the width and height of the image, respectively.

[0059] L1 Loss: To avoid significant spectral distortion in the generated optical image, the generated optical image should have a similar pixel intensity distribution to the real optical image. This patent uses L1 loss for constraint, and its specific expression is as follows:

[0060]

[0061] Where G(x) and y represent the optical image generated by the model and the corresponding real optical image, respectively, (i,j) represents the image pixel position, and W and H are the width and height of the image, respectively;

[0062] Adversarial Loss: The generator's main function is to produce realistic optical images, making it difficult for the discriminator to distinguish between real and fake images. The adversarial loss function used in this patent is the binary classification logical cross-entropy loss function, the specific expression of which is as follows:

[0063] L adversarial =log(1-D(x,G(x))) (6)

[0064] Here, log(·) is the logarithmic operator.

[0065] d. The simulated optical image generated in step c is fused with the SAR image and input into the second generative adversarial network to extract optical and SAR image features of different levels of ground features for impermeable surface extraction.

[0066] In practice, the simulated optical image generated in step c is concatenated with the SAR image to form a new multi-channel image, which is then input into the second generative adversarial network (GAN) for impermeable surface extraction. Considering the superior performance and robustness of UNet-based image segmentation, the generator in the second GAN is based on UNet, and the discriminator consists of a block composed of four convolutional layers, ReLU, and batch normalization layers. The generator's loss function includes adversarial loss, Dice loss, and cross-entropy loss, with the specific expressions as follows:

[0067] L Total =L BCE +αL Dice +βL adversarial (7)

[0068] Cross-entropy loss: Cross-entropy loss is the most commonly used pixel-level loss function in image semantic segmentation tasks, and its expression is as follows:

[0069]

[0070] Where N is the number of samples, y i Let p represent the label of the i-th sample. In this paper, the positive class represents the impermeable surface and is labeled 1, while the negative class represents the permeable surface and is labeled 0. i This represents the probability that the i-th sample is predicted as positive.

[0071] Dice Loss: To address the issue of foreground targets being too small, this patent uses the Dice coefficient as the loss function for impermeable surface segmentation. Dice loss is a function that measures the similarity between two contour regions, and its specific definition is:

[0072]

[0073] Among them, y i This represents the probability that pixel i is an impermeable surface. It is a true value.

[0074] Step e: With the support of the optical and SAR image domain transformation dataset constructed in step a and the impermeable surface dataset constructed in step b, and under the constraint of the target loss function, the first generative adversarial network and the second generative adversarial network constructed in steps c and d are trained, and the trained model is tested using test data to obtain a qualified trained model.

[0075] In practice, to avoid overfitting and improve the robustness of the model, data augmentation can be performed on the training set by flipping, rotating, scaling, translating, or adding random noise. At the same time, during the training process, an "early stopping" strategy can be adopted based on the model's test accuracy.

[0076] Step f: Input the SAR image of the impermeable surface area to be extracted into the model trained in step e for impermeable surface extraction and perform accuracy evaluation. In actual use, the model test results are evaluated using four indicators: overall accuracy based on the confusion matrix, Kappa coefficient, average intersection-union ratio, and F1 score.

[0077] This invention utilizes generative adversarial networks (GANs) to generate simulated optical images from SAR images. The generated simulated optical images and SAR images are then fused and input into a GAN-based impermeable surface extraction model for impermeable surface extraction. This improves the accuracy of impermeable surface extraction based on SAR images, enabling all-weather extraction of impermeable surfaces in cloudy and rainy tropical / subtropical cities. It fills a gap in the application of optical remote sensing. By employing the concept of domain transformation, it synthesizes spectral information that facilitates urban impermeable surface extraction from a single SAR data source, thus addressing the problem of low accuracy when using SAR images alone for urban impermeable surface extraction.

[0078] In some possible embodiments, an all-weather extraction system for urban impermeable surfaces based on stacked adversarial generative networks is provided, including the following modules:

[0079] The preprocessing module is used to preprocess optical and SAR images, construct an optical and SAR image domain transformation dataset, and randomly divide the dataset into training and test sets according to a certain ratio.

[0080] The impermeable surface dataset construction module is used to interpret SAR images based on visual interpretation of optical images and label each pixel as impermeable or permeable, constructing an impermeable surface dataset, and randomly dividing the dataset into training and testing sets according to a certain ratio; the SAR image to optical image generation module is used to construct a SAR image to optical image model, i.e., the first generative adversarial network model, inputting the optical and SAR image domain transformation sample data constructed in step a into the first generative adversarial network and constructing the model's target loss function; the urban impermeable surface extraction model construction module is used to construct an impermeable surface extraction model based on generative adversarial network, i.e., the second generative adversarial network model, fusing the preprocessed SAR image obtained in step a and the optical-like image obtained in step c as sample data into the second generative adversarial network model, and constructing the model's target loss function;

[0081] The model training and testing module is used to train the first and second generative adversarial network models under the constraints of the constructed target loss function and with the support of the dataset, and to test the trained models using test data, thereby obtaining the trained model.

[0082] The urban impervious surface extraction module is used to extract impervious surfaces in the city. It inputs the SAR image of the area where impervious surface extraction is required into the trained model to extract the impervious surface and evaluates its extraction accuracy.

[0083] A computer-readable storage medium stores a computer program thereon. When executed by a processor, the computer program causes the device containing the computer-readable storage medium to perform the all-weather extraction method for impermeable urban surfaces based on stacked generative adversarial networks, as described above. The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), random access memory, and other memories.

[0084] An electronic device includes a memory and a processor, wherein the memory stores a program executable on the processor, and the processor executes the program to implement the all-weather extraction method for urban impermeable surfaces based on stacked adversarial generative networks as described above.

[0085] If the modules / units integrated in the electronic device described in this application are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0086] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0087] The computer-readable storage medium stores computer-readable instructions, which are executed by a processor in an electronic device to implement the all-weather extraction method for urban impermeable surfaces based on stacked adversarial generative networks as described in any of the above embodiments.

[0088] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

[0091] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification and claims of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0092] Note that the above description is merely a preferred embodiment and application of the technical principles of the present invention. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the specific embodiments described herein, and may include many other effective embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for all-weather extraction of impermeable urban surfaces based on stacked adversarial generative networks, characterized by: Includes the following steps, Step a: By performing corresponding preprocessing on optical remote sensing images and SAR remote sensing images respectively, a dataset of optical and SAR image domain transformation is constructed, and the dataset is randomly divided into training set and test set according to a certain ratio. Step b: Construct an impermeable surface dataset: With the assistance of visual interpretation of optical images, interpret SAR images and label the land cover types pixel by pixel. The labeled land cover types include two categories: impermeable surfaces and permeable surfaces, thereby constructing an impermeable surface dataset. The dataset is then randomly divided into a training set and a test set according to a certain ratio. Step c, the construction of the generative adversarial network for the transformation from SAR image domain to optical image domain, i.e. the construction of the first generative adversarial network model, specifically includes: using the powerful nonlinear relationship modeling capability of deep neural networks, using the optical and SAR image domain transformation datasets constructed in step a, constructing the relationship between the backscattering intensity of SAR images and the spectral reflectance of optical remote sensing images based on the generative adversarial network, generating simulated optical images, i.e., quasi-optical images, thereby realizing the transformation of SAR images from the SAR image domain to the optical domain, obtaining the spectral information of simulated ground features, and supplementing the SAR images; Step d: Construct an urban impervious surface extraction model based on the fusion of optical and SAR image features, i.e., the construction of the second generative adversarial network model. Specifically, this includes: fusing the optical-like image generated in step c with the SAR image to obtain the fused image X, and inputting it into the second generative adversarial network to extract optical and SAR image features at different levels of ground features, thereby constructing an urban impervious surface extraction model based on the fusion of optical and SAR image features. Step e: With the support of the optical and SAR image domain transformation dataset constructed in step a and the impermeable surface dataset constructed in step b, and under the constraint of the target loss function, the first generative adversarial network model and the second generative adversarial network model constructed in steps c and d are trained, and the first generative adversarial network model and the second generative adversarial network model are tested using the test dataset to obtain the trained model. Step f: Input the SAR image of the impermeable surface area to be extracted into the trained model obtained in step e to extract the impermeable surface.

2. The method for all-weather extraction of urban impermeable surfaces based on stacked adversarial generative networks according to claim 1, characterized in that: In step c, the generative adversarial network for the transformation from the SAR image domain to the optical image domain is a conditional generative adversarial network, which includes a generator and a discriminator. The generator is used to generate simulated optical images from SAR images, and the discriminator is used to judge the authenticity of the generated optical images under the condition of real optical images. The two are adversarial to each other.

3. The method for all-weather extraction of urban impermeable surfaces based on stacked adversarial generative networks according to claim 1, characterized in that: Step c employs a pix2pix model, whose generator is a UNet model. The encoder consists of multiple 3×3 convolutional layers, batch normalization layers, and ReLU activation layers, and uses max pooling to downsample and extract multi-scale features from the image. The decoder also consists of multiple 3×3 convolutional layers, batch normalization layers, and ReLU activation layers, and uses upsampling to gradually restore the size of the features extracted by the encoder to the size of the input image. Furthermore, skip connections are used to connect the encoder and decoder features to achieve the fusion of high and low-level features, reducing the spatial detail information lost during downsampling. The discriminator is a conditional patchGAN, which determines whether the generated optical image is real or fake under the condition of the corresponding real optical image.

4. The method for all-weather extraction of urban impermeable surfaces based on stacked adversarial generative networks according to claim 1, characterized in that: Step c generates the target loss function of the adversarial network, including the generator and discriminator loss functions: ； in, For discriminator, For generator, The input SAR image, For real optical images; It represents expectations; To generate more realistic optical images, the generator loss function includes: total variational loss, gradient loss, L1 loss, and adversarial loss, as shown in the following expression: ； in, Let be the total generator loss function. For L1 loss function, Let be the total variational loss function. Let be the gradient loss function. To counteract the loss function, , as well as These are the weight parameters.

5. The method for all-weather extraction of urban impermeable surfaces based on stacked adversarial generative networks according to claim 1, characterized in that: The generator of the second generative adversarial network is UNet, and the discriminator consists of a block composed of four convolutional layers, ReLU, and batch normalization layers. The generator's loss function includes adversarial loss, Dice loss, and cross-entropy loss, with the specific expressions as follows: 。 6. The method for all-weather extraction of urban impermeable surfaces based on stacked adversarial generative networks according to claim 1 or 2, characterized in that: In step f, the model test results are evaluated using four metrics: overall accuracy based on the confusion matrix, Kappa coefficient, average intersection-union ratio, and F1 score.

7. A system for all-weather extraction of impermeable urban surfaces based on stacked adversarial generative networks, characterized in that: Includes the following modules, The preprocessing module is used to preprocess optical and SAR images, construct an optical and SAR image domain transformation dataset, and randomly divide the dataset into training and test sets according to a certain ratio. The impermeable surface dataset construction module is used to interpret SAR images based on visual interpretation of optical images and label them pixel by pixel as impermeable or permeable surfaces to construct an impermeable surface dataset. The dataset is then randomly divided into training and test sets according to a certain ratio. The SAR image to optical image generation module is used to construct an optical image model for SAR image generation, namely the first generative adversarial network model. The optical and SAR image domain transformation sample data constructed in step a are input into the first generative adversarial network, and the model target loss function is constructed. The urban impervious surface extraction model construction module is used to construct an impervious surface extraction model based on generative adversarial network, namely the second generative adversarial network model. The preprocessed SAR image obtained in step a and the optical-like image obtained in step c are fused and used as sample data to input into the second generative adversarial network model, and the model target loss function is constructed. The model training and testing module is used to train the first and second generative adversarial network models under the constraints of the constructed target loss function and with the support of the dataset, and to test the trained models using test data, thereby obtaining the trained model. The urban impervious surface extraction module is used to extract impervious surfaces in the city. It inputs the SAR image of the area where impervious surface extraction is required into the trained model to extract the impervious surface and evaluates its extraction accuracy.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it causes the device containing the computer-readable storage medium to perform the all-weather extraction method for urban impermeable surfaces based on stacked adversarial generative networks as described in any one of claims 1-6.

9. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a program that can run on the processor, and the processor executes the program to implement the all-weather extraction method for urban impermeable surfaces based on stacked adversarial generative networks as described in any one of claims 1-6.

Images