Urban impervious surface all-weather extraction method and system based on stacked generative adversarial network, storage medium and electronic equipment

Abstract

The invention discloses an urban impervious surface all-weather extraction method and system based on a stacked generative adversarial network, a storage medium and electronic equipment, and the method comprises the following steps: carrying out the corresponding preprocessing of an optical remote sensing image and an SAR remote sensing image, and dividing the images into a training set and a test set; constructing an impervious surface data set: constructing a generative adversarial network for conversion from an SAR image domain to an optical image domain, and constructing an urban impervious surface extraction model based on optical and SAR image feature fusion; and training and testing the model to obtain a trained training model. According to the method, the simulated optical image is generated from the SAR image by using the generative adversarial network, and then the generated simulated optical image and the SAR image are fused and input into the impervious surface extraction model based on the generative adversarial network for impervious surface extraction, so that the impervious surface extraction precision based on the SAR image is improved; all-weather extraction of the urban impervious surface in the cloudy and rainy tropical / subtropical region is realized, and the application blank of optical remote sensing is filled.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing technology, and in particular to a method, system, storage medium and electronic device for extracting urban impervious surfaces around the clock based on a stacked adversarial generation network. Background Art

[0002] Currently, in the past few decades, the world has been hit by a wave of rapid urbanization, resulting in a large number of natural surfaces being replaced by impervious surfaces. Impervious surfaces refer to urban artificial surfaces that prevent water from infiltrating into the soil, mainly including buildings, roads, parking lots, roofs, etc. The replacement of permeable natural surfaces with impervious surfaces has changed the material cycle process of the global ecosystem, leading to increased ecological risks and threatening human health. Urban impervious surfaces have become a key indicator for measuring the degree of urbanization and the quality of urban ecological environment. Building inclusive, safe, resilient and sustainable cities and human settlements is one of the 17 Sustainable Development Goals of the United Nations. Therefore, it is very important to conduct regular monitoring of impervious surfaces in rapidly urbanizing areas.

[0003] This patent mainly studies how to use only SAR images to achieve all-weather accurate extraction of urban impervious surfaces in cloudy and rainy tropical / subtropical areas, and make up for the application gap of optical remote sensing data. Optical remote sensing data has rich spectral information, high spatial resolution, good image quality, and can extract urban impervious surfaces with high precision, but optical remote sensing belongs to passive remote sensing. For cloudy and rainy areas, optical remote sensing cannot play its advantages, and there are obvious application scenario gaps, which cannot achieve all-weather accurate extraction of urban impervious surfaces. Synthetic aperture radar (SAR) SAR images belong to active remote sensing, with a long wavelength and independent of environmental conditions such as light. It has all-weather ground observation capabilities, but SAR is side-view imaging, with large geometric distortion, lack of spectral information of ground objects, and coherent speckle noise seriously affects image quality. However, the fastest urbanizing region in the world is currently located in Southeast Asia, which is a typical tropical or subtropical region with perennial clouds and rain. According to statistics, for cloudy and rainy tropical and subtropical regions, the single-point effective days of optical remote sensing data are less than 40%, and the regional effective days are less than 20%. This makes it difficult for optical satellites to effectively monitor impervious surfaces. The accuracy of urban impervious surface extraction based solely on SAR images is low and cannot meet application requirements.

[0004] Given its powerful ability to represent nonlinear relationships, deep learning technology has been widely used in various fields. With the development of multi-platform and multi-modal remote sensing sensors, today's remote sensing field is already in the era of remote sensing big data, which has enabled deep learning technology to be successfully applied to land use / cover classification, ground object recognition, remote sensing image retrieval, and other aspects, and has gradually developed into a mainstream method. In the extraction of impervious surfaces based on SAR images, deep learning extraction methods are mainly improved from two aspects: model structure and loss function. Improving the quality of SAR images from the source of the data to improve the accuracy of impervious surface extraction is not considered. Although these deep learning-based impervious surface extraction methods have achieved significantly better results than traditional machine learning impervious surface methods, their extraction accuracy is significantly lower than that of optical images, which is not enough to meet application requirements. Summary of the invention

[0005] The purpose of the present invention is to provide a method, system, storage medium and electronic device for all-weather extraction of urban impervious surfaces based on a stacked adversarial generative network, which can improve the accuracy of extracting urban impervious surfaces using SAR images alone.

[0006] The technical solution adopted by the present invention is:

[0007] The all-weather extraction method of urban impervious surface based on stacked adversarial generative network includes the following steps:

[0008] Step a, respectively preprocessing the optical remote sensing images and the SAR remote sensing images, thereby constructing a data set of optical and SAR image domain transformation, and randomly dividing the data set into a training set and a test set according to a certain ratio;

[0009] Step b, constructing an impervious surface dataset: With the assistance of visual interpretation of optical images, the SAR images are interpreted and the ground object types are annotated pixel by pixel. The annotated ground object types include impervious surfaces and permeable surfaces, thereby constructing an impervious surface dataset, and randomly dividing the dataset into a training set and a test set according to a certain ratio;

[0010] Step c, constructing a generative adversarial network for transforming SAR image domain to optical image domain, i.e., constructing a first generative adversarial network model, specifically comprising: utilizing the powerful nonlinear relationship modeling capability of deep neural networks, utilizing the optical and SAR image domain transformation data sets constructed in step a, constructing a relationship between the SAR image backscattering intensity and the optical remote sensing image spectral reflectivity based on a generative adversarial network, generating a simulated optical image, i.e., a quasi-optical image, thereby realizing the transformation of the SAR image from the SAR image domain to the optical domain, obtaining the spectral information of the simulated object, and supplementing the SAR image;

[0011] Step d, constructing an urban impervious surface extraction model based on the fusion of optical and SAR image features, that is, constructing a second generative adversarial network model, specifically comprising: fusing the quasi-optical image generated in step c with the SAR image to obtain a fused image X, and inputting the fused image X into the second generative adversarial network to extract optical and SAR image features at different levels of the ground object, and constructing an urban impervious surface extraction model based on the fusion of optical and SAR image features;

[0012] Step e: under the support of the optical and SAR image domain transformation data set constructed in step a and the impervious surface data set constructed in step b, and under the constraint of the target loss function, the first generative adversarial network model and the second generative adversarial network model constructed in steps c and d are trained, and the training model composed of the first generative adversarial network model and the second generative adversarial network is tested using the test data set to obtain a trained training model;

[0013] Step f: input the SAR image of the impervious surface area to be extracted into the trained model obtained in step e to extract the impervious surface.

[0014] The generative adversarial network for transforming the SAR image domain to the optical image domain in step c adopts a conditional generative adversarial network, which includes a generator and a discriminator. The generator is used to generate simulated optical images from SAR images, and the discriminator is used to judge the authenticity of the generated optical images under the condition of real optical images. The two are mutually adversarial.

[0015] The pix2pix model is used in the step c, and its generator is a UNet model. The encoder consists of multiple 3×3 convolutional layers, batch normalization layers and ReLU activation layers, and downsampling is performed through maximum pooling to extract multi-scale features of the image. The decoder also consists of multiple 3×3 convolutional layers, batch normalization layers and ReLU activation layers, and the feature size extracted by the encoder is gradually restored to the size of the input image through upsampling; in addition, the encoder features and the decoder features are connected through skip connections to achieve the fusion of high- and low-level features and reduce the spatial detail information lost in the downsampling operation; the discriminator is a conditional patchGAN, which judges whether the generated optical image is true or false under the condition of the corresponding real optical image.

[0016] The target loss function of the generated adversarial network in step c includes the generator and discriminator loss functions:

[0017]

[0018] Where D(·) is the discriminator, G(·) is the generator, x is the input SAR image, and y is the real optical image; E x～Pdata(x) It expresses expectations.

[0019] In order to better generate realistic optical images, the generator loss function includes: total variation loss, gradient loss, L1 loss and adversarial loss, as shown below:

[0020] L Total =L1+αL TV +βL gradient +δL adversarial (2)

[0021] Among them, L Total is the total generator loss function, L1 is the L1 loss function, L TV is the total variational loss function, L gradient is the gradient loss function, L adversarial To combat the loss function, α, β, and δ are weight parameters.

[0022] The generator of the second generative adversarial network is UNet, and the discriminator is composed of blocks consisting of 4 convolutional layers, ReLU and batch normalization layers; the loss function of the generator includes adversarial loss, Dice loss and cross entropy loss, and the specific expression is as follows:

[0023] L Total =L BCE +αL Dice +βL adversarial (3).

[0024] In step f, the model test results are evaluated using four indicators: overall accuracy based on the confusion matrix, Kappa coefficient, mean intersection and union ratio, and F1 score.

[0025] An all-weather urban impervious surface extraction system based on stacked generative adversarial networks.

[0026] Includes the following modules,

[0027] The preprocessing module is used to construct optical and SAR image domain transformation data sets by preprocessing the optical and SAR images, and randomly divide the data sets into training sets and test sets according to a certain ratio;

[0028] An impervious surface dataset construction module is used to interpret the SAR image and annotate each pixel as an impervious surface or a permeable surface according to the visual interpretation of the optical image, to construct an impervious surface dataset, and to randomly divide the dataset into a training set and a test set according to a certain ratio; a SAR image to optical image generation module is used to construct a SAR image generation-like optical image model, i.e., a first generative adversarial network model, to input the optical and SAR image domain transformation sample data constructed in step a into the first generative adversarial network, and to construct a model target loss function; an urban impervious surface extraction model construction module is used to construct an impervious surface extraction model based on a generative adversarial network, i.e., a second generative adversarial network model, to fuse the preprocessed SAR image obtained in step a and the optical-like image obtained in step c and input them as sample data into the second generative adversarial network model, and to construct a model target loss function;

[0029] A model training and testing module is used to train the constructed first generative adversarial network model and the second generative adversarial network model under the constraints of the constructed target loss function and with the support of the data set, and to test the trained models using the test data, thereby obtaining the trained models;

[0030] The urban impervious surface extraction module is used to extract urban impervious surfaces. The SAR image of the area where impervious surface extraction is required is input into the trained model to extract the impervious surface and evaluate its extraction accuracy.

[0031] A computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the device where the computer-readable storage medium is located executes any of the above-described methods for extracting urban impervious surfaces around the clock based on a stacked adversarial generative network.

[0032] An electronic device includes: a memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, the all-weather extraction method of urban impervious surfaces based on a stacked adversarial generative network as described above is implemented.

[0033] The present invention generates simulated optical images from SAR images by using a generative adversarial network, and then fuses the generated simulated optical images with the SAR images and inputs them into an impervious surface extraction model based on a generative adversarial network to perform impervious surface extraction, thereby improving the accuracy of impervious surface extraction based on SAR images, realizing all-weather extraction of urban impervious surfaces in cloudy and rainy tropical / subtropical regions, filling the application gap of optical remote sensing, and synthesizing spectral information that is helpful for urban impervious surface extraction from a single SAR data source through the idea of ​​domain transformation, thereby improving the problem of low accuracy in extracting urban impervious surfaces using only SAR images. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 is a flow chart of the present invention;

[0036] Figure 2 This is a schematic diagram of the network structure of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] like Figure 1 and 2 As shown, the present invention first generates a simulated optical image from the SAR image through a generative adversarial network; then, the generated simulated optical image and the SAR image are fused and input into the constructed impervious surface extraction model based on the generative adversarial network, and under the joint constraints of total variation, gradient, Laplace, binary cross entropy loss, etc., the accuracy of impervious surface extraction based solely on SAR images is improved, and the all-weather extraction of urban impervious surfaces in cloudy and rainy tropical / subtropical areas is achieved, filling the application gap of optical remote sensing.

[0039] Through the above method, the present invention can synthesize simulated optical images from a single SAR data source, fuse the synthesized simulated optical images with the SAR images and use them for impervious surface extraction, thereby making up for the lack of spectral information of ground objects in SAR images and improving the accuracy of impervious surface extraction based on SAR images.

[0040] See also Figure 1 The embodiment of the present invention provides a method for extracting urban impervious surfaces around the clock based on a stacked adversarial generative network, comprising the following steps:

[0041] a. Construction of optical and SAR image domain transformation sample library: By preprocessing optical and SAR images, an optical and SAR image domain transformation sample library is constructed.

[0042] In the specific implementation, the acquired optical remote sensing images are first subjected to a series of preprocessing such as atmospheric correction, radiation calibration, and geometric correction, and the acquired SAR remote sensing images are subjected to a series of preprocessing such as radiation calibration, multi-view processing, coherent speckle denoising, and geocoding. Then, the optical and SAR images are registered through polynomials, and the root mean square error (RMSE) of the geometric registration is less than one pixel, thereby constructing an optical and SAR image domain transformation data set, and randomly dividing the data set into a training set and a test set according to a certain ratio. This step can be performed in advance, and the existing optical and SAR image domain transformation sample library can be directly input when entering the process.

[0043] b. Construction of impervious surface dataset: With the assistance of optical images, the SAR images are annotated pixel by pixel with the type of objects through visual interpretation. The annotated object types include impervious surfaces and permeable surfaces. This is used to construct an impervious surface dataset, and the dataset is randomly divided into training and test sets according to a certain ratio. This step can be performed in advance, and the existing SAR image impervious surface dataset can be directly input when entering the process.

[0044] In the specific implementation, with the assistance of optical images that have been registered with the SAR images, and combined with other data such as the national geographic survey, OpenStreetMap data, and the third national land survey, the SAR images are visually interpreted and the types of land features are annotated. The land feature annotation types include impervious surfaces and permeable surfaces, where impervious surfaces can be marked as "1" and permeable surfaces as "0". After all pixels are annotated, the entire image can be cropped into image blocks of 512×512, 256×256, or 128×128 pixels based on computing resources, thereby forming an urban impervious surface dataset based on SAR images.

[0045] c. Construction of a generative adversarial network for transforming SAR image domain to optical image domain: Utilizing the powerful nonlinear relationship modeling capability of deep neural networks, the relationship between the SAR image scattering intensity and the optical image spectral reflectivity is constructed based on a generative adversarial network to generate simulated optical images, thereby realizing the transformation of SAR images from the SAR image domain to the optical domain, obtaining the spectral information of ground object simulation, and supplementing the SAR images.

[0046] In the specific implementation, in order to control the output of the generative model, the generative adversarial network for transforming the SAR image domain to the optical image domain adopts a conditional generative adversarial network, which includes a generator and a discriminator. The generator is used to generate simulated optical images from SAR images, and the discriminator is used to judge the authenticity of the generated optical images under the condition of real optical images. The two are mutually antagonistic. The pix2pix model is used in this patent. Its generator is a UNet model. The encoder consists of multiple 3×3 convolutional layers, batch normalization layers, and ReLU activation layers, and the multi-scale features of the image are extracted by downsampling through maximum pooling. The decoder also consists of multiple 3×3 convolutional layers, batch normalization layers, and ReLU activation layers, and the feature size extracted by the encoder is gradually restored to the size of the input image through upsampling. In addition, the encoder features and the decoder features are connected through jump connections to realize the fusion of high and low-level features and reduce the spatial detail information lost in the downsampling operation. The discriminator is a conditional patchGAN, which judges whether the generated optical image is true or false under the condition of the corresponding real optical image.

[0047] The target loss function of the first generative adversarial network model includes the generator and discriminator loss functions:

[0048]

[0049] Among them, D(·) is the discriminator, G(·) is the generator, x is the input SAR image, and y is the real optical image.

[0050] In order to better generate realistic optical images, the generator loss function includes: total variation loss, gradient loss, L1 loss and adversarial loss, as shown below:

[0051] L Total =L1+αL TV +βL gradient +δL adversarial (2)

[0052] Among them, L Total is the total generator loss function, L1 is the L1 loss function, L TV is the total variational loss function, L gradient is the gradient loss function, L adversarial To combat the loss function, α, β, and δ are weight parameters.

[0053] Total variation loss: The total variation of an image is also called the norm of the image gradient L1. By minimizing the total variation of an image, noise in the image can be removed while retaining important details such as edges. Therefore, in order to reduce the image affected by coherent speckle noise from SAR images in simulated optical images generated by SAR images, based on the principle of total variation of images, this patent uses the total variation loss of the image as a regular term to constrain the learning of the generator and guide the model to output an optical image with a smoother spatial output. The definition of the total variation loss function is as follows:

[0054]

[0055] Among them, G(x) represents the optical image generated by the first generative adversarial network model, (i, j) represents the image pixel position, and W and H are the width and height of the generated optical image, respectively.

[0056] Gradient loss: In the field of image processing, gradient is usually used to measure the spatial detail information in the image. In order to make the optical image output by the model contain rich spatial detail information, this patent constrains the generated optical image to have similar gradient information as the real optical image, that is, the designed gradient loss function is the mean square error between the generated optical image gradient and the real optical image gradient. The specific calculation formula is as follows:

[0057]

[0058] in, and are the gradients of the generated optical image and the real optical image, respectively. (i, j) represents the image pixel position. W and H are the width and height of the image, respectively.

[0059] L1 loss: In order to avoid obvious spectral distortion of the optical image generated by the model, the generated optical image should have a similar pixel intensity distribution to the real optical image. This patent uses L1 loss for constraint, and its specific expression is as follows:

[0060]

[0061] Among them, G(x) and y represent the optical image generated by the model and the corresponding real optical image, respectively, (i, j) represents the image pixel position, and W and H are the width and height of the image, respectively;

[0062] Adversarial loss: The main function of the generator is to generate realistic optical images, making it difficult for the discriminator to distinguish between true and false. The adversarial loss function used in this patent is a binary logical cross entropy loss function, and its specific expression is as follows:

[0063] L adversarial =log(1-D(x,G(x))) (6)

[0064] Wherein, log(·) is a logarithmic operator.

[0065] d. The simulated optical image generated in step c is fused with the SAR image and input into the second generative adversarial network to extract optical and SAR image features of different levels of the ground objects for impervious surface extraction.

[0066] In the specific implementation, the simulated optical image generated in step c is cascaded with the SAR image to form a new multi-channel image, which is input into the second generative adversarial network for impervious surface extraction. Considering the superior performance and robustness of image segmentation based on the UNet network, the generator in the second generative adversarial network uses UNet, and the discriminator consists of a block consisting of 4 convolutional layers, ReLU and batch normalization layers. The loss function of the generator includes adversarial loss, Dice loss and cross entropy loss, and the specific expression is as follows:

[0067] L Total =L BCE +αL Dice +βL adversarial (7)

[0068] Cross entropy loss: Cross entropy loss is the most commonly used loss function based on pixel level in image semantic segmentation tasks, and its expression is as follows:

[0069]

[0070] Where N is the number of samples, y i represents the label of the i-th sample. In this paper, the positive class is the impervious surface and is marked as 1, while the negative class is the pervious surface and is marked as 0. i It represents the probability that the i-th sample is predicted to be a positive class.

[0071] Dice loss: To address the problem of too small a proportion of foreground targets, this patent uses the Dice coefficient as the loss function for impervious surface segmentation. Dice loss is a function that measures the similarity between two contour regions. Its specific definition is:

[0072]

[0073] Among them, y i It indicates the probability that pixel i is an impervious surface. is true value.

[0074] Step e: Under the support of the optical and SAR image domain transformation datasets constructed in step a and the impervious surface dataset constructed in step b, and under the constraint of the target loss function, the first generative adversarial network and the second generative adversarial network constructed in steps c and d are trained, and the training models are tested using the test data to obtain qualified training models;

[0075] In the specific implementation, in order to avoid overfitting of the model and improve the robustness of the model, data enhancement can be performed on the training set by flipping, rotating, scaling, translating, adding random noise, etc. At the same time, during the training process, the "early stopping" strategy can be adopted according to the test accuracy of the model.

[0076] Step f: Input the SAR image of the impervious surface area to be extracted into the model trained in step e to extract the impervious surface and perform accuracy evaluation. In actual use, the model test results are evaluated using four indicators: overall accuracy based on the confusion matrix, Kappa coefficient, mean intersection and union ratio, and F1 score.

[0077] The present invention generates simulated optical images from SAR images by using a generative adversarial network, and then fuses the generated simulated optical images with the SAR images and inputs them into an impervious surface extraction model based on a generative adversarial network to perform impervious surface extraction, thereby improving the accuracy of impervious surface extraction based on SAR images, realizing all-weather extraction of urban impervious surfaces in cloudy and rainy tropical / subtropical regions, filling the application gap of optical remote sensing, and synthesizing spectral information that is helpful for urban impervious surface extraction from a single SAR data source through the idea of ​​domain transformation, thereby improving the problem of low accuracy in extracting urban impervious surfaces using only SAR images.

[0078] In some possible embodiments, a system for extracting urban impervious surfaces around the clock based on a stacked adversarial generative network is provided, comprising the following modules:

[0079] The preprocessing module is used to construct optical and SAR image domain transformation data sets by preprocessing the optical and SAR images, and randomly divide the data sets into training sets and test sets according to a certain ratio;

[0080] An impervious surface dataset construction module is used to interpret the SAR image and annotate each pixel as an impervious surface or a permeable surface according to the visual interpretation of the optical image, to construct an impervious surface dataset, and to randomly divide the dataset into a training set and a test set according to a certain ratio; a SAR image to optical image generation module is used to construct a SAR image generation-like optical image model, i.e., a first generative adversarial network model, to input the optical and SAR image domain transformation sample data constructed in step a into the first generative adversarial network, and to construct a model target loss function; an urban impervious surface extraction model construction module is used to construct an impervious surface extraction model based on a generative adversarial network, i.e., a second generative adversarial network model, to fuse the preprocessed SAR image obtained in step a and the optical-like image obtained in step c and input them as sample data into the second generative adversarial network model, and to construct a model target loss function;

[0081] A model training and testing module is used to train the constructed first generative adversarial network model and the second generative adversarial network model under the constraints of the constructed target loss function and with the support of the data set, and to test the trained models using the test data, thereby obtaining the trained models;

[0082] The urban impervious surface extraction module is used to extract urban impervious surfaces. The SAR image of the area where impervious surface extraction is required is input into the trained model for impervious surface extraction and its extraction accuracy is evaluated.

[0083] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the device where the computer-readable storage medium is located executes the above-mentioned all-weather extraction method for urban impervious surfaces based on a stacked adversarial generative network. The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory and other memory.

[0084] An electronic device includes: a memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, the all-weather extraction method of urban impervious surfaces based on a stacked adversarial generative network as described above is implemented.

[0085] If the module / unit integrated in the electronic device described in this application is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware devices through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0086] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0087] Computer-readable instructions are stored in the computer-readable storage medium, and the computer-readable instructions are executed by a processor in an electronic device to implement the all-weather extraction method for urban impervious surfaces based on a stacked adversarial generative network as described in any of the above embodiments.

[0088] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0089] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

[0091] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0092] Note that the above are only preferred embodiments of the present invention and the principles of the application technology. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention is described in more detail through the above embodiments, the present invention is not limited to the specific embodiments described herein, and may include more other effective embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An all-weather extraction method for urban impervious surfaces based on a stacked generative adversarial network, characterized by: The following steps are included: Step a, respectively preprocessing the optical remote sensing images and the SAR remote sensing images, thereby constructing a data set of optical and SAR image domain transformation, and randomly dividing the data set into a training set and a test set according to a certain ratio; Step b, constructing an impervious surface dataset: With the assistance of visual interpretation of optical images, the SAR images are interpreted and the ground object types are annotated pixel by pixel. The annotated ground object types include impervious surfaces and permeable surfaces, thereby constructing an impervious surface dataset, and randomly dividing the dataset into a training set and a test set according to a certain ratio; Step c, constructing a generative adversarial network for transforming SAR image domain to optical image domain, i.e., constructing a first generative adversarial network model, specifically comprising: utilizing the powerful nonlinear relationship modeling capability of deep neural networks, utilizing the optical and SAR image domain transformation data sets constructed in step a, constructing a relationship between the SAR image backscattering intensity and the optical remote sensing image spectral reflectivity based on a generative adversarial network, generating a simulated optical image, i.e., a quasi-optical image, thereby realizing the transformation of the SAR image from the SAR image domain to the optical domain, obtaining the spectral information of the simulated object, and supplementing the SAR image; Step d, constructing an urban impervious surface extraction model based on the fusion of optical and SAR image features, that is, constructing a second generative adversarial network model, specifically comprising: fusing the quasi-optical image generated in step c with the SAR image to obtain a fused image X, and inputting the fused image X into the second generative adversarial network to extract optical and SAR image features at different levels of the ground object, and constructing an urban impervious surface extraction model based on the fusion of optical and SAR image features; Step e: under the support of the optical and SAR image domain transformation data set constructed in step a and the impervious surface data set constructed in step b, and under the constraint of the target loss function, the first generative adversarial network model and the second generative adversarial network model constructed in steps c and d are trained, and the training model composed of the first generative adversarial network model and the second generative adversarial network is tested using the test data set to obtain a trained training model; Step f: input the SAR image of the impervious surface area to be extracted into the trained model obtained in step e to extract the impervious surface.

2. According to claim 1, the method for extracting urban impervious surfaces around the clock based on a stacked adversarial generative network is characterized by: The generative adversarial network for transforming the SAR image domain to the optical image domain in step c adopts a conditional generative adversarial network, which includes a generator and a discriminator. The generator is used to generate simulated optical images from SAR images, and the discriminator is used to judge the authenticity of the generated optical images under the condition of real optical images. The two are mutually adversarial.

3. The all-weather extraction method for urban impervious surfaces based on stacked adversarial generative networks according to claim 1 is characterized by: The pix2pix model is used in the step c, and its generator is a UNet model. The encoder consists of multiple 3×3 convolutional layers, batch normalization layers and ReLU activation layers, and downsampling is performed through maximum pooling to extract multi-scale features of the image. The decoder also consists of multiple 3×3 convolutional layers, batch normalization layers and ReLU activation layers, and the feature size extracted by the encoder is gradually restored to the size of the input image through upsampling; in addition, the encoder features and the decoder features are connected through skip connections to achieve the fusion of high- and low-level features and reduce the spatial detail information lost in the downsampling operation; the discriminator is a conditional patchGAN, which judges whether the generated optical image is true or false under the condition of the corresponding real optical image.

4. The all-weather extraction method for urban impervious surfaces based on stacked adversarial generative networks according to claim 1, characterized in that: The target loss function of the generated adversarial network in step c includes the generator and discriminator loss functions: Where D(·) is the discriminator, G(·) is the generator, x is the input SAR image, and y is the real optical image; E x～Pdata(x) It expresses expectations. In order to better generate realistic optical images, the generator loss function includes: total variation loss, gradient loss, L1 loss and adversarial loss, as shown below: L Total =L1+αL TV +βL gradient +δL adversarial (2) Among them, L Total is the total generator loss function, L1 is the L1 loss function, L TV is the total variational loss function, L gradient is the gradient loss function, L adversarial To combat the loss function, α, β, and δ are weight parameters.

5. The all-weather extraction method for urban impervious surfaces based on stacked adversarial generative networks according to claim 1, characterized in that: The generator of the second generative adversarial network is UNet, and the discriminator is composed of blocks consisting of 4 convolutional layers, ReLU and batch normalization layers; the loss function of the generator includes adversarial loss, Dice loss and cross entropy loss, and the specific expression is as follows: L Total =L BCE +αL Dice +βL adversarial (7)。 6. The all-weather extraction method for urban impervious surfaces based on stacked adversarial generative networks according to claim 1 or 2, characterized in that: In step f, the model test results are evaluated using four indicators: overall accuracy based on the confusion matrix, Kappa coefficient, mean intersection and union ratio, and F1 score.

7. An all-weather urban impervious surface extraction system based on a stacked adversarial generative network, characterized by: Includes the following modules, The preprocessing module is used to construct optical and SAR image domain transformation data sets by preprocessing the optical and SAR images, and randomly divide the data sets into training sets and test sets according to a certain ratio; The module for constructing an impervious surface dataset is used to interpret SAR images and annotate each pixel as an impervious surface or a pervious surface based on visual interpretation of optical images, construct an impervious surface dataset, and randomly divide the dataset into a training set and a test set according to a certain ratio. A SAR image to optical image generation module is used to construct an optical image model for SAR image generation, i.e., a first generative adversarial network model, input the optical and SAR image domain transformation sample data constructed in step a into the first generative adversarial network, and construct a model target loss function; An urban impervious surface extraction model construction module is used to construct an impervious surface extraction model based on a generative adversarial network, i.e., a second generative adversarial network model, fuse the preprocessed SAR image obtained in step a and the quasi-optical image obtained in step c and input them into the second generative adversarial network model as sample data, and construct a model target loss function; A model training and testing module is used to train the constructed first generative adversarial network model and the second generative adversarial network model under the constraints of the constructed target loss function and with the support of the data set, and to test the trained models using the test data, thereby obtaining the trained models; The urban impervious surface extraction module is used to extract urban impervious surfaces. The SAR image of the area where impervious surface extraction is required is input into the trained model to extract the impervious surface and evaluate its extraction accuracy.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the device where the computer-readable storage medium is located executes the all-weather extraction method for urban impervious surfaces based on a stacked adversarial generative network as described in any one of claims 1 to 6.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, it implements the all-weather extraction method for urban impervious surfaces based on a stacked adversarial generative network as described in any one of claims 1 to 6.

Images