Water body extraction method and equipment based on remote sensing image super-resolution reconstruction and medium

Abstract

The invention provides a water body extraction method and device based on remote sensing image super-resolution reconstruction and a medium, and relates to the field of remote sensing image processing, and the method comprises the steps: obtaining and preprocessing remote sensing image data, and making a data set; designing a remote sensing image super-resolution reconstruction model based on the combination of a generative adversarial network and a dense connection network of an image self-similarity principle; performing model training by using the data set; obtaining and preprocessing a remote sensing image of a to-be-detected area; carrying out the super-resolution reconstruction of the original image of the to-be-detected region based on the trained model, and obtaining a high-spatial-resolution reconstructed image; and extracting a reconstructed image water body by using a water body index method, and making a water body map of the to-be-detected area. An apparatus and a medium are used to implement the method. The method has the beneficial effects that the remote sensing image achieves higher spatial resolution through super-resolution reconstruction, so that the remote sensing water body drawing precision is improved, and a better effect is obtained in the application of water body extraction and surface water monitoring.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image processing, and in particular to a water body extraction method, equipment and medium based on remote sensing image super-resolution reconstruction. Background Art

[0002] Surface water is an important component of terrestrial water resources and an important source of water for human production and life, such as industry, agriculture, and daily drinking. Unfortunately, water pollution, natural disasters, and changes in land use have led to severe water shortages in many countries. Therefore, monitoring changes in surface water is an important aspect of water resources planning and management, and an important measure for economic development and water conservation. In order to achieve effective water resources management, reliable methods for water extraction and mapping are necessary.

[0003] With the great progress of remote sensing technology, the collection of remote sensing data and the development of computer science, remote sensing images have gradually become an important source of information for water body monitoring and extraction. However, due to the limitation of spatial resolution, remote sensing images with low spatial resolution such as MODIS and Landsat are prone to mixed pixels, especially at the edge of water bodies, where mixed pixels are more serious, greatly affecting the accuracy of water body boundary extraction. This problem is more prominent for the extraction of small water bodies. At the same time, although remote sensing images with high spatial resolution such as SPOT 6 / 7, IKONOS, Quick-bird, GF-2, etc. can draw these small water body maps, they provide less band data and lack short-wave infrared bands that are sensitive to water bodies. In addition, they all have the problem of high acquisition cost, making it difficult to obtain long-term series data, and thus unable to monitor changes in the range of water body boundaries.

[0004] It is worth noting that the Sentinel-2 remote sensing satellite provided by the European Space Agency can provide more detailed multispectral images, and has a higher frequency revisit period, and can provide remote sensing data including 10m and 20m spatial resolution. However, although the temporal and spatial resolution of Sentinel-2 remote sensing images can best meet the current needs of water body monitoring, there is still a problem of insufficient precision in the extraction of smaller water bodies, and the bands currently used mainly for water body extraction have different spatial resolutions in Sentinel-2. Therefore, it is necessary to improve the spatial resolution of the 10m and 20m bands of Sentinel-2 to the same higher precision in order to more accurately extract water body boundaries and thus monitor water body changes.

[0005] Super-resolution reconstruction has attracted much attention in recent years as a low-cost and effective method to improve the spatial resolution of images. The remote sensing image super-resolution reconstruction method is based on image super-resolution reconstruction. Through this technology, high-spatial-resolution remote sensing images can be generated from low-spatial-resolution remote sensing images to improve the quality and availability of remote sensing images, thereby achieving the effect of extracting more detailed water bodies.

[0006] Upsampling can be used to improve the spatial resolution of images, such as the commonly used interpolation-based method. However, this method cannot provide ideal reconstruction effects when reconstructing images to very fine spatial resolutions, and the reconstruction effect of water body edges is poor.

[0007] At present, most of the super-resolution reconstruction methods based on deep learning unify the bands of different spatial resolutions of Sentinel-2, that is, reconstruct the 20m band to 10m. However, the 10m spatial resolution is still difficult to extract small water bodies. In addition, methods that can reconstruct Sentinel-2 images to higher spatial resolutions such as 5m or 2m require the use of high-spatial-resolution commercial satellites as labels for training. However, these high-spatial-resolution image data can only provide fewer bands. Therefore, the 10m band of Sentinel-2 remote sensing images is mainly reconstructed, and the reconstruction results lack information in the 20m band, especially the short-wave infrared band data that is sensitive to water bodies. Summary of the invention

[0008] The purpose of the present invention is to solve the problem that the easily available Sentinel-2 remote sensing images have low spatial resolution and cannot accurately extract small water bodies. The present invention provides a water body extraction method, device and medium for super-resolution reconstruction of remote sensing images based on a combination of a generative adversarial network and a densely connected network based on the principle of image self-similarity, aiming to reconstruct both the 10m and 20m bands of Sentinel-2 images into images with higher (2.5m) spatial resolution so as to extract small water bodies with higher accuracy, and mainly includes the following steps: S1, remote sensing image data acquisition and preprocessing and data set preparation; S2. Design a remote sensing image super-resolution reconstruction model based on the combination of a generative adversarial network and a densely connected network based on the principle of image self-similarity; S3, use the dataset to train the remote sensing image super-resolution reconstruction model; S4, acquisition and preprocessing of remote sensing image data of the area to be measured; S5, performing super-resolution reconstruction of the original remote sensing image of the area to be measured based on the trained super-resolution reconstruction model to obtain a high spatial resolution reconstructed image; S6. Use the water index method to extract water bodies from the reconstructed image and produce a water body map of the area to be tested.

[0009] Furthermore, remote sensing image data, including: Sentinel-2 remote sensing images and high spatial resolution remote sensing images; Sentinel-2 remote sensing images are low spatial resolution data, including two spatial resolutions of 10m and 20m. The Sentinel-2 remote sensing images with a spatial resolution of 10m include four bands: red, green, blue and near-infrared, and the Sentinel-2 remote sensing images with a spatial resolution of 20m include six bands: red edge 1, red edge 2, red edge 3, red edge 4, shortwave infrared 1 and shortwave infrared 2.

[0010] Step S1 is specifically as follows: S11. Preprocessing the remote sensing image data, specifically: resampling and band combining the Sentinel-2 remote sensing images to obtain data with spatial resolutions of 10m and 20m, geometrically correcting the high spatial resolution remote sensing images, and resampling them to 2.5m; S12, align the preprocessed Sentinel-2 images with a spatial resolution of 10m and the high spatial resolution remote sensing images, and crop them into samples of 32×32 and 128×128 sizes respectively. Then filter the samples to filter out the samples with severe shadows and cloud images, as well as the samples with large differences between Sentinel-2 remote sensing images and high spatial resolution remote sensing images. Focus on selecting water-containing images, so that they account for 70% of the overall training samples, as the low-high spatial resolution sample data set 1; S13, the preprocessed Sentinel-2 remote sensing image 10m and 20m data are cropped into samples of 64×64 and 32×32 size, and the samples are downsampled to images with spatial resolution of 20m and 40m respectively using Gaussian filter. The original Sentinel-2 20m data is used as verification data and combined with the corresponding downsampled images to obtain the sample data set 2 with a spatial resolution of 2 times.

[0011] Furthermore, the remote sensing image super-resolution reconstruction model consists of two parts: Model A and Model B, specifically: Model A is a super-resolution reconstruction model based on a generative adversarial network. It inputs a Sentinel-2 remote sensing image with a spatial resolution of 10m and outputs a super-resolution reconstructed image with a spatial resolution of 2.5m. Model B is a densely connected network based on the principle of image self-similarity. It inputs high spatial resolution images and low spatial resolution images, whose spatial resolutions are twice that of each other. It transfers the high-frequency information of high spatial resolution data to guide the super-resolution reconstruction of low spatial resolution data, and reconstructs its spatial resolution to the same as that of high spatial resolution images. Among them, the high spatial resolution image and the low spatial resolution image are the 10m and 20m bands of Sentinel-2 remote sensing images, respectively.

[0012] Furthermore, step S3 is specifically as follows: S31, splitting both dataset 1 and dataset 2 into training sets and test sets according to different proportions, the training set is used to train model parameters, and the test set is used for model evaluation; S32. For data set 1, the high spatial resolution remote sensing images of the low-high spatial resolution sample pairs in the training set are used as the real verification data, and the 10m remote sensing images of Sentinel-2 are used as the input of model A, and the predicted images of model A are output; for data set 2, the original 20m remote sensing images of Sentinel-2 with a spatial resolution twice that of the sample pairs in the training set are used as the real verification data, and the 2x downsampled images corresponding to the 10m and 20m data are used as the input of model B, and the predicted images of model B are output; S33, calculate the mean absolute pixel error as the loss function between the real and predicted images, initialize the network weights to small random values ​​based on the loss function using the HeUniform method during training, and use stochastic gradient descent for optimization; S34, setting an iteration cycle, retaining the model at the end of each cycle, inputting the test set into the retained model and calculating the loss value, when the loss value converges, stopping the training, and using the retained model at this time as the trained remote sensing image super-resolution reconstruction model.

[0013] Furthermore, step S5 is specifically as follows: S51, input the preprocessed Sentinel-2 remote sensing image a with a spatial resolution of 10m into model A, output a super-resolution reconstructed image a with a spatial resolution of 2.5m, and downsample to obtain a remote sensing image a with a spatial resolution of 5m; S52, input the preprocessed 20m Sentinel-2 remote sensing image b into model B, and input the preprocessed remote sensing image a with a spatial resolution of 10m as auxiliary information, and guide model B to output a reconstructed image b with a spatial resolution of 10m. According to the same steps, input the remote sensing images b with spatial resolutions of 10m and 5m to be reconstructed, and use the remote sensing images a with spatial resolutions of 5m and 2.5m as auxiliary information, and use model B to perform two subsequent reconstructions, and output the super-resolution reconstructed image b with a spatial resolution of 2.5m; S53 , combining the super-resolution reconstructed image a with a spatial resolution of 2.5 m and the super-resolution reconstructed image b by bands, and outputting a reconstructed image of 10 bands of the area to be measured at a spatial resolution of 2.5 m.

[0014] Furthermore, step S6 is specifically as follows: S61, selecting a water index method for water body extraction to process the reconstructed image, calculating the water index value of each pixel in the area to be measured, and automatically calculating the water index threshold based on Otsu; S62, classifying the pixels according to the threshold value, the pixels with index values ​​greater than the threshold value are water bodies, and the rest are non-water bodies; S63, mapping the water body extraction result into a grayscale image to generate a water body image of the area to be tested.

[0015] Furthermore, step S6 further includes: The high spatial resolution image of the area to be tested is preprocessed, registered with the reconstructed image, and a high spatial resolution water body reference map is produced. Based on the water body reference map, the accuracy of the predicted water body map of the area to be tested is evaluated.

[0016] A medium, wherein the medium stores instructions and data for realizing a water body extraction method based on remote sensing image super-resolution reconstruction.

[0017] A computer device comprises: a processor and the medium; the processor loads and executes instructions and data in the medium to implement a water body extraction method based on remote sensing image super-resolution reconstruction.

[0018] The technical solution provided by the present invention has the following beneficial effects: the present invention proposes a remote sensing image super-resolution reconstruction model based on deep learning, guides the reconstruction of lower spatial resolution bands according to the self-similarity relationship of remote sensing images at different spatial resolutions, and better retains the original spectral information, so as to generate a more complete band and a more accurate reconstructed image, and obtain better results in the application of water body extraction and surface water monitoring. Among them, model A obtains the connection between the real high spatial resolution image and the 10m spatial resolution image of Sentinel-2, and reconstructs the 10m band to 2.5m; model B learns the spatial texture and spectral mixing relationship between the different bands of Sentinel-2 remote sensing images 10m and 20m, and reconstructs the 20m band. It solves the problems of low temporal resolution, high cost and difficulty in obtaining high spatial resolution remote sensing images, while the low spatial resolution remote sensing images that are easy to obtain are limited by spatial resolution and are not enough to extract smaller water bodies, so as to obtain better results in the application of water body extraction and surface water monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which: Figure 1 It is a technical roadmap of a water body extraction method based on super-resolution reconstruction of remote sensing images in an embodiment of the present invention; Figure 2 It is an experimental flow chart of a water body extraction method based on super-resolution reconstruction of remote sensing images in an embodiment of the present invention; Figure 3 is a schematic diagram of comparison between an original low spatial resolution image and a reconstructed image in an embodiment of the present invention; Figure 4 is a schematic diagram of comparing a water body map of a region to be tested predicted in an embodiment of the present invention with a water body reference map; Figure 5 It is a schematic diagram of the operation of the hardware device in the embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0021] The embodiments of the present invention provide a method, device and medium for water body extraction based on remote sensing image super-resolution reconstruction.

[0022] Please refer to Figure 1 , Figure 1 This is a technical roadmap of a water body extraction method based on remote sensing image super-resolution reconstruction in an embodiment of the present invention, which specifically includes the following steps: The first step is to acquire and preprocess remote sensing image data and create data sets.

[0023] Remote sensing image data, including: Sentinel-2 remote sensing images and high spatial resolution remote sensing images: Sentinel-2 remote sensing images are low spatial resolution data, including two spatial resolutions of 10m and 20m. The Sentinel-2 remote sensing images with a spatial resolution of 10m include four bands: red, green, blue and near-infrared, and the Sentinel-2 remote sensing images with a spatial resolution of 20m include six bands: red edge 1, red edge 2, red edge 3, red edge 4, shortwave infrared 1 and shortwave infrared 2.

[0024] The high spatial resolution remote sensing images used in the training are real SPOT high spatial resolution images, and their spatial resolution is resampled to the 2.5 m required for verification through preprocessing.

[0025] The specific implementation steps of the first step are: Step 1: Preprocess the remote sensing image data, specifically: resample and combine the bands of Sentinel-2 remote sensing images to obtain data with spatial resolutions of 10m and 20m, perform geometric correction on high spatial resolution remote sensing images, and resample to 2.5m.

[0026] In order to obtain better results, the time of acquiring experimental data and verification data should be as close as possible.

[0027] Specifically, we first clarify the specific requirements of the required Sentinel-2 remote sensing images based on the obtained verification data, including the geographical scope and time range, which helps to narrow the search scope and find data that meets the needs; then, based on the ESA's Copernicus data platform, we use keywords, geographical location, time range, cloud range and other conditions to search in the data source to screen out remote sensing image data that meets the needs.

[0028] For data that meets your needs, you can use the preview function provided by the data source to view the thumbnail of the image or some sample data to further confirm whether it meets your requirements; once you have confirmed the required remote sensing image data, you can choose to download it.

[0029] Since the Sentinel-2 data used in the implementation is L2a data, it has been processed by radiometric calibration, atmospheric correction and geometric correction. Therefore, the preprocessing of the downloaded data during the implementation mainly includes: resampling and band combination of Sentinel-2 data, geometric correction and resampling of high spatial resolution remote sensing images.

[0030] Step 2: The preprocessed Sentinel-2 images with a spatial resolution of 10m and high spatial resolution remote sensing images are registered and cropped into samples of 32×32 and 128×128 sizes, respectively. The samples are then screened to filter out samples with severe shadows and cloud images, as well as samples with large differences between Sentinel-2 remote sensing images and high spatial resolution remote sensing images. Water-containing images are selected so that they account for 70% of the overall training samples as the low-high spatial resolution sample dataset 1.

[0031] Step 3: The preprocessed Sentinel-2 remote sensing image 10m and 20m data are cropped into samples of 64×64 and 32×32 size, and the samples are downsampled to images with spatial resolution of 20m and 40m using Gaussian filter. The original Sentinel-2 20m data is used as the verification data and combined with the corresponding downsampled images to obtain the sample dataset 2 with a spatial resolution of 2 times.

[0032] The second step is to design a remote sensing image super-resolution reconstruction model based on a combination of a generative adversarial network and a densely connected network based on the principle of image self-similarity.

[0033] The remote sensing image super-resolution reconstruction model consists of two parts: Model A and Model B, specifically: Model A is a super-resolution reconstruction model based on a generative adversarial network. By using real SPOT high-spatial-resolution remote sensing images as verification data to fuse real high-frequency information, Model A has a good reconstruction effect on 10m Sentinel-2 data and can reconstruct 4 bands to 2.5m; Model B is a densely connected network based on the principle of image self-similarity. It can guide the super-spatial resolution process of low-spatial-resolution bands by accessing high-spatial-resolution bands, thereby transferring high-frequency information of high-spatial-resolution bands to low-spatial-resolution input bands. To achieve this goal, bilinear interpolation is first used to upsample the low-spatial-resolution band to the same spatial resolution as the high-spatial-resolution band and concatenate it with the high-spatial-resolution data. Then, convolutional layers, RELU nonlinear activation functions, and jump connections are used for calculation, where the convolutional layer size is 128×128 and the convolution kernel is 3×3. There is a jump connection from the bilinear interpolation input directly to the output in the network architecture, which means that the complete network actually learns how to get the residual information from the bilinear interpolation image to the desired output. Among them, in order to strengthen the extracted features, a convolutional block attention module CbaM that mixes the spatial attention mechanism and the channel attention mechanism is added to the residual module to better learn the self-similarity between bands. By learning the transfer of spatial details between bands of different spatial resolutions, model B can establish a self-similar relationship between bands of different spatial resolutions. Because the spatial resolution of the original Sentinel-2 data to be reconstructed is 10m and 20m, model B learns a 2-fold similarity relationship between low- and high-spatial-resolution images, and in the reconstruction process of lower spatial-resolution data (original 20m Sentinel-2 band data), higher spatial-resolution data (the original 10m Sentinel-2 4 bands) are required as auxiliary information.

[0034] The original Sentinel-2 high and low spatial resolution data were downsampled to 20m and 40m respectively, and then verified with the original low spatial resolution data (20m) to learn the transfer of spatial details between bands of different spatial resolutions, so that model B can establish a self-similar relationship between bands with 2 times the spatial resolution.

[0035] The third step is to use the data set to train the remote sensing image super-resolution reconstruction model, specifically: Step 1: Split both dataset 1 and dataset 2 into training sets and test sets in different proportions. The training set is used to train model parameters, and the test set is used for model evaluation.

[0036] Step 2: For dataset 1, the high spatial resolution remote sensing images of the low-high spatial resolution sample pairs in the training set are used as real verification data, and the 10m remote sensing images of Sentinel-2 are used as the input of model A, and the predicted images of model A are output.

[0037] For dataset 2, the original Sentinel-2 20m remote sensing images with a spatial resolution twice that of the sample pairs in the training set are used as real verification data, and the 2x downsampled images corresponding to the 10m and 20m data are used as the input of model B, and the predicted images of model B are output.

[0038] Step 3: Calculate the mean absolute pixel error (L1 norm) as the loss function between the real and predicted images. During the training process, the network weights are initialized to small random values ​​using the HeUniform method based on the loss function and optimized using stochastic gradient descent. Step 4: Set the iteration cycle, retain the model at the end of each cycle, input the test set to the retained model and calculate the loss value. When the loss value converges, stop training and use the retained model at this time as the trained remote sensing image super-resolution reconstruction model.

[0039] The fourth step is to acquire and preprocess the remote sensing image data of the area to be measured.

[0040] On the basis of obtaining the training model, the specific implementation of the embodiment can be completed. For the experimental flow chart of super-resolution reconstruction and water extraction of the remote sensing image of the area to be measured in the embodiment, please refer to Figure 2 .

[0041] Before implementing the method provided by the present invention, it is necessary to prepare and preprocess the data in advance, and select the implementation area of ​​the experiment: Experimental area of ​​the present invention: This method has strong generalization and does not need to specify a special area. In order to verify the advancement of the model, the present invention selects a large number of small water bodies with different water body boundary types as experimental samples; The experimental data used are Sentinel-2 data, with the spatial resolution of the bands used being 10m and 20m. The validation data used are GF-2 data, with its spatial resolution resampled to the 2.5m required for validation through preprocessing.

[0042] The fifth step is to perform super-resolution reconstruction of the original remote sensing image of the measured area based on the trained super-resolution reconstruction model to obtain a high spatial resolution reconstructed image. Specifically: Step 1: Input the preprocessed Sentinel-2 remote sensing image a with a spatial resolution of 10m into model A, output the super-resolution reconstructed image a with a spatial resolution of 2.5m, and downsample it to obtain the remote sensing image a with a spatial resolution of 5m.

[0043] Step 2: Input the preprocessed 20m Sentinel-2 remote sensing image b into model B, and input the preprocessed remote sensing image a with a spatial resolution of 10m as auxiliary information to guide model B to output a reconstructed image b with a spatial resolution of 10m. Follow the same steps to input the images b with spatial resolutions of 10m and 5m to be reconstructed, and use the remote sensing image a with spatial resolutions of 5m and 2.5m as auxiliary information, and use model B to perform two subsequent reconstructions to output a super-resolution reconstructed image b with a spatial resolution of 2.5m.

[0044] For detailed process, please refer to Figure 2 , Figure 2 This is an experimental flow chart of the water body extraction method based on super-resolution reconstruction of remote sensing images in an embodiment of the present invention. The preprocessed Sentinel-2 remote sensing image with a spatial resolution of 10m is used as data a, and the preprocessed Sentinel-2 remote sensing image with a resolution of 20m is used as data b. The reason for performing three reconstructions in this step is that the data of Sentine-2 for training are 20m and 40m, and their spatial resolutions are a 2-fold relationship. Therefore, the self-similarity relationship learned in model B is an auxiliary when upsampling the bands of the downsampling result information pair of the 10m data a and the 20m data b:

[0045] Among them, W and H represent the height and width of the sample data, respectively. It has been verified that although some high-frequency information may be missing, this 2-fold relationship can be transferred to the upsampling of data with higher spatial resolution, as long as the band information of data a and b remains unchanged. Therefore, using the band information of data a with higher spatial resolution obtained by model A, the spatial resolution of data b can be reconstructed to 10m, 5m and 2.5m in turn, thereby obtaining multispectral information with higher spatial resolution. In the first reconstruction process, the input of the model is 20m data b and 10m data a, and the reconstruction result is 10m data b. According to the model, all bands of data b are first upsampled to 10m based on the bilinear interpolation method. Then the upsampled data is spliced ​​and combined with the 10m data a, and calculated through network structures such as convolution and residual, and the corresponding relationship learned through training is used to transfer high-frequency details from the 10m data a to obtain the residual information corresponding to data b. Finally, the upsampled data b is added to the residual information to obtain the reconstruction result of data b at 10m. Similarly, based on the two-fold similarity relationship of the model, two subsequent reconstructions can be performed using the 5m and 2.5m data a as auxiliary information, respectively. The input data b are the first and second reconstruction results, respectively.

[0046] Step 3: Combine the super-resolution reconstructed image a with a spatial resolution of 2.5 m and the super-resolution reconstructed image b to output a water body reconstructed image of 10 bands at a spatial resolution of 2.5 m in the test area.

[0047] The comparison diagram of the original low spatial resolution image and the reconstructed image in the embodiment of the present invention can be referred to Figure 3 The reconstruction effects of the method of the present invention are given by taking the 10m near-infrared band (Band 8) and the 20m short-wave infrared band 1 (Band 11) as examples. Figure 3 The subscripts (a) and (b) in the figure indicate the original image and reconstructed image of the near-infrared band, respectively, and the subscripts (c) and (d) indicate the original image and reconstructed image of the short-wave infrared band, respectively. Figure 3 From the large-scale image in the first row, the difference between the original and reconstructed images is not obvious, but according to the local enlarged image in the second row, it can be seen that the reconstruction result restores more image details. Especially in the short-wave infrared 1 band, the edges of the original image are blurred and the jagged phenomenon of the island outline is serious, while the reconstruction result has a clearer boundary. It shows that the method provided by the present invention can significantly improve the image accuracy in the embodiment.

[0048] The sixth step is to use the water index method to extract water bodies from the reconstructed image and produce a water body map of the area to be tested, specifically: Step 1: Select the water index method for water body extraction to process the reconstructed image, calculate the water index value of each pixel in the test area, and automatically calculate the water index threshold based on Otsu.

[0049] Select the water index method for water body extraction, and calculate the water index value of each pixel in the test area based on the above reconstructed image; commonly used water indexes include NDWI, MNDWI and EWI, and the calculation method is as follows:

[0050] Among them, RED, GREEN, NIR, and SWIR1 are the 10m red (Band 4), green (Band 3), near infrared band (Band 8) and 20m shortwave infrared 1 band (Band 11) of the Sentinel-2 remote sensing image, respectively. In this embodiment, the widely used MNDWI index method is selected for calculation.

[0051] Step 2: Classify the pixels according to the threshold. The pixels with index values ​​greater than the threshold are water bodies, and the rest are non-water bodies.

[0052] Step three, map the water body extraction results into a grayscale image to generate a water body map of the area to be tested.

[0053] Step 6 also includes: The high spatial resolution image of the area to be tested is preprocessed, registered with the reconstructed image, and a high spatial resolution water body reference map is produced. Based on the water body reference map, the accuracy of the predicted water body map of the area to be tested is evaluated.

[0054] This step is to register the reconstructed image with the high spatial resolution GF-2 image. This is because if it is registered with the original image, the spatial resolution of the rough original image is 4 or even 8 times that of the original image, which affects the accuracy of the registration result and may still have a large error.

[0055] This step uses GF-2 data as the basic data, and performs projection transformation and spatial registration on the super-resolution reconstructed Sentinel-2 data, with the spatial error controlled within 0.5 pixels. Because a larger range of images is used for reconstruction, this step also requires cropping the reconstructed image based on the cropped GF-2 image for which a water body reference map has been prepared.

[0056] Figure 4The water body extraction results of two samples of the embodiment of the present invention are given. The subscript (a) indicates a water body reference map produced by high spatial resolution images, the subscript (b) indicates that the image is a water body map of the test area extracted based on the original image, and the subscript (c) indicates that the image is a water body map of the test area extracted based on the reconstructed image. Since the original Sentinel-2 image has a low spatial resolution, the boundary of the corresponding extracted water body is obviously jagged. The water body boundary extracted by the image reconstructed by the method provided by the present invention is smoother, the extraction effect is better, and the accuracy is higher. The extracted water body is also more similar to the reference image.

[0057] Calculate the error between the predicted water body map and the water body reference map. In the embodiment of the present invention, the overall accuracy (OA), producer accuracy (PA), user accuracy (UA) and peak signal-to-noise ratio (PSNR) are used to evaluate the accuracy of water body extraction results based on super-resolution reconstruction of remote sensing images. OA, PA and UA are all indicators used to measure classification accuracy. OA can be obtained by comparing the correct prediction results of each category with the total number of labels, PA is obtained by calculating the ratio of a certain category of correct prediction results to the true labels of that category, and UA is obtained by calculating the ratio of a certain category of correct prediction results to the total number of prediction results of that category. According to the water body prediction accuracy of the two-classification results in this embodiment, the calculation formulas of OA, PA and UA are as follows:

[0058] Among them, TN is the number of correctly predicted non-water samples, TP is the number of correctly predicted water samples, FN is the number of incorrectly predicted non-water samples, and FP is the number of incorrectly predicted water samples. If n is the total number of samples, we can get The values ​​of OA, PA and UA are all between 0 and 1. The closer they are to 1, the better the accuracy.

[0059] PSNR is usually used to evaluate the quality of lossy images. It is an engineering measure that represents the ratio between the maximum signal and the introduced background noise, as well as the mean square error between the image and the signal. Since most signals have a large range of dynamic changes, logarithmic decibels are usually used to represent PSNR. PSNR mainly evaluates the similarity of images based on the difference in pixel values ​​between two different images. It is an error-based evaluation indicator. The larger the PSNR value, the more lossless it is. The calculation formula is as follows:

[0060] Wherein, H and W are respectively the height and width of the predicted sample data in the embodiment, and X and Y represent the predicted result and the true label respectively.

[0061] The embodiments of the present invention are implemented on a large number of experimental samples with different water body boundary types, and the accuracy results calculated according to the above accuracy evaluation method are shown in Table 1.

[0062] Table 1 Average precision values ​​in the examples

[0063] According to Table 1, the method provided by the present invention improves the water extraction accuracy of the experimental data in the embodiment.

[0064] See also Figure 5 , Figure 4 It is a schematic diagram of the working of the hardware device of an embodiment of the present invention, and the hardware device specifically includes: a computer device 401, a processor 402 and a medium 403.

[0065] A computer device 401: The computer device 401 implements the water body extraction method based on remote sensing image super-resolution reconstruction.

[0066] Processor 402: The processor 402 loads and executes the instructions and data in the medium 403 to implement the water body extraction method based on remote sensing image super-resolution reconstruction.

[0067] Medium 403: The medium 403 stores instructions and data; the medium 403 is used to implement the water body extraction method based on remote sensing image super-resolution reconstruction.

[0068] The beneficial effects of the present invention are as follows: the present invention proposes a remote sensing image super-resolution reconstruction model based on deep learning, guides the reconstruction of lower spatial resolution bands according to the self-similarity relationship of remote sensing images at different spatial resolutions, and better retains the original spectral information, so as to generate a more complete band and a more accurate reconstructed image, and obtain better results in the application of water body extraction and surface water monitoring. Among them, model A obtains the connection between the real high spatial resolution image and the 10m spatial resolution image of Sentinel-2, and reconstructs the 10m band to 2.5m; model B learns the spatial texture and spectral mixing relationship between the different bands of Sentinel-2 remote sensing images 10m and 20m, to reconstruct the 20m band. It solves the problems of low temporal resolution, high cost and difficulty in obtaining high spatial resolution remote sensing images, while the low spatial resolution remote sensing images that are easy to obtain are limited by spatial resolution and are not enough to extract smaller water bodies, so as to obtain better results in the application of water body extraction and surface water monitoring.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A water body extraction method based on remote sensing image super-resolution reconstruction, characterized in that: The specific steps include: S1, remote sensing image data acquisition and preprocessing and data set preparation; S2. Design a remote sensing image super-resolution reconstruction model based on the combination of a generative adversarial network and a densely connected network based on the principle of image self-similarity; S3, use the dataset to train the remote sensing image super-resolution reconstruction model; S4, acquisition and preprocessing of remote sensing image data of the area to be measured; S5, performing super-resolution reconstruction of the original remote sensing image of the area to be measured based on the trained super-resolution reconstruction model to obtain a high spatial resolution reconstructed image; S6. Use the water index method to extract water bodies from the reconstructed image and produce a water body map of the area to be tested.

2. The water body extraction method based on remote sensing image super-resolution reconstruction as claimed in claim 1, characterized in that: The remote sensing image data includes: Sentinel-2 remote sensing images and high spatial resolution remote sensing images; Sentinel-2 remote sensing images are low spatial resolution data, including two spatial resolutions of 10m and 20m. The Sentinel-2 remote sensing images with a spatial resolution of 10m include four bands: red, green, blue and near-infrared, and the Sentinel-2 remote sensing images with a spatial resolution of 20m include six bands: red edge 1, red edge 2, red edge 3, red edge 4, shortwave infrared 1 and shortwave infrared 2.

3. The water body extraction method based on remote sensing image super-resolution reconstruction as claimed in claim 2, characterized in that: Step S1 is specifically as follows: S11. Preprocessing the remote sensing image data, specifically: resampling and band combining the Sentinel-2 remote sensing images to obtain data with spatial resolutions of 10m and 20m, geometrically correcting the high spatial resolution remote sensing images, and resampling them to 2.5m; S12, align the preprocessed Sentinel-2 images with a spatial resolution of 10m and the high spatial resolution remote sensing images, and crop them into samples of 32×32 and 128×128 sizes respectively. Then filter the samples to filter out the samples with severe shadows and cloud images, as well as the samples with large differences between Sentinel-2 remote sensing images and high spatial resolution remote sensing images. Focus on selecting water-containing images, so that they account for 70% of the overall training samples, as the low-high spatial resolution sample data set 1; S13. The preprocessed Sentinel-2 remote sensing image 10m and 20m data are cropped into samples of 64×64 and 32×32 sizes, respectively, and the samples are downsampled to images with spatial resolutions of 20m and 40m using a Gaussian filter. The original Sentinel-2 20m data is used as verification data and combined with the corresponding downsampled images to obtain a sample dataset 2 with a spatial resolution of 2 times.

4. The water body extraction method based on remote sensing image super-resolution reconstruction as claimed in claim 3, characterized in that: The remote sensing image super-resolution reconstruction model consists of two parts: Model A and Model B, specifically: Model A is a super-resolution reconstruction model based on a generative adversarial network. It inputs a Sentinel-2 remote sensing image with a spatial resolution of 10m and outputs a super-resolution reconstructed image with a spatial resolution of 2.5m. Model B is a densely connected network based on the principle of image self-similarity. It inputs high spatial resolution images and low spatial resolution images, whose spatial resolutions are twice that of each other. It guides the super-resolution reconstruction of low spatial resolution data by transferring the high-frequency information of high spatial resolution data, and reconstructs its spatial resolution to the same as that of high spatial resolution images. Among them, the high spatial resolution image and the low spatial resolution image are the 10m and 20m bands of Sentinel-2 remote sensing images, respectively.

5. The method for water body extraction based on remote sensing image super-resolution reconstruction as claimed in claim 4, characterized in that: Step S3 is specifically as follows: S31, splitting both dataset 1 and dataset 2 into training sets and test sets according to different proportions, the training set is used to train model parameters, and the test set is used for model evaluation; S32. For data set 1, the high spatial resolution remote sensing images of the low-high spatial resolution sample pairs in the training set are used as the real verification data, and the 10m remote sensing images of Sentinel-2 are used as the input of model A, and the predicted images of model A are output; for data set 2, the original 20m remote sensing images of Sentinel-2 with a spatial resolution twice that of the sample pairs in the training set are used as the real verification data, and the 2x downsampled images corresponding to the 10m and 20m data are used as the input of model B, and the predicted images of model B are output; S33, calculate the mean absolute pixel error as the loss function between the real and predicted images, initialize the network weights to small random values ​​based on the loss function using the HeUniform method during training, and use stochastic gradient descent for optimization; S34, setting an iteration cycle, retaining the model at the end of each cycle, inputting the test set into the retained model and calculating the loss value, when the loss value converges, stopping the training, and using the retained model at this time as the trained remote sensing image super-resolution reconstruction model.

6. The method for water body extraction based on remote sensing image super-resolution reconstruction as claimed in claim 5, characterized in that: Step S5 is specifically as follows: S51, input the preprocessed Sentinel-2 remote sensing image a with a spatial resolution of 10m into model A, output a super-resolution reconstructed image a with a spatial resolution of 2.5m, and downsample to obtain a remote sensing image a with a spatial resolution of 5m; S52, input the preprocessed 20m Sentinel-2 remote sensing image b into model B, and input the preprocessed remote sensing image a with a spatial resolution of 10m as auxiliary information, and guide model B to output a reconstructed image b with a spatial resolution of 10m; according to the same steps, input the remote sensing images b with spatial resolutions of 10m and 5m to be reconstructed respectively, and use the remote sensing images a with spatial resolutions of 5m and 2.5m as auxiliary information respectively, use model B to perform subsequent reconstruction twice, and finally output the super-resolution reconstructed image b with a spatial resolution of 2.5m; S53 , combining the super-resolution reconstructed image a with a spatial resolution of 2.5 m and the super-resolution reconstructed image b by bands, and outputting a reconstructed image of 10 bands of the area to be measured at a spatial resolution of 2.5 m.

7. The method for water body extraction based on remote sensing image super-resolution reconstruction as claimed in claim 6, characterized in that: Step S6 is specifically as follows: S61, selecting a water index method for water body extraction to process the reconstructed image, calculating the water index value of each pixel in the area to be measured, and automatically calculating the water index threshold based on Otsu; S62, classifying the pixels according to the threshold value, the pixels with index values ​​greater than the threshold value are water bodies, and the rest are non-water bodies; S63, mapping the water body extraction result into a grayscale image to generate a water body image of the area to be tested.

8. The method for water body extraction based on remote sensing image super-resolution reconstruction as claimed in claim 7, characterized in that: Step S6 also includes: The high spatial resolution image of the area to be tested is preprocessed, registered with the reconstructed image, and a high spatial resolution water body reference map is produced. Based on the water body reference map, the accuracy of the predicted water body map of the area to be tested is evaluated.

9. A medium, characterized in that: The medium stores instructions and data for implementing a water body extraction method based on remote sensing image super-resolution reconstruction as described in any one of claims 1 to 8.

10. A computer device, characterized in that: include: Processor and medium; the processor loads and executes instructions and data in the medium to implement a water body extraction method based on super-resolution reconstruction of remote sensing images as described in any one of claims 1 to 8.

Images