Water body extraction method, system and device based on conditional random field and sub-pixel positioning

Abstract

The invention discloses a water body extraction method, system and device based on a conditional random field and sub-pixel positioning, and relates to the technical field of remote sensing image processing. The method comprises the following steps: preprocessing data to be extracted to obtain low-resolution original data, and inputting the low-resolution original data into a generator network in a super-resolution generative adversarial network model to obtain corresponding high-resolution image data subjected to super-resolution reconstruction; carrying out image binarization processing and connected domain processing on the high-resolution image data, and removing noise to obtain a potential region of a water body; expanding the boundary of the potential area of the water body according to a preset threshold value, and generating a super-resolution image of the lake buffer area; and segmenting the super-resolution image of the lake buffer area into a plurality of super-pixel objects to obtain a final water body extraction result. According to the method for extracting the water body in the Antarctic Rastman hilly area, the influence of the stability of ice covers, hydrological cycle, ecological system health conditions and global climate changes of the Antarctic can be deeply understood.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a method, system and device for extracting water based on conditional random fields and sub-pixel positioning. Background Art

[0002] Affected by global climate change, the natural environment in the Antarctic region has undergone drastic changes. The most obvious change is that the rate of increase in water temperature in the Antarctic region is much higher than the average level of water temperature change around the world. The phenomenon of climate warming is extremely prominent in the Antarctic water body. In this regard, the thickness of the Antarctic ice sheet has decreased, the area of ​​glaciers has shrunk rapidly, and many ice shelves have begun to melt continuously. Due to the impact of human activities, the water quality of the water bodies in the Antarctic region is also deteriorating, which has caused serious damage to the ecosystem in the Antarctic region and has further had a far-reaching impact on global climate change.

[0003] The large-scale and high-speed changes in Antarctic water bodies not only disrupt the overall operation mode of the global water cycle, but also cause a series of problems such as changes in the distribution of water bodies in various places and rising global sea levels. Changes in the water environment and climate in the Antarctic region have also changed the movement and distribution patterns of global heat and gas, leading to increased instability in the global climate, and thus increasing the risk of regional or global natural disasters. Therefore, changes in water bodies in the Antarctic region have attracted great attention from all walks of life in the international community and have become one of the core topics of current research.

[0004] The climate conditions in the Antarctic region are extreme, with extremely low temperatures and strong winds, which not only limit the possibility of field surveys, but also place extremely high demands on the acquisition and processing of remote sensing data. Lakes on the surface of the ice sheet are usually formed in the summer. The lake area is relatively small and unevenly distributed. The appearance and disappearance of lakes have large temporal and spatial variability, which makes continuous monitoring difficult to achieve. The morphology and physical properties of lakes are also affected by factors such as ice sheet movement, meltwater supply and discharge, and climate change, which further increase the difficulty of lake extraction.

[0005] The formation and change of lakes have strong spatiotemporal dynamic characteristics, and remote sensing data needs to have sufficiently high spatiotemporal resolution to accurately capture the change process of lakes. However, due to the orbital design of remote sensing satellites and the limitation of observation time, it is difficult to obtain remote sensing data with high spatiotemporal resolution. Especially in the Antarctic region, the satellite transit frequency is low and the data acquisition cycle is long, which makes continuous monitoring of lake change process difficult.

[0006] The Larsmann Hills are located in the coastal area of ​​East Antarctica. This special geographical location makes it a key area for studying the exchange of materials and energy between the Antarctic continent and the Southern Ocean. The strong ice and wind erosion of the Larsmann Hills has caused the exposed bedrock on the local surface to have strange shapes such as honeycomb. Due to the complex and changeable local climate characteristics and the large temperature difference in a year, the distribution of lake water bodies in the Larsmann Hills is complex, scattered and changeable. Due to its special geological and climatic conditions, the Larsmann Hills area has become an important node in the internal water cycle of the Antarctic continent. The change in the area of ​​the Larsmann Lakes has become an important indicator for the study of climate change and ecological environment change. With the increasing abundance of high-resolution remote sensing observation data and the improvement of processing technology, the use of remote sensing technology to conduct long-term monitoring of lakes, rivers and glacier meltwater in the region can provide a deep understanding of the dynamic changes of ice sheet meltwater and its impact on the surrounding sea areas, and can more effectively monitor the area change information of more small lakes, revealing the key processes of inland water bodies in Antarctica such as recharge, flow, evaporation and freeze-thaw; it is helpful to build a more accurate hydrological model of Antarctica, thereby improving the prediction ability of global climate models. Remote sensing monitoring can also help us identify possible groundwater flow paths and the distribution of subglacial water bodies, which is of great scientific value for understanding the hydrodynamic processes at the bottom of the ice sheet and the mechanism of ice sheet movement. Summary of the invention

[0007] The present invention provides a water extraction method, system and device based on conditional random fields and sub-pixel positioning, taking lakes and other water bodies in the Larsmann Hills region of Antarctica as representative research objects, and studying the problem of extracting water bodies from lakes with small spatial scale and large seasonal differences. The present invention can not only help researchers gain a deeper understanding of the stability of the Antarctic ice sheet, the hydrological cycle, the health of the ecosystem and the impact of global climate change, but also provide key scientific support for responding to global climate change and environmental protection. It is specifically achieved through the following technologies.

[0008] In a first aspect of the present invention, a method for extracting water bodies based on conditional random fields and sub-pixel positioning is provided, the steps of which include: preprocessing the image data of the water body to be extracted to obtain low-resolution original data;

[0009] Constructing a super-resolution generative adversarial network model, inputting the low-resolution original data into the generator network of the super-resolution generative adversarial network model for processing, and scoring and discriminating through the discriminator network to obtain the corresponding high-resolution image data after super-resolution reconstruction;

[0010] Performing image binarization processing on the high-resolution image data, extracting the water body potential area, and removing noise to obtain the water body potential area; expanding the boundary of the water body potential area according to a preset threshold value to generate a water body buffer zone super-resolution image;

[0011] The water body buffer super-resolution image is segmented into several super-pixel objects, a conditional random field model is constructed, and the super-pixel objects are segmented into water bodies and non-water bodies based on the conditional random field model and the minimum energy function loss criterion to obtain the final water body extraction result.

[0012] Furthermore, the method for preprocessing the water body image data to be extracted includes: performing atmospheric correction on the water body image data to be extracted, converting the zenith reflectivity of the L1C data into the L2A ground surface reflectivity, and obtaining the low-resolution original data.

[0013] Furthermore, the super-resolution generative adversarial network model includes a generator network, a discriminator network and a loss function; the generator network includes a residual block unit and a sub-phase element convolution unit;

[0014] The training method of the super-resolution generative adversarial network model includes:

[0015] Input the low-resolution original data into the generator network, perform two-dimensional convolution processing with a convolution kernel of 9×9 and a first activation function processing, and complete initial feature extraction;

[0016] The image data after initial feature extraction is input into the residual unit for several times of 3×3 two-dimensional convolution processing, normalization processing and first activation function processing;

[0017] The image data after the initial feature extraction skips the processing of the residual block unit and directly performs a first element-level addition with the image data after the processing of the residual block unit;

[0018] The image data obtained by the first element-level addition is subjected to a two-dimensional convolution with a convolution kernel of 3×3 and normalized to obtain convolved image data;

[0019] The image data after initial feature extraction skips the residual block unit process and performs a second element-level addition with the convolution image data;

[0020] The image data after the second element-level addition processing is input into the sub-pixel convolution unit, and the convolution kernel 3×3 two-dimensional convolution, pixel mapping, activation function processing, and convolution kernel 9×9 two-dimensional convolution are performed in sequence to obtain a high-resolution image;

[0021] The discriminator network is used to score and judge the generated high-resolution image to confirm the authenticity of the image.

[0022] Furthermore, the method of performing image binarization processing and connected domain processing on the high-resolution image data to remove noise and obtain the potential area of ​​the water body includes:

[0023] Select a scene using the high-resolution image data generated by the super-resolution generative adversarial network model, calculate the normalized water index, mark the area with a normalized water index <0.05 as a water body, and mark the area with a normalized water index ≥0.05 as a non-water body, to obtain a binary image;

[0024] The connected domain processing method is used for the binary image to calculate the area of ​​each water body connected domain; the water body connected domains with an area smaller than the noise area threshold are marked as noise and filtered out; and the remaining water body connected domains are marked as potential lake areas.

[0025] Furthermore, for each of the water body connected domains in the potential area of ​​the lake, the lake boundary is expanded according to a preset threshold value to generate a super-resolution image of the lake buffer zone.

[0026] Furthermore, the method of obtaining the final water body extraction result by using the lake buffer super-resolution image includes:

[0027] Segmenting the super-resolution image of the lake buffer into different super-pixel objects;

[0028] Constructing a conditional random field model, wherein a unit potential energy function in the conditional random field model is composed of the band reflectance of the optical data and its corresponding GLCM features, and the potential energy is composed of the cross-correlation functions of different superpixel objects;

[0029] Based on the conditional random field model and the minimum energy function loss criterion, the superpixel object is segmented into active water body and non-water body areas to obtain the final water body extraction result.

[0030] The second aspect of the present invention provides a water body extraction system based on conditional random fields and sub-pixel positioning, including an original image acquisition and preprocessing module, a super-resolution reconstruction module, and a lake water body extraction module;

[0031] The original image acquisition and preprocessing module is used to receive the image data of the water body to be extracted, and preprocess the image data of the water body to be extracted to obtain low-resolution original data;

[0032] The super-resolution reconstruction module is used to construct and train a super-resolution generative adversarial network model, input the low-resolution original data and output the corresponding high-resolution image data;

[0033] The lake water body extraction module is used to perform image binarization and connected domain processing on the high-resolution image data to remove noise and obtain potential water areas; expand the boundaries of the potential water areas to generate super-resolution images of water buffer zones; segment the super-resolution images of water buffer zones to obtain a number of super-pixel objects, and based on a conditional random field model and a minimum energy function loss criterion, perform super-pixel segmentation on the super-pixel objects into water bodies and non-water bodies to obtain a final water body extraction result.

[0034] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is used to enable a computer to execute the above-mentioned water body extraction method based on conditional random fields and sub-pixel positioning.

[0035] In a fourth aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it is used to implement the above-mentioned water body extraction method based on conditional random fields and sub-pixel positioning.

[0036] Compared with the prior art, the present invention is beneficial in that:

[0037] 1. The present invention utilizes a super-resolution generative adversarial network model to achieve super-resolution reconstruction of optical images and improve the ability to describe lake boundary features. On this basis, the conditional random field model of the water buffer zone is used to extract lake water based on the minimum energy function loss criterion.

[0038] 2. The method for extracting water from the Larsmann Hills region of Antarctica obtained by the present invention can not only provide a deep understanding of the stability of the Antarctic ice sheet, the hydrological cycle, the health of the ecosystem and the impact of global climate change; it can also provide key scientific support for responding to global climate change and environmental protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 The present invention provides an overall flow chart of the water extraction method provided in the embodiment of the present invention.

[0040] Figure 2 In the water body extraction method provided in the embodiment of the present invention, the generator network of the super-resolution generative adversarial network model is used to reconstruct the low-resolution original data with super-resolution, and finally a flowchart of high-resolution image data is obtained.

[0041] Figure 3 A flowchart of using a discriminator network of a super-resolution generative adversarial network model to discriminate the authenticity of an input image in a water body extraction method provided in an embodiment of the present invention.

[0042] Figure 4A flow chart of obtaining a final water body extraction result by using the high-resolution image data in the water body extraction method provided in an embodiment of the present invention.

[0043] Figure 5 Comparison chart of water extraction results using different algorithms. DETAILED DESCRIPTION

[0044] The technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] like Figure 1 As shown, the water body extraction method based on conditional random fields and sub-pixel positioning provided by the present invention has the following general steps: pre-processing the water body image data to be extracted to obtain low-resolution original data;

[0046] Constructing a super-resolution generative adversarial network model, inputting the low-resolution original data into the generator network of the super-resolution generative adversarial network model for processing, and scoring and discriminating through the discriminator network to obtain the corresponding high-resolution image data after super-resolution reconstruction;

[0047] Performing image binarization processing on the high-resolution image data, extracting the water body potential area, and removing noise to obtain the water body potential area; expanding the boundary of the water body potential area according to a preset threshold value to generate a water body buffer zone super-resolution image;

[0048] The water body buffer super-resolution image is segmented into a number of super-pixel objects, a conditional random field model is constructed, and based on the conditional random field model and the minimum energy function loss criterion, the super-pixel objects are segmented into water bodies and non-water bodies to obtain the final water body extraction result.

[0049] Example

[0050] Sentinel-2, also known as Sentinel-2 satellite, is a high-resolution multispectral imaging satellite carrying a multispectral imager (MSI) for land monitoring. It can provide images of vegetation, soil and water cover, inland waterways and coastal areas, and can also be used for emergency rescue services. This embodiment uses the Antarctic Larsmann Hills Lake in the Sentinel-2 optical image as the object to extract the Larsmann Hills Lake. The Sentinel-2A satellite was launched by the European Space Agency in 2015, and the Sentinel-2B was launched in 2016. It can be downloaded and used for free (https: / / dataspace.copernicus.eu / ).

[0051] Step 1: Data preprocessing

[0052] Sentinel-2 optical images contain two types of data, L1C and L2A, of which L1C data has not been atmospherically corrected. In order to eliminate the influence of the atmosphere on lake extraction, it is necessary to convert the zenith reflectance of the L1C data into the L2A surface reflectance (i.e., perform atmospheric correction) to obtain the pre-processed Sentinel-2 data, i.e., the original L2A surface reflectance (resolution 10m).

[0053] Optionally, the process is implemented using Sen2cor software.

[0054] Step 2: Use the super-resolution generative adversarial network model to perform super-resolution processing on the pre-processed low-resolution impact data of Sentinel-2 to obtain high-resolution image data.

[0055] The main steps include: building a super-resolution generative adversarial network model (specifically a generator network, a discriminator network, and a loss function); inputting the Sentinel-2 data (original L2A surface reflectivity) preprocessed in step 1 as the training set and test set, and using the generator network, the discriminator network, and the loss function to train and test the super-resolution generative adversarial network model to obtain the optimal super-resolution generative adversarial network model.

[0056] In actual application, the same method as above is adopted to input the pre-processed Sentinel-2 data to be tested into the model, perform super-resolution processing, and finally generate the corresponding high-resolution image data.

[0057] 1. Build a super-resolution generative adversarial network model

[0058] The super-resolution generative adversarial network model consists of a generator network, a discriminator network, and a loss function.

[0059] (1) Generator Network

[0060] The generator network is composed of a first input unit, a residual block unit (Residual Blocks), a sub-pixel convolution unit and a first output unit. There are multiple residual blocks (Residual Blocks).

[0061] Furthermore, the generator network adopts a deep convolutional neural network with 16 residual block units. The specific network structure parameters of the generator network include convolution kernel size, step size and number of feature maps.

[0062] Among them, the first input unit includes a two-dimensional convolution layer (convolution kernel 9×9, the number of feature maps is 64, and the step size is 1) and the first activation function.

[0063] Each residual block contains: a two-dimensional convolution layer (convolution kernel 3×3, the number of feature maps is 64, and the step size is 1), a normalization layer (Batch Normalization, BN), the first activation function, and a skip connection. The sub-pixel convolution unit consists of a two-dimensional convolution layer, a pixel map, and a first activation function. The sub-pixel convolution unit is used for upsampling. The convolution kernel size of the two-dimensional convolution layer is 3×3, and the number of output feature maps is related to the upsampling factor.

[0064] Generally, parameterized P ReLU (Parametric ReLU) is used as the first activation function of the residual block unit and the sub-pixel convolution unit to improve the nonlinear mapping ability of the super-resolution generative adversarial network model.

[0065] (2) Discriminator network

[0066] The discriminator network consists of a second input unit, a convolutional layer, a fully connected unit, and a second output unit. The discriminator network is used to extract and analyze the super-resolution feature data of different scales through a convolutional neural network, score the generated super-resolution reconstructed data, and determine whether the output image is a high-resolution image that meets the requirements.

[0067] ① The second input unit: includes input of real high-resolution images (HR, obtained by drones or aerial platforms) and high-resolution images (SR) generated by the generator network.

[0068] ②Convolutional layer: The discriminator network extracts image features through a series of convolutional layers. Each convolutional layer is usually followed by an activation function and a Batch Normalization (BN) layer.

[0069] The discriminator network consists of 8 convolutional layers with a kernel size of 3×3. The number of feature maps in each convolutional layer increases with the depth of the network; initially, there are 64 feature maps, which increase exponentially to 512 feature maps. To reduce the spatial resolution of the input image, each time the number of feature maps doubles, the image size is reduced by strided convolution.

[0070] Second activation function: Leaky ReLU activation function is used after each of the above convolutional layers so that negative inputs can also have non-zero gradients, enhancing feature extraction capabilities.

[0071] ③Fully connected unit: After all convolutions and downsampling are completed, the feature map will be flattened into a one-dimensional vector and passed to the fully connected unit. The fully connected unit includes 2 fully connected layers.

[0072] For example, the fully connected layer Dense (1024): contains 1024 neurons, followed by a Leaky ReLU activation function. Dense (1024) maps the high-dimensional features upsampled by Pixel Shuffler back to the output high-resolution image.

[0073] Fully connected layer Dense (1): Output layer, containing 1 neuron, used to determine whether the input image is a real image or a generated image. The last layer is usually a convolution operation without an activation function (such as ReLU), which directly predicts the final pixel value.

[0074] ④ Second output unit: The last layer is processed using the third activation function (Sigmoid activation function) and makes judgments based on the output results.

[0075] As the number of layers increases, the number of feature maps gradually increases. In order to avoid the problem of dead loop during model training, the Leaky ReLU activation function is used, and the hyperparameter of the activation function is 0.2.

[0076] (3) Loss Function

[0077] In order to evaluate the super-resolution reconstruction performance in the network, the loss function consists of three parts: adversarial loss, perceptual loss, and content loss. Adversarial loss describes the error between the high-resolution image data generated by the super-resolution generative adversarial network model and the real high-resolution data. Perceptual loss is calculated based on the Euclidean distance between the generated high-resolution image and the real image. Content loss represents the mean square error (MSE) between the generated image and the real image in pixel space.

[0078] The loss function is used to calculate the loss of the generator network and the discriminator network after each forward propagation. The forward propagation refers to the generator network receiving low-resolution image input and generating high-resolution images, and the discriminator network receiving the high-resolution image generated by the generator network and the real high-resolution image, and outputting the true and false probabilities.

[0079] The problem of solving the loss function can be transformed into the maximum-minimum optimization problem of the adversarial model of the generator network and the discriminator network, as shown in the following formula.

[0080] , Formula Ⅰ;

[0081] In the formula, are the parameters of the generator network, are the parameters of the discriminator network, Represents the distribution from real data A high-resolution image sample extracted from represents a super-resolution image sample generated from a low-resolution image, is the probability that the real high-resolution image output by the discriminator network is the real image, is the probability that the super-resolution generated image output by the discriminator network is a real image, It means that the goal of the generative network is to minimize this term so that the generated image meets the discrimination condition of the discriminator network.

[0082] The core goal of the generative adversarial network is to enable the generator network and the discriminator network to jointly optimize the objective function through adversarial training:

[0083] For the generator, the purpose of the loss function is to: (1) minimize the probability that the discriminator misclassifies the generated sample as false, that is, to minimize the probability of image super-resolution error; and (2) alternately update the objective functions of the discriminator and the generator during the iteration process until the termination condition of the iteration is met.

[0084] For the discriminator, the loss function has two purposes: (1) maximizing the probability that the discriminator correctly classifies the real sample, that is, maximizing the probability that the discriminant image is the high-resolution image after upsampling; (2) maximizing the probability that the discriminator correctly identifies the generated sample as false, that is, maximizing the probability that the discriminant image is the original low-resolution image that has not been upsampled.

[0085] 2. The training process of the super-resolution generative adversarial network model, using the trained super-resolution generative adversarial network model to obtain high-resolution image data.

[0086] First, the generator network is introduced to enable the super-resolution generative adversarial network model to achieve a certain generation capability without adversarial loss; then, the parameters of the generator network are fixed, and the parameters of the discriminator network are updated to reduce the loss function; and then the Adam optimizer (loss function) is used for training to make the super-resolution generative adversarial network model have better convergence performance and output the final super-resolution reconstructed image. In order to update the weight parameters of the corresponding convolution kernel, each layer of convolution in the generator will participate in the parameter update of the Adam optimizer.

[0087] The Adam optimizer is mainly used to update the parameters of the generator network and the discriminator network. Specifically, in each iteration, the network parameters are optimized and adjusted according to the gradient calculated by the loss function. Accordingly, the Adam optimizer is also used to minimize the loss function of the generator and the discriminator. In Adam optimization, the learning rate is 10 -4 , the first and second order momentum parameters They are 0.9 and 0.999 respectively.

[0088] The specific training process and practical application process of the super-resolution generative adversarial network model are as follows:

[0089] (1) If Figure 2 As shown, the generator network output is used to obtain high-resolution image data

[0090] ① Initial feature extraction: The preprocessed Sentinel-2 data in step 1, i.e., the low-resolution image (LR), is input into the generator network of the super-resolution generative adversarial network model, and two-dimensional convolution (convolution kernel 9×9, number of feature maps is 64, step size is 1) is performed to extract the basic features of the low-resolution image; then, the P ReLU (Parametric ReLU) activation function is used to increase the nonlinear expression capability.

[0091] ② Residual unit processing: The image data after initial feature extraction is input into the residual unit for two-dimensional convolution processing. Each layer of two-dimensional convolution processing includes two-dimensional convolution (convolution kernel 3×3, number of feature maps is 64, step size is 1), normalization and P ReLU activation function processing.

[0092] ③ The image data after initial feature extraction skips the processing of the residual block unit and directly performs the first element-wise sum processing with the image data processed by the residual block unit to form residual features. This can alleviate the gradient vanishing problem in deep networks and improve training efficiency.

[0093] ④ The residual features after the first element-level addition are again subjected to two-dimensional convolution (convolution kernel 3×3, number of feature maps is 64, step size is 1) and normalization.

[0094] ⑤ The image data after initial feature extraction skips the residual block unit process and the two-dimensional convolution and normalization process of step ④, and performs a second element-wise sum process with the image data processed in step ④. The purpose of this process is to improve the efficiency of gradient propagation and avoid gradient disappearance.

[0095] ⑥ Sub-pixel convolution processing: The image data with stable features after the second element-level addition processing is input into the sub-pixel convolution unit, and the Pixel Shuffle method is used for upsampling.

[0096] Specifically, the following steps are performed in sequence: 2D convolution processing (convolution kernel 3×3, number of feature maps 64, step length 1), generating 256 feature maps; pixel mapping processing, i.e. rearranging pixels of feature maps to improve resolution, and finally generating 1024 feature maps, which is equivalent to expanding 256-dimensional features to 1024, that is, upsampling by 2 times; and then P ReLU activation function processing. Finally, the low-resolution image is gradually restored to a high-resolution image.

[0097] ⑦ The data processed by the sub-phase element convolution unit is processed by the first output unit for the last two-dimensional convolution (convolution kernel 9×9, number of feature maps is 64, step size is 1) to obtain the final reconstructed high-resolution image data (SR).

[0098] (2) Use the discriminator network to score the high-resolution results output by the generator network to determine the authenticity of the input image.

[0099] ① Input the high-resolution image (SR) generated by the generator network and the real high-resolution image (HR) into the second input unit, perform two-dimensional convolution (convolution kernel 3×3, number of feature maps is 64, step size is 1) processing to extract the basic features of the low-resolution image; then use the Leaky ReLU activation function for processing.

[0100] ② Input the image data processed by the second input unit into the convolution layer for processing. Each processing includes two-dimensional convolution (convolution kernel 3×3), normalization and second activation function processing. After all convolutions and upsampling are completed, the feature map will be flattened into one-dimensional vector data.

[0101] ④ Pass the one-dimensional vector data obtained after convolutional layer processing to the fully connected layer.

[0102] ⑤ Input the data processed by the convolution layer into the second output unit, use the Sigmoid activation function, and the output value range is [0,1].

[0103] The closer the output value is to 1, the output is the original low-resolution image. The closer the output value is to 0, the model believes that the output image is a generated high-resolution image (SR).

[0104] (3) Loss function

[0105] During the training or super-resolution processing of the super-resolution generative adversarial network model, a two-dimensional convolution layer of a sub-element scale is constructed based on the super-resolution parameter size set at the time of input; the input low-resolution image is processed by two-dimensional convolution to extract basic features, and feature enhancement is achieved through the combination of multiple residual block units, so as to construct a feature extraction unit for obtaining a high-resolution image based on the low-resolution image, and finally pass it through the discriminator network.

[0106] Step 3: Generate super-resolution image of lake mask area

[0107] Due to the freeze-thaw effect caused by temperature changes in the Lasman Hills region, the lakes in this area show periodic changes of freezing and thawing. During the freezing period, the contrast between the lake water and the surrounding objects is reduced due to the cover of ice and surface snow, and the error in extracting the lake water area is large. For optical images, the reflectivity of the water body shows an obvious absorption effect from the visible light to the near-infrared band. Therefore, in order to reduce the amount of calculation and improve the accuracy of lake area extraction, it is necessary to extract the potential location of the lake and generate a lake buffer (i.e., lake mask area).

[0108] like Figure 1 As shown, the specific steps include:

[0109] 1. Image Binarization

[0110] (1) Sentinel-2 optics contains data from 13 bands, from visible light to near-infrared bands. Select the band 3 (green band) and band 8 (near-infrared band, NIR) data (i.e., high-resolution image data) of the Sentinel-2 image generated in step 2 during the summer period, and calculate its normalized water index (NDWI) according to the following formula Ⅱ; at this time, the OSTU global threshold is used to obtain the potential lake area.

[0111] Formula II;

[0112] (2) NDWI < 0.05 is marked as water body; otherwise, it is marked as non-water body, and finally a binary image is obtained.

[0113] 2. Lake area extraction

[0114] For the binary image obtained, the connected domain processing method is used to calculate the area of ​​each water body connected domain; the area < 0.001km 2 The water body connected domains that exceed the noise area threshold are marked as noise and filtered out; the remaining water body connected domains are marked as potential lake areas.

[0115] 3. Lake buffer (lake mask area) generation

[0116] Lakes have seasonal changes, and the NDWI index may also cause errors. Therefore, for each water body connected domain object, a specific threshold (such as 50m) is set to expand the boundary of the lake and generate a super-resolution image of the lake buffer (lake mask area).

[0117] Step 4: Lake water extraction based on conditional random field model (CRF model)

[0118] like Figure 4 As shown in the figure, the main process of this step includes: (1) segmenting the super-resolution image of the lake buffer into different super-pixel objects; (2) constructing a conditional random field model, in which the unit potential function is composed of the band reflectance of the optical data and its corresponding GLCM (Gray-Level Co-occurrence Matrix) features, and the potential is composed of the cross-correlation functions of different super-pixel objects; (3) based on the conditional random field model and the minimum energy function loss criterion, the Graph-Cut method is used to achieve super-pixel level segmentation to obtain the final water body extraction result.

[0119] The specific steps are as follows:

[0120] 1. Use the Meanshift superpixel segmentation method to segment the super-resolution image of the lake buffer into different superpixel objects;

[0121] 2. Use the following formulas III to V to construct a conditional random field model (CRF model).

[0122] Formula III;

[0123] Formula IV;

[0124] Formula V;

[0125] in, is the normalization coefficient; is the weight parameter of the CRF model; is a potential energy function, which represents the connection potential between different superpixel objects and is constructed using a cross-correlation function (CRF potential energy function); Represents each superpixel The unit potential energy function of represents the characteristics of superpixels, Represents the characteristics of superpixel i (this embodiment includes spectral reflectance and various calculated grayscale histogram features); Indicates the type of water body (sea ice or sea water), is the input feature of the optical data; i and j represent superpixel units i and j respectively; S is the set of all superpixel units; N i is the set that does not contain superpixel i.

[0126] In addition to the spectral reflectance of each band, the features used also include: mean, standard deviation, homogeneity, contrast, correlation coefficient, and entropy, which are calculated using the following formulas VI to X respectively; the window size is set to 4, the separable distance is 4, the step size of the sliding window is 4, and the grayscale level is 64.

[0127] Formula VI

[0128] Formula VII;

[0129] Formula VIII;

[0130] Formula IX;

[0131] Formula Ⅹ.

[0132] Among them, i and j are gray levels, k and K are the maximum gray levels (set to 64); Sd is the superpixel unit of the 8-neighborhood; μ x and μ y are the means along the row and column directions respectively; σ x and σ x are the standard deviations along the row and column directions, respectively.

[0133] 3. Water extraction based on minimum energy function.

[0134] Based on the conditional random field model constructed in the above steps and the minimum energy function loss criterion, the Graph-Cut method is used to realize superpixel segmentation, and the image is segmented into water bodies and non-water bodies to obtain the final water body extraction result.

[0135] like Figure 5 As shown in FIG. 1 , the experimental water body objects of the present invention are selected from the water body lakes (Jinbu Lake and Mochou Lake) near the Larsmann Hills in Antarctica. The water body extraction results of different algorithms. Figure 4 It can be seen that by adopting the above method provided by the present invention, the best results can be obtained not only in terms of the integrity of water area extraction but also in terms of the preservation of edge details.

[0136] The above specific embodiments describe the implementation of the present invention in detail, but the present invention is not limited to the specific details in the above embodiments. Within the scope of the claims and technical concept of the present invention, the technical solution of the present invention can be modified and changed in many simple ways, and these simple modifications all belong to the protection scope of the present invention.

Claims

1. A water body extraction method based on conditional random fields and sub-pixel positioning, characterized in that: The steps include: pre-processing the water body image data to be extracted to obtain low-resolution original data; Constructing a super-resolution generative adversarial network model, inputting the low-resolution original data into the generator network of the super-resolution generative adversarial network model for processing, and scoring and discriminating through the discriminator network to obtain corresponding high-resolution image data after super-resolution reconstruction; Performing image binarization processing on the high-resolution image data, extracting the water body potential area, and removing noise to obtain the water body potential area; expanding the boundary of the water body potential area according to a preset threshold value to generate a water body buffer zone super-resolution image; The water body buffer super-resolution image is segmented into a number of super-pixel objects, a conditional random field model is constructed, and based on the conditional random field model and the minimum energy function loss criterion, the super-pixel objects are segmented into water bodies and non-water bodies to obtain the final water body extraction result.

2. The water body extraction method based on conditional random fields and sub-pixel positioning according to claim 1 is characterized in that: The method for preprocessing the water body image data to be extracted includes: performing atmospheric correction on the water body image data to be extracted, converting the zenith reflectivity of the L1C data into the L2A ground surface reflectivity, and obtaining the low-resolution original data.

3. The water body extraction method based on conditional random fields and sub-pixel positioning according to claim 1 is characterized in that: The super-resolution generative adversarial network model includes a generator network, a discriminator network and a loss function; the generator network includes a residual block unit and a sub-phase element convolution unit; The training method of the super-resolution generative adversarial network model includes: Input the low-resolution original data into the generator network, perform two-dimensional convolution processing with a convolution kernel of 9×9 and a first activation function processing, and complete initial feature extraction; The image data after initial feature extraction is input into the residual unit for several times of 3×3 two-dimensional convolution processing, normalization processing and first activation function processing; The image data after the initial feature extraction skips the processing of the residual block unit and directly performs a first element-level addition with the image data after the processing of the residual block unit; The image data obtained by the first element-level addition is subjected to a two-dimensional convolution with a convolution kernel of 3×3 and normalized to obtain convolved image data; The image data after initial feature extraction skips the residual block unit process and performs a second element-level addition with the convolution image data; The image data after the second element-level addition processing is input into the sub-pixel convolution unit, and the convolution kernel 3×3 two-dimensional convolution, pixel mapping, activation function processing, and convolution kernel 9×9 two-dimensional convolution are performed in sequence to obtain a high-resolution image; The discriminator network is used to score and judge the generated high-resolution image to confirm the authenticity of the image.

4. The water body extraction method based on conditional random fields and sub-pixel positioning according to claim 1 is characterized in that: The method of performing image binarization processing and connected domain processing on the high-resolution image data to remove noise and obtain the potential area of ​​the water body includes: Select a scene using the high-resolution image data generated by the super-resolution generative adversarial network model, calculate the normalized water index, mark the area with a normalized water index <0.05 as a water body, and mark the area with a normalized water index ≥0.05 as a non-water body, to obtain a binary image; The connected domain processing method is used for the binary image to calculate the area of ​​each water body connected domain; the water body connected domains with an area smaller than the noise area threshold are marked as noise and filtered out; and the remaining water body connected domains are marked as potential lake areas.

5. The water body extraction method based on conditional random fields and sub-pixel positioning according to claim 4 is characterized in that: For each of the water body connected domains in the potential area of ​​the lake, the lake boundary is expanded according to a preset threshold value to generate a super-resolution image of the lake buffer zone.

6. The water body extraction method based on conditional random fields and sub-pixel positioning according to claim 1 is characterized in that: The method of obtaining the final water body extraction result by using the lake buffer super-resolution image includes: Segmenting the super-resolution image of the lake buffer into different super-pixel objects; Constructing a conditional random field model, wherein a unit potential energy function in the conditional random field model is composed of the band reflectance of the optical data and its corresponding GLCM features, and the potential energy is composed of the cross-correlation functions of different superpixel objects; Based on the conditional random field model and the minimum energy function loss criterion, the superpixel object is segmented into active water body and non-water body areas to obtain the final water body extraction result.

7. A water extraction system based on conditional random fields and sub-pixel positioning, characterized in that: It includes original image acquisition and preprocessing module, super-resolution reconstruction module, and lake water extraction module; The original image acquisition and preprocessing module is used to receive the image data of the water body to be extracted, and preprocess the image data of the water body to be extracted to obtain low-resolution original data; The super-resolution reconstruction module is used to construct and train a super-resolution generative adversarial network model, input the low-resolution original data and output the corresponding high-resolution image data; The lake water body extraction module is used to perform image binarization and connected domain processing on the high-resolution image data to remove noise and obtain potential water areas; expand the boundaries of the potential water areas to generate super-resolution images of water buffer zones; segment the super-resolution images of water buffer zones to obtain a number of super-pixel objects, and based on a conditional random field model and a minimum energy function loss criterion, perform super-pixel segmentation on the super-pixel objects into water bodies and non-water bodies to obtain a final water body extraction result.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is used to enable a computer to execute the water body extraction method based on conditional random fields and sub-pixel positioning as described in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it is used to implement the water body extraction method based on conditional random fields and sub-pixel positioning as described in any one of claims 1-6.

Images