Water body extraction method based on high-resolution remote sensing image

Abstract

The invention discloses a water body extraction method based on a high-resolution remote sensing image, and the method comprises the following steps: 1, carrying out the optimization and improvement of U-Net < 3 + >, and proposing to use improved GhostNet V2 as an encoder network of U-Net < 3 + >, so as to solve the problems that an original network is poor in water body feature extraction capability in a remote sensing image water body extraction task, and is large in important feature loss; 2, introducing an improved cavity space convolution pooling pyramid module (DC-ASPP) between an encoder and a decoder in the U-Net < 3 + > model to solve the problem that the size of a water body of an original network in a remote sensing image is varied, so that misclassification or missing extraction of the network is easily caused in the extraction process; 3, introducing a channel and space mixed attention mechanism module (EMAM), and placing the EMAM between a decoder and an encoder of each layer of U-Net3 + to solve the problem that the boundary between a water body and a complex background is fuzzy, so that the segmentation of the water body boundary of the original network is inaccurate, and the water body extraction effect of the model on the remote sensing image is further improved.

Description

Technical Field

[0001] This invention relates to semantic segmentation of high-resolution remote sensing images, specifically a water body extraction method based on high-resolution remote sensing images, which accurately segments water bodies in high-resolution remote sensing images and belongs to the field of deep learning. Background Technology

[0002] In recent years, convolutional neural networks (CNNs) have been widely used in the field of water body extraction from high-resolution remote sensing images. They eliminate the need for manual image feature calculation, directly learning texture, edge, and other features from remote sensing images, effectively solving the problem of feature information extraction from high-dimensional, massive datasets. Semantic segmentation technology based on CNNs is currently one of the important methods for image content recognition. It eliminates the need for manual feature summarization, automatically identifying and extracting object information contained in the image, and then performing accurate predictive segmentation, achieving pixel-level segmentation. Traditional high-resolution remote sensing image water body extraction methods face challenges such as the continuous increase in remote sensing image data and the varying sizes of water body regions, complex backgrounds, and inaccurate water boundary segmentation in high-resolution remote sensing images. These challenges result in low accuracy for traditional methods in water body extraction tasks. Therefore, semantic segmentation technology based on deep learning is one of the cutting-edge research directions in remote sensing image water body extraction.

[0003] Deep learning-based semantic segmentation algorithms have achieved good results in semantic segmentation tasks. However, current research on water body extraction from high-resolution remote sensing images often faces challenges such as complex backgrounds, varying water body sizes, and inaccurate water body boundary segmentation. Furthermore, deep learning-based semantic segmentation algorithms are inaccurate in water body boundary segmentation, resulting in low segmentation accuracy and incomplete extraction. This invention aims to address the problems faced by current high-resolution remote sensing image water body extraction tasks by designing a deep learning-based high-resolution remote sensing image water body extraction network, utilizing the characteristics of water body information in remote sensing images. While maintaining high-precision water body segmentation and extraction in conventional scenes, this invention improves the extraction effect of water bodies in complex scenes and small areas, and enhances the accuracy of water body boundary segmentation. Summary of the Invention

[0004] This invention provides a water body extraction method based on high-resolution remote sensing images, which solves the problems disclosed in the background art.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: High-resolution remote sensing images were acquired, preprocessed, and a water body extraction dataset was constructed. The dataset was divided into a training set and a test set. Input the training set into a pre-built convolutional neural network model; The network model is trained by setting training parameters based on the training set, continuously updating the training parameters to obtain the optimal network model, and saving the optimal parameters. The test set is input into the optimal network model, and the optimal parameters are used for testing to obtain the water body segmentation results in the high-resolution remote sensing image, and the water body in the high-resolution remote sensing image is extracted.

[0006] Furthermore, the method for preprocessing high-resolution remote sensing images is as follows: The labels in the high-resolution remote sensing images are divided into two categories: one representing water bodies and the other representing the background. The high-resolution remote sensing images were cropped to 512×512 pixels, and images that did not contain water bodies were removed.

[0007] Furthermore, the training set accounts for 80% and the test set accounts for 20%.

[0008] Furthermore, the network model includes a feature extraction module, a densely connected dilated spatial convolutional pooling pyramid module, a hybrid attention mechanism module, and a loss function module; To address the shortcomings of the U-Net3+ neural network in high-resolution remote sensing image water feature extraction, namely its weak water feature extraction capability and significant loss of important features, this invention employs an improved GhostNetV2 as the feature extraction network to enhance U-Net3+. GhostNetV2 introduces a long-range attention mechanism, which can enhance the model's representational ability. Simultaneously, as a lightweight network, GhostNetV2 has advantages such as fewer parameters, faster speed, and moderate depth, and is used as the backbone feature extraction network in various models to reduce the total number of parameters and improve training speed. Using the improved GhostNetV2 as the encoder network can enhance the model's water feature extraction capability while maintaining its lightweight nature, thus achieving a balance between training efficiency and model accuracy.

[0009] The varying scales of water bodies in high-resolution remote sensing images can easily lead to misclassification or omissions in feature extraction during the network process. To expand the receptive field of the network model and extract water features at different scales, this invention proposes a multi-scale water extraction module (Dense Connect ASPP, DC-ASPP). This module extracts water features at multiple scales through dilated convolutions with different dilation rates and fuses feature maps using dense connections to mitigate the grid effect. Furthermore, to reduce the number of parameters, the DC-ASPP module employs depth-separable dilated convolutions and adds 1x1 convolutions after the dilated convolutions. Using 1x1 convolutions in the network layers effectively reduces the number of channels in the feature maps, significantly reducing computational complexity and improving training and inference efficiency. Since the ASPP module does not extract features from channel information, global average pooling is added to combine global and local information, considering that global average pooling can prioritize channels with more important information. Finally, since the ASPP module does not directly consider the importance of spatial location in the feature map, and considering that max pooling and average pooling on the channel dimension can preserve key information in space, max pooling and average pooling were added to enhance the model's ability to identify the importance of each part in the feature map and its sensitivity to location information.

[0010] To address the issue of inaccurate water body boundary segmentation due to blurred boundaries and low contrast between water bodies and complex backgrounds, traditional U-Net3+ uses a splicing operation in its skip connections, indiscriminately fusing all feature space and channel information. To achieve more accurate boundary localization in water body segmentation, this invention proposes an Enhanced Mixed Attention Mechanism (EMAM). EMAM is placed between the decoder and encoder in each layer of U-Net3+. It highlights important features in both the spatial and channel domains while weakening less important features, effectively strengthening the representation of water body features and thus improving the model's water body segmentation capability, ultimately achieving high-precision water body segmentation.

[0011] Since water extraction only requires distinguishing between water areas and non-water areas, this invention uses the pixel-by-pixel binary cross-entropy loss function (BCELoss) to calculate the network's loss in learning water features, as shown in the following formula: In the formula, L and W are the length and width of the remote sensing image, respectively; ykt is the network's predicted output value; and Pkt is the probability of the network's predicted value belonging to a certain category. BCELoss solves the problems of saturation gradients and slow updates in the derivative form of Sigmoid functions by utilizing an inter-class competition mechanism, effectively avoiding the phenomenon of gradient vanishing leading to a decrease in learning rate.

[0012] This invention addresses the shortcomings of U-Net3+ in high-resolution remote sensing image water extraction by proposing improvements: First, an improved lightweight GhostNetV2 is used as the encoder, enhancing feature representation capabilities through a long-range attention mechanism, balancing model accuracy and training efficiency. Second, a DC-ASPP multi-scale module is designed, fusing separable convolutional structures with different dilation rates and hole depths with dense connections, combined with global average pooling and dual-channel pooling methods to achieve multi-scale feature extraction and suppress grid effects. Furthermore, an EAM hybrid attention mechanism is proposed, using channel-spatial dual-domain feature weighting between the encoder and decoder to enhance water boundary localization capabilities. Finally, a pixel-by-pixel binary cross-entropy loss function is used to make the network more suitable for water extraction tasks. These improvements address the issues of weak feature extraction capabilities, poor multi-scale adaptation, and blurred boundaries, thereby improving the accuracy and robustness of water segmentation in complex backgrounds. Attached Figure Description

[0013] Figure 1 The figure shown is an overall block diagram of an improved U-Net3+ provided by the present invention; Figure 2 The image shows a long-distance attention mechanism module for GhostNetV2 provided by this invention; Figure 3 The figure shown is a basic unit of an improved GhostNetv2 provided by the present invention; Figure 4 The image shows an improved ASPP module provided by this invention; Figure 5 The image shows a novel hybrid attention mechanism module provided by this invention; Figure 6 The image shows a novel spatial attention mechanism module provided by this invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application.

[0016] To address the issues of weak water feature extraction capability and significant loss of important features in high-resolution remote sensing image water extraction tasks using the U-Net3+ neural network, this invention introduces an improved GhostNetv2 as the encoder for the water extraction network. GhostNetv2 reduces computational cost through ghost convolution.

[0017] The standard convolution operation is represented as follows, assuming the input image data is... Where c is the number of input channels, and h and w are the height and width of the input data, the image is first convolved using partial convolution kernels to obtain a partial feature map, which can be represented as: in, y′ represents the convolution operation, b is the bias term, y′ is the output feature map, and f is the convolution kernel.

[0018] Through convolutional layers, the input data X generates n output feature maps through convolution operations, and the size of each output feature map is... The generated feature map is represented as In this convolution calculation process, if the size of the convolution kernel f is k·k, the number of floating-point operations required can be expressed by the formula... The calculations are performed using n and c, which are typically very large, such as 256 or 512, requiring a large number of floating-point operations, possibly up to hundreds of thousands.

[0019] The Ghost module performs a cheap linear transformation on y′. This invention uses a 3×3 channel convolution to perform linear operations on the feature map to obtain the remaining feature maps: in, It is the i-th intrinsic feature map in y′. It is a cheap linear operation used to generate the j-th feature map. This represents the feature map ultimately generated by the Ghost module.

[0020] When using Ghost convolution, if the kernel size is d·d, then the computational complexity is: Where s represents the number of feature maps generated by the inexpensive linear transformation operation.

[0021] The computational compression ratio of the Ghost module is: GhostNetv2 captures long-range dependencies between pixels at different spatial locations through a long-range attention mechanism. Figure 2This is a graph illustrating the long-range attention mechanism. The process of generating the attention graph can be represented as follows: given the input data... First, a 1×1 convolution operation is used to transform the input data X into a new feature representation. The specific formula for generating attention maps on fully connected layers is as follows: Where: ⊙ represents element-wise multiplication; F represents the learnable weight parameters in the fully connected layer, used to calculate the relationship between the feature vectors at position (h, w) and position (h′, w′); a is the generated attention map.

[0022] Because feature maps typically have low rank, they do not need to be densely connected to all input and output feature vectors at different spatial locations. However, the above formula densely multiplies all feature vectors element-wise with learnable weights and sums them to capture global information, with computational cost increasing quadratically with the feature size. Therefore, the DFC module decomposes the above formula into two fully connected layers and aggregates features along the horizontal and vertical directions, respectively.

[0023] Here, FH and FW are weight parameters in the DFC module, applied sequentially to the feature vector to capture long-distance dependencies in two directions, respectively. Finally, the output is the attention map generated by the DFC attention module.

[0024] This invention replaces the activation function in the basic unit of GhostNetv2 with the Mish function and uses the improved GhostNetv2 as the encoder in the water extraction task model, such as... Figure 3 As shown. In the task of water body extraction from remote sensing images, although the ReLU function is computationally efficient as an activation function, its gradient vanishing and neuron death problems in the negative interval affect the model's adaptability to complex backgrounds (such as shadows, buildings, and noise). The Mish function, on the other hand, is continuously differentiable and preserves information in the negative interval, making it suitable for fine segmentation of complex water body edges. In deep networks, Mish improves the model's generalization ability and reduces missed detections compared to ReLU. The formula for the Mish function is as follows: To address the challenge of distinguishing between water bodies of varying sizes in water extraction tasks, this invention proposes an improved ASPP module, such as... Figure 4As shown, the dilation rate of the dilation-depth separable convolution references the parameter settings of the original ASPP structure, with dilation rates set to 6, 12, and 18 respectively. Features from a 1x1 convolutional layer are fused in the branch of the convolutional layer with a dilation rate of 6, in the branch with a dilation rate of 12, and in the branch with a dilation rate of 18. Two parallel branches focusing on channel and spatial feature information are also included. The branch focusing on channel information integrates global feature information through global average pooling, and then... One convolutional kernel is used to increase non-linearity, and then the sigmoid function is used to restrict the pixel values ​​of the feature map to between 0 and 1. Finally, channel-weighted summation is performed with the original input feature map to increase attention to important channel information. The branch focusing on spatial information first obtains two H values ​​by performing max pooling and average pooling operations on the channel dimension. W The feature map is 1. Then, the results of global max pooling and global average pooling are concatenated by channel to obtain a feature map of size H. W 2, then through 1 A convolutional kernel is used to perform a nonlinear transformation, and then the sigmoid function is applied to restrict the pixel values ​​of the feature map to between 0 and 1, resulting in a spatial attention weight matrix. Finally, this matrix is ​​weighted with the original input feature map to increase attention to important spatial information features. In the reconstructed dilated convolutional structure, on the one hand, pixel sampling of multi-scale water feature maps can be performed more densely, achieving the purpose of multi-spatial scale feature fusion; on the other hand, it can provide a larger receptive field, which helps to reduce the loss of spatial hierarchical information and water information in remote sensing images.

[0025] To address the issues of complex boundaries and low contrast between water and background, this invention proposes a hybrid attention mechanism module, the block diagram of which is shown below. Figure 5 As shown, the hybrid attention mechanism feature fusion module consists of a spatial attention module (SAM) and an ECA channel attention module connected in series. The spatial attention mechanism module (SAM) emphasizes the global spatial features of the main branches, using detail information to extract details from the branches. The ECA module is placed between the encoder and decoder in the same layer of U-Net3+. The specific structure of the SAM module is shown below. Figure 6As shown. First, SAM receives feature information from the encoder, which is then processed by two branches. In the main branch, dynamic parameter pooling (DE-Pooling) is used to balance the differences in spatial features and assign spatial attention weights to each pixel. Then, spatial information is interacted through a combination of Conv1×1+BN+ReLU convolutions to obtain spatial attention map A. In the detail information extraction branch, the input features are fed into a combination of two Conv3×3+BN+ReLU convolutions to extract a feature map B containing detail information. Therefore, the generation of the spatial attention map can be expressed by the following formula: Where F1x1 represents a kernel size of 1. 1. Pixel-level multiplication is used to create an interaction between the spatial attention map and the dimensionality-reduced features, resulting in a spatial weight map of the backbone features. Furthermore, the backbone feature space weight map T and the shallow feature map B are stacked through weighted fusion to obtain the spatial feature mask M = T + λB. Then, the spatial feature mask M passes through the ECA channel attention mechanism module. ECA is an improvement on SENet. As we know from the channel attention module in SENet, avoiding dimensionality reduction is crucial for learning channel attention; appropriate cross-channel interactions can significantly reduce model complexity while maintaining performance.

[0026] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A water body extraction method based on high-resolution remote sensing images, characterized in that, The application relates to a high-resolution remote sensing image water body extraction method. Step 1: publicly disclosed remote sensing data set GID is manually annotated, and the annotated data set is divided into a training set and a test set according to a set proportion; Step 2: a high-resolution remote sensing image semantic segmentation network is built; Step 3: the high-resolution remote sensing image semantic segmentation network built in step 2 is pre-trained and parameter-optimized; Step 4: the preprocessed test set data set is input into the high-resolution remote sensing image water body extraction method network which has been trained in step 3, and the water body extraction result of the high-resolution remote sensing image is output. The high-resolution remote sensing image semantic segmentation network comprises a feature extraction module, a densely connected hollow spatial convolution pooling pyramid module, a hybrid attention mechanism module and a loss function module.

2. The water body extraction method based on high-resolution remote sensing images according to claim 1, wherein, In step 2, the feature extraction module takes an improved GhostNetV2 as an encoder network of U-Net3+, and the GhostNetV2 introduces a long-distance attention mechanism.

3. The water body extraction method based on high-resolution remote sensing images according to claim 2, characterized in that, 4. The high-resolution remote sensing image water body extraction method according to claim 2, wherein the densely connected hollow spatial convolution pooling pyramid module in step 2 extracts water body features in a multi-scale mode through hollow convolution with different expansion rates, and fuses feature maps in a densely connected mode.

5. The high-resolution remote sensing image water body extraction method according to claim 2, wherein the hybrid attention mechanism module in step 2 is composed of a new spatial attention module SAM and a channel attention module ECA connected in series; the spatial attention mechanism module emphasizes the global spatial features of the main branch and fills in details with the detail information extraction branch. The channel attention module dynamically adjusts the weight of different feature channels and captures features related to water bodies.

6. The method according to claim 5, wherein the water body is extracted by using a high resolution remote sensing image. In step 2, the loss function is a per-pixel binary cross-entropy loss function for calculating the water area feature loss of network learning, and the formula is as follows:

7. The water body extraction method based on high-resolution remote sensing images according to claim 5, characterized in that: ​ 。

Images