Lake water body extraction method, system, equipment, medium and product

Abstract

The invention discloses a lake water body extraction method, system and device, a medium and a product, and relates to the field of target recognition, and the method comprises the steps: extracting a water body in a remote sensing image of a to-be-detected region based on a lake water body extraction model, obtaining a lake water body image, and determining a water body region of the to-be-detected region; the category of each pixel point is marked in the lake water body image, and the category of the pixel point is a water body or a non-water body; the lake water body extraction model comprises a lightweight feature extraction network, a global average pooling layer, an edge extraction module, a first convolution attention module, a second convolution attention module and a feature fusion module. According to the method, the complexity of the lake water body extraction model is reduced on the premise of ensuring the accuracy of the lake water body extraction model, the reasoning time of the lake water body extraction model can be reduced, the category of each pixel point in the remote sensing image is determined by adopting the lake water body extraction model, and the accuracy and efficiency of lake water body extraction are improved.

Description

Technical Field

[0001] The present application relates to the field of target recognition, and in particular to a lake water extraction method, system, equipment, medium and product based on a lightweight feature extraction network and edge features. Background Art

[0002] In the process of implementing ecological protection measures, it is necessary to monitor the lake water area and lake distribution for a long time to evaluate the effectiveness of ecological protection measures; in the process of water resources management, by extracting the lake water area and lake distribution, reliable data can be provided for water resources assessment and management, thereby optimizing water resources allocation; in the flood season, by real-time monitoring of lake water changes, flood disasters can be predicted and prevented, casualties and property losses can be reduced, and after natural disasters occur, rapid acquisition of lake water information can help formulate emergency response plans; in urban construction, by monitoring lake water information, it can help plan urban green spaces, water areas and other natural landscapes, and further combine the Internet of Things technology to realize intelligent monitoring and management of lake water bodies and improve the level of urban management. Therefore, it is necessary to extract lake water information accurately and quickly. Summary of the invention

[0003] The purpose of this application is to provide a lake water extraction method, system, equipment, medium and product, which can improve the accuracy and efficiency of lake water extraction.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a lake water extraction method, comprising:

[0006] Acquire remote sensing images of the area to be inspected;

[0007] Extracting water bodies in the remote sensing image based on a lake water body extraction model to obtain a lake water body image to determine the water body area of ​​the area to be detected; the lake water body extraction model is obtained by pre-training with a training sample set; the training sample set includes a plurality of sample remote sensing images and a lake water body image corresponding to each sample remote sensing image; the lake water body image is marked with a category of each pixel point, and the category of the pixel point is water body or non-water body;

[0008] Among them, the lake water extraction model includes a lightweight feature extraction network, a global average pooling layer, an edge extraction module, a first convolutional attention module, a second convolutional attention module and a feature fusion module; the global average pooling layer is connected to the lightweight feature extraction network; the first convolutional attention module is respectively connected to the lightweight feature extraction network and the edge extraction module; the second convolutional attention module is respectively connected to the lightweight feature extraction network and the global average pooling layer; the feature fusion module is respectively connected to the lightweight feature extraction network, the global average pooling layer, the first convolutional attention module and the second convolutional attention module.

[0009] In a second aspect, the present application provides a lake water extraction system, comprising:

[0010] An image acquisition unit, used to acquire remote sensing images of the area to be detected;

[0011] A water body extraction unit is used to extract the water body in the remote sensing image based on a lake water body extraction model to obtain a lake water body image to determine the water body area of ​​the area to be detected; the lake water body extraction model is obtained by pre-training with a training sample set; the training sample set includes a plurality of sample remote sensing images and a lake water body image corresponding to each sample remote sensing image; the lake water body image is marked with a category of each pixel point, and the category of the pixel point is water body or non-water body;

[0012] Among them, the lake water extraction model includes a lightweight feature extraction network, a global average pooling layer, an edge extraction module, a first convolutional attention module, a second convolutional attention module and a feature fusion module; the global average pooling layer is connected to the lightweight feature extraction network; the first convolutional attention module is respectively connected to the lightweight feature extraction network and the edge extraction module; the second convolutional attention module is respectively connected to the lightweight feature extraction network and the global average pooling layer; the feature fusion module is respectively connected to the lightweight feature extraction network, the global average pooling layer, the first convolutional attention module and the second convolutional attention module.

[0013] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned lake water extraction method.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned lake water extraction method when executed by a processor.

[0015] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the above-mentioned lake water extraction method when executed by a processor.

[0016] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0017] The present application provides a lake water extraction method, system, equipment, medium and product, which constructs a lake water extraction model based on a lightweight feature extraction network, a global average pooling layer, an edge extraction module, a convolutional attention module and a feature fusion module. The complexity of the lake water extraction model is reduced while ensuring the accuracy of the lake water extraction model, and the reasoning time of the lake water extraction model can be reduced. The lake water extraction model is used to determine the category (water body or non-water body) of each pixel point in the remote sensing image, and the water body area of ​​the area to be detected is determined, thereby improving the accuracy and efficiency of lake water extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a schematic diagram of the structure of BiSeNet;

[0020] Figure 2 This is a schematic diagram of the structure of a lightweight semantic segmentation network;

[0021] Figure 3 This is an application environment diagram of a lake water extraction method in one embodiment of the present application;

[0022] Figure 4 A schematic diagram of a process for extracting lake water provided in one embodiment of the present application;

[0023] Figure 5 Schematic diagram of the processing stages of the lightweight feature extraction network;

[0024] Figure 6 A schematic diagram of the structure of a lake water extraction model provided in one embodiment of the present application;

[0025] Figure 7 This is a schematic diagram of the processing flow of the first block in the semantic segmentation module;

[0026] Figure 8 This is a schematic diagram of the processing flow of blocks other than the first block in the semantic segmentation module;

[0027] Fig. 9 Schematic diagram of the structure of the convolutional attention module;

[0028] Fig.10 This is a schematic diagram of the structure of the channel attention mechanism in the convolutional attention module;

[0029] Fig.11 Schematic diagram of the structure of the spatial attention mechanism in the convolutional attention module;

[0030] Fig.12 It is a structural diagram of the feature fusion module;

[0031] Fig.13 A schematic diagram of the processing flow of a lake water extraction model provided in one embodiment of the present application;

[0032] Fig.14 Schematic diagram of the dimensional change process of the image extraction model in lake water. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] The current lake water extraction model introduces a large number of parameters and complex structures in the network structure while achieving high accuracy. The complex network structure increases the computational complexity of the model, making the training and inference process more time-consuming and difficult to debug. In addition, the complex network structure will increase the storage and deployment costs of the model, which may face challenges when applied in a resource-constrained environment. Therefore, designing and using appropriately simplified network structures is crucial to improving the efficiency of the model, reducing the debugging cost, and the feasibility of deployment applications. There are usually three ways to reduce the complexity of the model: one is to limit the computational complexity by cropping or adjusting the input size. Although this method is effective, it usually does not meet actual needs and loses spatial detail information. The second is to reduce the network channels to increase the inference speed, which will weaken the expression and richness of spatial information. The third is to abandon the deep part of the model, which will result in a limited receptive field and fail to cover large objects, thereby reducing the discrimination ability. In general, these methods are all in exchange for time performance by reducing network complexity, often at the expense of model accuracy.

[0035] Therefore, the present application provides a lake water body extraction model (A Lake Water Body Extraction Network Based on STDC and Edge Features, SE-LWBENet) based on a lightweight semantic segmentation network (Short-Term Dense Concatenate network, STDC) and edge features, striving to reduce the model complexity and reduce the model's reasoning time while ensuring the accuracy of the model. SE-LWBENet uses the STDC backbone network to use edge extraction operators to extract edges from the input image and the target image respectively, and injects the edge information extracted from the input image into the deep network for training. At the same time, the edge information extracted from the target image is calculated with the STDC shallow spatial information for loss, and the training process of the network is constrained together with the target extraction loss. The convolutional attention module (Convolutional Block Attention Module, CBAM) adjusts the weights of the channel and spatial information of the deep semantic feature map that incorporates the edge information, so as to more accurately grasp the contextual information of the semantic segmentation. Finally, the deep and shallow information are fused through the feature fusion module (Feature Fusion Module, FFM), and the image resolution is restored through upsampling technology, thereby achieving accurate extraction of lake water bodies.

[0036] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0037] First, the relevant concepts and methods involved in this application are introduced.

[0038] (1) BiSeNet and STDC networks.

[0039] In the semantic segmentation method based on deep learning, it is required not only to obtain rich spatial information, but also to obtain sufficient receptive field, which is a problem that must be coordinated between the speed and accuracy of model reasoning. In practice, the model can be lightweight by reducing the number of feature map channels in the convolution calculation process or directly discarding the feature extraction related steps.

[0040] BiSeNet is a lightweight dual-branch semantic segmentation network. One branch is used to extract shallow spatial detail features, and the other branch extracts deep semantic features to obtain sufficient receptive field and rich contextual information. Then, the feature fusion module is introduced to effectively combine spatial details and contextual features. Figure 1However, BiSeNet adds a coding path specifically for spatial information extraction, which is time-consuming, and the efficiency of semantic segmentation tasks is affected to a certain extent by using existing image classification networks (such as ResNet) as the backbone network for extracting semantic information.

[0041] The STDC network uses a single-channel dense cascade module as the backbone for extracting semantic information to obtain receptive fields and multi-scale information. It deletes the branch in BiSeNet that is specifically used to extract spatial details, and instead obtains spatial detail features from the shallow part of the backbone network and semantic features from the deep part. The loss between the target edge extracted by the Laplacian operator and the shallow spatial information is used as a constraint module for spatial detail extraction and added to the loss calculation of semantic segmentation, which effectively improves the segmentation performance of the model. Figure 2 shown.

[0042] (2) Edge detection operator.

[0043] Semantic segmentation has a strong task relevance to edge detection in traditional computer vision tasks. Therefore, traditional edge detection algorithms are used to enhance the input features of spatial information to obtain better prediction results. Edge detection operators are techniques used to detect the edges of objects in images. Edges are places where the grayscale level in an image changes significantly, usually indicating the boundaries between objects or the shape of an object. The goal of edge detection operators is to extract this edge information from an image. Common edge detection operators are shown in Table 1.

[0044] Table 1 Edge detection operators

[0045]

[0046]

[0047] In Table 1, except for Laplacian as a second-order differential operator using a convolution kernel to extract edges, the other operators are first-order differential operators composed of two convolution kernels, which are used to calculate the horizontal gradient G at each pixel position in the image. x and the vertical gradient G y In edge detection, the horizontal and vertical gradients are usually combined to obtain the gradient size and direction G of the edge:

[0048] The Laplacian operator is mainly used to accurately locate step edge points in an image, but it is sensitive to noise and easily loses edge direction information. The Sobel operator assigns different weights according to pixel distance, but fails to accurately simulate the visual characteristics of the human eye, and sometimes the extracted contour effect is poor. The Scharr operator is more sensitive to grayscale changes, but may lose details. The Roberts operator is suitable for low-noise images, but is sensitive to noise and has a better effect on image segmentation with obvious edges and less noise. The Prewitt operator has a good effect on edge detection in the horizontal and vertical directions, and has a suppressive effect on noise. Therefore, this application uses the Prewitt operator for edge extraction.

[0049] The lake water extraction method provided in the embodiment of the present application can be applied to Figure 3 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The terminal 102 can send the remote sensing image to be processed to the server 104. After receiving the remote sensing image to be processed, the server 104 extracts the water body in the remote sensing image based on the lake water body extraction model. The server 104 can feed back the obtained lake water body image to the terminal 102.

[0050] In an exemplary embodiment, Figure 4 As shown, a lake water extraction method is provided, comprising the following steps 201 to 202. Among them:

[0051] Step 201, obtaining a remote sensing image of the area to be detected.

[0052] Specifically, remote sensing image data is first obtained to ensure that the image quality and coverage meet the requirements of lake water extraction. Then the remote sensing image is preprocessed, including image enhancement, denoising and other operations, to improve image quality and extraction effect.

[0053] Step 202: extract the water body in the remote sensing image based on the lake water body extraction model to obtain a lake water body image to determine the water body area of ​​the area to be detected.

[0054] The lake water extraction model is obtained by pre-training with a training sample set. The training sample set includes a plurality of sample remote sensing images and a lake water image corresponding to each sample remote sensing image. The lake water image is marked with the category of each pixel point, and the category of the pixel point is water body or non-water body.

[0055] Wherein, the lake water extraction model includes a lightweight feature extraction network, a global average pooling (GAP) layer, an edge extraction module, a first convolutional attention module, a second convolutional attention module and a feature fusion module. The global average pooling layer is connected to the lightweight feature extraction network. The first convolutional attention module is respectively connected to the lightweight feature extraction network and the edge extraction module. The second convolutional attention module is respectively connected to the lightweight feature extraction network and the global average pooling layer. The feature fusion module is respectively connected to the lightweight feature extraction network, the global average pooling layer, the first convolutional attention module and the second convolutional attention module.

[0056] This application uses the STDC backbone as the feature extraction network in the lake water extraction model. The STDC backbone is an efficient feature extraction network that enhances the spatial and temporal perception of features through dilated convolution. STDC maintains low computational complexity and high feature extraction capabilities through short-term dense connections. After the feature extraction is completed, a global average pooling operation is also performed. This process not only reduces the dimension and number of parameters of the feature map, but also effectively extracts global feature information, so that the lake water extraction model can better capture the global structural features of the image, thereby improving the performance of classification or recognition tasks.

[0057] The edge features of the input image and the target image are extracted respectively, and the extracted edge features of the input image are injected into the deep network for training. At the same time, the edge features of the target image are used to calculate the loss with the STDC shallow spatial features, and the training process of the model is constrained together with the final image segmentation loss.

[0058] After fusing the edge features and deep semantic feature maps, CBAM is used to adjust the weights of channel and spatial information, so that the lake water extraction model can better focus on the channels and pixel positions related to the target image, thereby enhancing the performance of the model. Then, FFM is used to fuse the deep and shallow features, and bilinear interpolation is used to improve the feature map resolution, enhance the extraction effect, and further improve the accuracy of water extraction.

[0059] Specifically, the preprocessed remote sensing image is input into the lake water extraction model, the image features are extracted by the STDC backbone, and the edge features extracted by the Prewitt operator are used for training. The CBAM is used to adjust the weights of channel and spatial information to enhance the attention of the lake water extraction model to the relevant areas of the target image. The deep and shallow features are fused through FFM, and the resolution of the feature map is improved. Finally, the accurately extracted lake water image is output, and high-precision semantic segmentation of the lake water body is achieved.

[0060] like Figure 5 and Figure 6 As shown in the figure, the lightweight feature extraction network includes five stages. In the first to fifth stages, the feature map is downsampled by convolution operation with a step size of 2, and feature maps with downsampling rates of 1 / 2, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 are generated respectively. A GAP layer is added after the fifth stage, which can provide global semantic information as the maximum receptive field. Two CBAMs are used to adjust the attention of the feature map of the fourth stage that integrates the edge information of the input image and the deep feature map of the fifth stage. The feature map of the third stage retains rich spatial detail information, which is fused with the semantic information from the fourth, fifth and GAP layers through FFM, thereby achieving a comprehensive integration of spatial details and semantic information. The edge information extracted from the target image and the feature map from the third stage are subjected to the loss operation on spatial details, and the spatial detail loss and the final lake water extraction loss are used to jointly constrain the model training process. The combination of target output loss and detail extraction loss to jointly constrain the model training process helps to improve the edge recognition ability, global segmentation performance and feature fusion effect of the SE-LWBENet model.

[0061] The target output head adjusts the number of feature map channels to the number of target category channels for target recognition, and the detail extraction head obtains the same number of channels as the feature map after edge extraction of the target image for loss calculation. Both the target output head and the detail extraction head have 3×3 convolutions.

[0062] In a specific example, step 202 includes the following steps 301 to 308 .

[0063] Step 301, extracting semantic features of the remote sensing image through the lightweight feature extraction network to obtain a spatial feature map, a first semantic feature map and a second semantic feature map.

[0064] Specifically, the lightweight feature extraction network includes a first convolutional layer, a second convolutional layer, a first semantic segmentation module, a second semantic segmentation module and a third semantic segmentation module connected in sequence. The first convolutional layer, the second convolutional layer, the first semantic segmentation module, the second semantic segmentation module and the third semantic segmentation module correspond to their five stages respectively. The first convolutional layer and the second convolutional layer are both 3×3 convolutional layers. Each semantic segmentation module includes a number of blocks.

[0065] The number of blocks included in each semantic segmentation module can be determined according to the specific situation. In this embodiment, the number of blocks included in the first semantic segmentation module, the second semantic segmentation module and the third semantic segmentation module in the STDC backbone network are 3, 4 and 2 respectively. The first Block in each semantic segmentation module needs to perform downsampling and pooling operations with a step size of 2 to increase the network receptive field, and the spatial resolution of the remaining Blocks remains unchanged. Each Block contains four convolution operations. The first convolution uses a 1×1 convolution kernel, and the remaining three use a 3×3 convolution kernel. In order to increase the computing speed, the number of convolution kernels used for the four convolutions is 1 / 2, 1 / 4, 1 / 8, and 1 / 8 of the number of output channels, respectively. Finally, the feature maps obtained by the four convolutions are fused, which can not only obtain the features extracted by each layer and reduce the computational complexity, but also ensure that the segmentation accuracy is not reduced. Figure 7 and Figure 8 As shown, C in is the channel input of the input feature map, C out is the number of channels of the output feature map.

[0066] The spatial feature map is a feature map output by the first semantic segmentation module, the first semantic feature map is a feature map output by the second semantic segmentation module, and the second semantic feature map is a feature map output by the third semantic segmentation module.

[0067] Step 302: performing a pooling operation on the second semantic feature map through the global average pooling layer to obtain a global semantic feature map.

[0068] Step 303: extract edges from the remote sensing image using the edge extraction module to obtain an edge feature map.

[0069] Specifically, step 303 includes the following steps 401 to 405 .

[0070] Step 401, down-sampling the remote sensing image using a 1 / 8 scale Prewitt operator to obtain a first preliminary edge feature map.

[0071] Step 402: down-sample the remote sensing image using a 1 / 16 scale Prewitt operator to obtain a second preliminary edge feature map.

[0072] Specifically, edge information in an image may present different features at different scales. For example, rough edges are more obvious at low resolution (1 / 16 downsampling), while detailed edges may be easier to capture at high resolution (1 / 8 downsampling). This application can capture edge information at different levels more comprehensively by extracting edge features at different scales.

[0073] The use of multi-scale edge features can enhance the robustness of the lake water extraction model, enabling it to accurately extract edges when faced with images of different complexities, and help the lake water extraction model better learn details of different scales in the image during training, thereby improving the overall performance.

[0074] By extracting edge features at different downsampling ratios, the lake water extraction model can better fuse these multi-scale features with deep semantic features. This fusion can enhance the lake water extraction model's ability to identify edge areas while maintaining an understanding of global semantic information.

[0075] Step 403, after sequentially performing convolution (3×3), batch normalization (BatchNormalization, BN), nonlinear activation (Rectified Linear Unit, ReLU), convolution (1×1), and batch normalization on the first preliminary edge feature map, it is integrated with the first preliminary edge feature map, and then sequentially performing nonlinear activation, convolution, and batch normalization to obtain a third preliminary edge feature map.

[0076] Step 404, after performing convolution (3×3), batch normalization, nonlinear activation, convolution (1×1), and batch normalization on the second preliminary edge feature map in sequence, it is integrated with the second preliminary edge feature map, and then nonlinear activation is performed to obtain a fourth preliminary edge feature map.

[0077] Specifically, further convolution operations are performed on the preliminary edge feature map through a 3×3 convolution kernel, which can refine and enhance the edge features, making them more prominent and enabling more clear identification of the details of the edge area. The BN processing after the 3×3 convolution can stabilize the training process, accelerate convergence, and reduce the sensitivity of the lake water extraction model to the initialization weights, which can keep the feature distribution of the lake water extraction model consistent between different training batches, thereby improving the generalization ability of the lake water extraction model.

[0078] ReLU is a commonly used activation function, which is used to introduce nonlinearity, so that the lake water extraction model can represent more complex feature patterns, suppress negative features, and highlight positive features, thereby making edge features more representative.

[0079] 1×1 convolution is usually used to reduce the dimension and compress the number of channels of the feature map while retaining important feature information. The 1×1 convolution operation can reduce the amount of calculation while maintaining the expressiveness of the feature. BN after 1×1 convolution further stabilizes the output of the network and avoids the problem of gradient disappearance or explosion.

[0080] The input can be directly connected to the output through the skip connection to avoid information loss in the network layer. This connection method can retain the information of the input features, enhance the gradient flow of the network, and make the training more stable. Especially in deep networks, it can alleviate the gradient vanishing problem. Through the skip connection, the original edge features are fused with the features after the convolution operation, which not only retains the initial edge information but also increases the network's expressive power.

[0081] Through multi-level processing and optimization of edge features, this application not only enhances the expression of edge information, but also improves the sensitivity and accuracy of the lake water extraction model to the edge area, which helps to more accurately identify the water body boundaries in the lake water extraction task, thereby improving the overall performance of the lake water extraction model.

[0082] Step 405: Connect the third preliminary edge feature map and the fourth preliminary edge feature map to obtain an edge feature map.

[0083] Step 304: connect the first semantic feature map with the edge feature map, and adjust the weights of channel and spatial information through the first convolutional attention module to obtain a first attention feature map.

[0084] Step 305: Adjust the weights of the channel and spatial information of the second semantic feature map through the second convolutional attention module to obtain a second attention feature map.

[0085] CBAM is a lightweight attention module that can effectively improve the convolutional layer's ability to extract key information. CBAM is a module that combines the channel attention mechanism and the spatial attention mechanism to enhance the representation ability and performance of deep neural networks. The channel attention mechanism is used to adjust the importance between different channels in the feature map, thereby enhancing the network's attention to different features and further improving the feature representation ability. The spatial attention mechanism focuses on features at specific spatial locations, and can more accurately locate and identify target areas in tasks such as image segmentation, thereby improving the model's performance in details and local information. CBAM is first processed by the channel attention mechanism, and then by the spatial attention mechanism to achieve global control and fine-tuning of the feature map, such as Fig. 9 shown.

[0086] The channel attention mechanism in CBAM is as follows Fig.10 As shown in Figure 1, firstly, the global maximum pooling (MaxPool) and global average pooling (AvgPool) are calculated for each channel of the input feature map F (H×W×C) to obtain feature maps with dimensions of (1×1×C). Among them, H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map. is the maximum pooling feature of the feature map, is the average pooling feature of the feature map. Then Entering the shared fully connected layer (Shared MLP), in this layer, the number of channels is first compressed to the original 1 / r (r = 16) through a 1×1 convolution kernel, and then restored to the dimension of (1×1×C) through a 1×1 convolution kernel, which is equivalent to a fully connected transformation. After that, the two output feature matrices are added, and finally the attention weight M of each channel is obtained through the Sigmoid activation function. c (F), the attention weight M of each channel is obtained from the feature map F c The calculation process of (F) is:

[0087]

[0088] Where σ is the Sigmoid activation function, W0 is the weight from the fully connected input layer to the middle layer, and W1 is the weight from the fully connected middle layer to the output layer. c (F) multiply to get the feature map F after giving channel attention c (H×W×C).

[0089] The spatial attention mechanism in CBAM is as follows Fig.11 As shown, the feature map F after channel attention is given c Each pixel position of is subjected to maximum pooling (MaxPool) and average pooling (AvgPool) along the channel direction to obtain two (H×W×1) feature maps. These two feature maps are concatenated and converted into a single-channel feature map after a convolution calculation with a convolution kernel size of 7×7. The spatial attention weight M of each pixel is also obtained by the Sigmoid function. s (F c ), by F c Get M s (F c ) is calculated as follows:

[0090]

[0091] Among them, f 7×7 It is a convolution operation with a convolution kernel size of 7×7.

[0092] The feature map F after giving channel attention c and the spatial attention weight M of each pixel s (F c ) to obtain the feature map F after giving spatial attention. cs (H×W×C).

[0093] Step 306, integrating the global semantic feature map, the first attention feature map and the second attention feature map to obtain a deep semantic feature map.

[0094] Specifically, the first attention feature map is sequentially subjected to convolution and batch normalization processing to obtain a third semantic feature map. After the global semantic feature map is integrated with the second attention feature map, convolution, batch normalization and bilinear interpolation processing are sequentially performed to obtain a fourth semantic feature map. After the third semantic feature map is integrated with the fourth semantic feature map, bilinear interpolation processing is performed to obtain a deep semantic feature map.

[0095] Step 307: The feature fusion module performs feature fusion on the spatial feature map and the deep semantic feature map to obtain a water body feature map that includes both spatial features and semantic features.

[0096] The semantic segmentation task needs to consider both the deep semantic information and shallow spatial information of the image. FFM can effectively integrate the deep semantic information and shallow spatial information, so that the lake water extraction model can simultaneously obtain the semantic features and spatial features of the image, improving the overall segmentation accuracy of the lake water extraction model.

[0097] The structure of the feature fusion module is as follows: Fig.12 As shown, the spatial feature map is first concatenated with the deep semantic feature map, and then processed by 3×1 convolution+BN+ReLU. After that, global information is extracted by global average pooling. The 1×1 convolution layer can further extract features, and the ReLU activation function is used to introduce nonlinearity to enhance the network's expressiveness. The weight is limited between 0 and 1 by the Sigmoid function to achieve attention adjustment of the fused information. This helps the lake water extraction model pay more attention to important features and improves the flexibility and robustness of the lake water extraction model. Jump connections facilitate information transfer and gradient flow, which helps solve the problem of gradient vanishing or gradient explosion in deep network training. Feature addition can retain the information of the original features and prevent information loss, while improving the stability and convergence speed of the lake water extraction model.

[0098] Step 308, performing convolution, bilinear interpolation and normalization processing on the water body feature map containing both spatial features and semantic features in sequence, determining the category of each pixel point in the water body feature map containing both spatial features and semantic features, so as to obtain a final lake water body image.

[0099] In a specific example, if Fig.13As shown, this application uses the loss between the edge information of the target image and the spatial details of the backbone network as an auxiliary loss function to supervise the overall training process of the lake water extraction model. Therefore, the loss function during the training of the lake water extraction model includes the target output loss and the detail extraction loss, that is, L = L seg +L detail ; Where L is the loss function of the lake water extraction model training, L seg is the target output loss value, L detail Extract loss value for details.

[0100] The target output loss is the cross entropy loss. The cross entropy loss function is used for classification tasks. It measures the accuracy of the model's prediction category by calculating the cross entropy between the predicted value and the true value. The formula is as follows:

[0101]

[0102] in, is the predicted value, y i is the one-hot encoding of the category label, c is the number of predicted categories (including background), and σ1 is the Softmax function.

[0103] The predicted value is the result generated by the lake water extraction model for classifying each pixel. In semantic segmentation, the lake water extraction model outputs the probability distribution of each pixel belonging to each category. Each pixel corresponds to multiple predicted values, corresponding to the probabilities of different categories. In this application, there are two predicted values, namely the probabilities of water body and non-water body. In practical applications, the predicted value usually selects the category with the highest probability as the final prediction result. This probability value reflects the confidence of the lake water extraction model that the pixel belongs to a certain category. The category label is the true classification annotated for each pixel, which is used to train the model and evaluate the model performance. In semantic segmentation, the category label is a two-dimensional array of the same size as the input image, in which each element represents the true category of the corresponding pixel.

[0104] The detail extraction loss includes binary cross entropy loss and Dice loss, namely L detail =L bce +L dice Among them, L bce is the binary cross entropy loss value, L dice is the Dice loss value.

[0105] The binary cross entropy loss is:

[0106]

[0107] Among them, H is the height of the feature map, W is the width of the feature map, and σ2 is the Sigmoid function.

[0108] The Dice loss function is a loss function used for image segmentation tasks, which measures the similarity between the predicted results and the true labels. The Dice loss function is designed to solve problems such as class imbalance and blurred boundaries. Its calculation method is:

[0109]

[0110] Among them, ε is a very small number that can prevent the denominator from being 0.

[0111] The lake water extraction model provided in this application has at least the following advantages over conventional lake water extraction models:

[0112] (1) Reduced model complexity: Conventional high-precision semantic segmentation network models are usually complex and difficult to apply in resource-constrained environments. The SE-LWBENet provided in this application uses the STDC backbone network for feature extraction. Through lightweight design, it reduces the model complexity and is suitable for various hardware environments.

[0113] (2) Improved edge feature extraction capability: Traditional methods do not fully extract edge features, resulting in unclear edge recognition. The SE-LWBENet provided in this application uses the Prewitt operator to extract edge features from the input image and the target image, injects edge information into the deep network, and improves the model's ability to capture edges.

[0114] (3) Improved model training effect: Traditional models lack effective constraint mechanisms during training, which affects model accuracy. The SE-LWBENet provided in this application calculates the loss through the edge features of the target image and the shallow spatial features of STDC, and jointly constrains the model training process with the final lake water extraction loss, thereby improving the training effect.

[0115] (4) Improved feature fusion effect: Feature fusion is the key to improving model accuracy. The SE-LWBENet provided in this application uses CBAM to adjust the weights of channel and spatial information of the fused edge features and deep semantic feature maps, so that the model can better focus on the channels and pixel positions related to the target image, thereby improving the feature fusion effect.

[0116] (5) Improved feature map resolution: High-resolution feature maps are the basis for accurate extraction of lake water bodies. The SE-LWBENet provided in this application uses the FFM module to fuse deep and shallow features, and improves the feature map resolution through bilinear interpolation, ultimately achieving high-precision lake water extraction.

[0117] In summary, this application effectively integrates the edge features of the Prewitt operator, STDC, CBAM and FFM to construct a lightweight lake water extraction model, which combines edge features and deep semantic features to improve the ability to capture details and boundaries. By combining the edge feature extraction and loss calculation mechanism with the STDC shallow features, the edge features extracted from the input image are injected into the deep network, and the loss is calculated in combination with the edge features of the target image, which helps to improve the accuracy of water extraction and makes the model pay more attention to edge details, thereby achieving better performance in high-precision water extraction tasks. It successfully solves the technical problems of high complexity of high-precision semantic segmentation models, insufficient edge feature extraction, insufficient training constraints, poor feature fusion effect, insufficient feature map resolution, and achieves accurate extraction of lake water.

[0118] In order to verify the generalization ability of the lake water extraction model proposed in this application, the overall performance of the lake water extraction model was tested using self-built datasets and public datasets, and the performance was compared with the commonly used lightweight semantic segmentation network.

[0119] Specifically, SE-LWBENet was used to extract lake water bodies in the three lake areas respectively. According to the extraction results, it can be determined that SE-LWBENet is very ideal for extracting lake water bodies in the first and second lake areas. From the extraction effect of lake water bodies in the third lake area, whether it is a large area of ​​water or multiple small areas of water, the overlap of their boundaries is also very consistent.

[0120] SE-LWBENet is subjected to ablation experiments on self-built datasets and public datasets respectively. The experimental results are shown in Tables 2 and 3.

[0121] Table 2 Ablation experiments of SE-LWBENet on self-built datasets

[0122]

[0123]

[0124] Table 3 Ablation experiments of SE-LWBENet on public datasets

[0125]

[0126] The following conclusions can be drawn from Tables 2 and 3:

[0127] (1) Performance of each operator in SE-LWBENet: Under the premise that edge features, edge loss and CBAM are complete, for the self-built dataset, the performance of each operator in the model is ranked as Prewitt>Sobel>Roberts>Laplacian>Scharr. For the public dataset, the performance of each operator in the model is ranked as Prewitt>Roberts>Laplacian>Sobel>Scharr.

[0128] (2) Performance of edge features, edge loss, and CBAM in SE-LWBENet: Under the premise of using the optimal operator Prewitt, when edge features, edge loss, and CBAM are used simultaneously, the model achieves the highest MIoU value; when two of the modules are selected, the model performance including edge loss and CBAM is the best; when edge features or edge loss are used alone, the model performance is not as good as when edge features and edge loss are used simultaneously.

[0129] The above ablation experiments show that the Prewitt operator combined with edge feature extraction, edge loss calculation and CBAM is the optimal configuration of the SE-LWBENet network.

[0130] Further study the dimension change process of the feature map in SE-LWBENet: Taking the input image dimension of 224×224×3 as an example, the dimension change process of the image in the SE-LWBENet network is as follows Fig.14 shown.

[0131] In SE-LWBENet, the feature map is divided into two paths from the input end. One path enters the STDC trunk, and then performs 1 / 2 downsampling in each stage, and the other path extracts edge features from the input image. The edge features extracted from the input image are directly fused with the feature map output by the fourth stage, and then the weights of the feature map position and channel are adjusted through CBAM. Only the number of feature map channels is adjusted in the same layer, and the change of feature map scale is only involved when cross-layer feature fusion. The main computation of SE-LWBENet is concentrated on the STDC trunk, and most of the other branches are doing edge feature extraction and dimension adjustment. Since the edge of the image is a binary feature and is usually a single channel, the number of parameters is very limited. The final image resolution is completed by eight times upsampling at one time, and the image resolution is not restored by progressive upsampling based on the classic encoding-decoding structure. Therefore, although SE-LWBENet has a large number of branches, the overall model parameters are very small.

[0132] The model performance of SE-LWBENet is compared with that of five commonly used lightweight semantic segmentation networks: Class-Agnostic Segmentation Network (CANet), PP-LiteSeg, BiSeNet v1, BiSeNet v2 and STDC.

[0133] (1) Comparison of visual observation performance

[0134] By observing the extraction results of lake water bodies by SE-LWBENet and various lightweight semantic segmentation models, it can be seen that BiSeNetv1 has large errors in both edge detail detection and noise resistance. Compared with BiSeNetv1, BiSeNetv2, CANet and PP-LiteSeg have improved performance in spatial detail detection, but still have flaws in noise resistance. STDC has significantly improved in both spatial detail recognition and noise resistance. SE-LWBENet can not only completely segment large water areas, but also extract the subtle boundaries of water bodies or small area boundaries more accurately; in terms of noise resistance, SE-LWBENet is significantly better than other models in extracting the lake body in the fifth column. Therefore, the SE-LWBENet proposed in this application is superior to other advanced lightweight network models in both spatial detail recognition and noise resistance.

[0135] In order to more accurately quantify the performance of the lake water extraction model and verify the generalization ability of the lake water extraction model, the lightweight semantic segmentation models and SE-LWBENet were tested using self-built datasets and public datasets. The performance indicators of each model are shown in Tables 4 and 5.

[0136] Table 4 Performance comparison of lightweight models on self-built datasets

[0137]

[0138] Table 5 Performance comparison of lightweight models on public datasets

[0139]

[0140] It can be seen from Tables 4 and 5 that the order of model complexity is CANet>BiSeNet v1>STDC>SE-LWBENet>PP-LiteSeg>BiSeNet v2. The order of model accuracy is SE-LWBENet>STDC>BiSeNetv1>CANet>PP-LiteSeg>BiSeNetv2. From the ranking of model complexity and accuracy, it can be seen that although PP-LiteSeg and BiSeNet v2 have the lowest model complexity, the corresponding model accuracy is also poor. The SE-LWBENet proposed in this application is second only to PP-LiteSeg and BiSeNetv2 in model complexity, but it has achieved the best model accuracy. Compared with STDC, which ranks second in model accuracy, SE-LWBENet has model accuracy on self-built data sets that is 0.02, 0.03, and 0.05 percentage points higher, respectively; on public data sets, the model accuracy is 0.08, 0.09, and 0.16 percentage points higher, respectively, and the inference speed is increased by more than 7 times. Experimental results show that SE-LWBENet has better spatial detail extraction and noise resistance performance, a lighter model, and strong generalization ability. It is a more excellent lake water extraction network model.

[0141] In an exemplary embodiment, a lake water body extraction system is provided, comprising: an image acquisition unit and a water body extraction unit. The image acquisition unit is used to acquire a remote sensing image of a region to be detected. The water body extraction unit is used to extract the water body in the remote sensing image based on a lake water body extraction model to obtain a lake water body image to determine the water body area of ​​the region to be detected.

[0142] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned lake water extraction method when executing the computer program.

[0143] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned lake water body extraction method is implemented.

[0144] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the above-mentioned lake water body extraction method when executed by a processor.

[0145] It should be noted that the data involved in this application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0146] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A lake water extraction method, characterized in that: The lake water extraction method comprises: Acquire remote sensing images of the area to be inspected; Extracting water bodies in the remote sensing image based on a lake water body extraction model to obtain a lake water body image to determine the water body area of ​​the area to be detected; the lake water body extraction model is obtained by pre-training with a training sample set; the training sample set includes a plurality of sample remote sensing images and a lake water body image corresponding to each sample remote sensing image; the lake water body image is marked with a category of each pixel point, and the category of the pixel point is water body or non-water body; Among them, the lake water extraction model includes a lightweight feature extraction network, a global average pooling layer, an edge extraction module, a first convolutional attention module, a second convolutional attention module and a feature fusion module; the global average pooling layer is connected to the lightweight feature extraction network; the first convolutional attention module is respectively connected to the lightweight feature extraction network and the edge extraction module; the second convolutional attention module is respectively connected to the lightweight feature extraction network and the global average pooling layer; the feature fusion module is respectively connected to the lightweight feature extraction network, the global average pooling layer, the first convolutional attention module and the second convolutional attention module.

2. The lake water extraction method according to claim 1, characterized in that: Extracting the water body in the remote sensing image based on the lake water body extraction model to obtain a lake water body image specifically includes: Extracting semantic features of the remote sensing image through the lightweight feature extraction network to obtain a spatial feature map, a first semantic feature map, and a second semantic feature map; Performing a pooling operation on the second semantic feature map through the global average pooling layer to obtain a global semantic feature map; Perform edge extraction on the remote sensing image by using the edge extraction module to obtain an edge feature map; Connecting the first semantic feature map with the edge feature map, and adjusting the weights of channel and spatial information through the first convolutional attention module to obtain a first attention feature map; Adjusting the weights of the channel and spatial information of the second semantic feature map through the second convolutional attention module to obtain a second attention feature map; Integrate the global semantic feature map, the first attention feature map, and the second attention feature map to obtain a deep semantic feature map; The spatial feature map and the deep semantic feature map are subjected to feature fusion by the feature fusion module to obtain a water body feature map containing both spatial features and semantic features; The water body feature map including spatial features and semantic features is sequentially subjected to convolution, bilinear interpolation and normalization processing to determine the category of each pixel point in the water body feature map including both spatial features and semantic features, so as to obtain a final lake water body image.

3. The lake water extraction method according to claim 2, characterized in that: The lightweight feature extraction network includes a first convolutional layer, a second convolutional layer, a first semantic segmentation module, a second semantic segmentation module and a third semantic segmentation module connected in sequence; The spatial feature map is a feature map output by the first semantic segmentation module, the first semantic feature map is a feature map output by the second semantic segmentation module, and the second semantic feature map is a feature map output by the third semantic segmentation module.

4. The lake water extraction method according to claim 2, characterized in that: The edge extraction module is used to extract the edge of the remote sensing image to obtain an edge feature map, which specifically includes: Downsampling the remote sensing image using a 1 / 8 scale Prewitt operator to obtain a first preliminary edge feature map; Downsampling the remote sensing image using a 1 / 16 scale Prewitt operator to obtain a second preliminary edge feature map; After sequentially performing convolution, batch normalization, nonlinear activation, convolution, and batch normalization on the first preliminary edge feature map, the first preliminary edge feature map is integrated with the first preliminary edge feature map, and then sequentially performing nonlinear activation, convolution, and batch normalization on the first preliminary edge feature map to obtain a third preliminary edge feature map; After sequentially performing convolution, batch normalization, nonlinear activation, convolution, and batch normalization on the second preliminary edge feature map, the second preliminary edge feature map is integrated with the second preliminary edge feature map, and then the nonlinear activation process is performed to obtain a fourth preliminary edge feature map; The third preliminary edge feature map is connected with the fourth preliminary edge feature map to obtain an edge feature map.

5. The lake water extraction method according to claim 2, characterized in that: Integrating the global semantic feature map, the first attention feature map and the second attention feature map to obtain a deep semantic feature map specifically includes: Performing convolution and batch normalization processing on the first attention feature map in sequence to obtain a third semantic feature map; After integrating the global semantic feature map with the second attention feature map, convolution, batch normalization and bilinear interpolation are performed in sequence to obtain a fourth semantic feature map; After the third semantic feature map and the fourth semantic feature map are integrated, bilinear interpolation processing is performed to obtain a deep semantic feature map.

6. The lake water extraction method according to claim 1, characterized in that: The loss function during the training of the lake water body extraction model includes target output loss and detail extraction loss; the target output loss is a cross entropy loss, and the detail extraction loss includes a binary cross entropy loss and a Dice loss.

7. A lake water extraction system, applied to the lake water extraction method according to any one of claims 1 to 6, characterized in that: The lake water extraction system comprises: An image acquisition unit, used to acquire remote sensing images of the area to be detected; A water body extraction unit is used to extract the water body in the remote sensing image based on a lake water body extraction model to obtain a lake water body image to determine the water body area of ​​the area to be detected; the lake water body extraction model is obtained by pre-training with a training sample set; the training sample set includes a plurality of sample remote sensing images and a lake water body image corresponding to each sample remote sensing image; the lake water body image is marked with a category of each pixel point, and the category of the pixel point is water body or non-water body; Among them, the lake water extraction model includes a lightweight feature extraction network, a global average pooling layer, an edge extraction module, a first convolutional attention module, a second convolutional attention module and a feature fusion module; the global average pooling layer is connected to the lightweight feature extraction network; the first convolutional attention module is respectively connected to the lightweight feature extraction network and the edge extraction module; the second convolutional attention module is respectively connected to the lightweight feature extraction network and the global average pooling layer; the feature fusion module is respectively connected to the lightweight feature extraction network, the global average pooling layer, the first convolutional attention module and the second convolutional attention module.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the lake water extraction method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the lake water extraction method described in any one of claims 1-6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the lake water extraction method described in any one of claims 1-6 is implemented.

Images