Water body region extraction method and device

Abstract

The present invention relates to the technical field of remote sensing image processing, and provides a method and device for extracting a water body region, the method comprising: determining a remote sensing image of an ice lake region, the ice lake region including glaciers and lakes around the glaciers; extracting spectral features of the remote sensing image, and determining a spectral feature image for highlighting the region corresponding to the lake in the remote sensing image; extracting a water body region in the remote sensing image based on the spectral feature image, and determining a water body image, the water body image indicating a region in the remote sensing image that may represent a lake; encoding the water body image to extract water body features, and determining water body features; decoding a water body feature map to segment the water body region in the remote sensing image, and determining a water body segmentation image corresponding to the remote sensing image, the water body segmentation image being used to distinguish between the region corresponding to the lake in the remote sensing image and other regions. Through the technical solution of the embodiment of the present invention, the region representing the lake in the remote sensing image can be segmented more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a method and a device for extracting a water body region. Background Art

[0002] Glacial lakes are a type of lake developed around glaciers. They are important water resources in plateau areas. Monitoring glacial lakes can provide a basis for ecological planning and climate change in plateau areas. An important means of monitoring glacial lakes is to extract glacial lake information from remote sensing images.

[0003] Ice lake extraction aims to provide a method that can quickly extract ice lake information from images. According to the input remote sensing image, the characteristics of the ice lake in the image are captured and the ice lake information is highlighted to achieve fast and accurate extraction of the ice lake. At present, although some results have been achieved in ice lake extraction, most methods still cannot achieve automated high-precision extraction.

[0004] Existing research methods for glacial lake extraction are mainly based on prior knowledge. Specifically, methods based on prior knowledge include: ① Visual interpretation. Experienced experts are often required to identify glacial lake information in remote sensing images, mark them, and draw glacial lake thematic maps. This type of method requires a certain understanding of glacial lakes to accurately identify glacial lake information. Moreover, this type of method is time-consuming and laborious. Some small glacial lakes and glacial lakes that are easily confused with the background are often difficult to identify. ② Methods based on water body index. Since the reflectivity of glacial lakes is low in the longer wavelength band and high in the shorter wavelength band, a water body index can be designed based on this principle to highlight the water body information in remote sensing images and suppress background information. Then, according to the water body index range of the glacial lake, a threshold is set to separate the glacial lake information. Typical water body indices include NDWI[2], MNDWI[3], EWI[4], etc. This type of method is simple to operate, but has large errors. Many non-ice lake targets, such as mountain shadows, glaciers in a molten state, and clouds, also show exponential characteristics similar to ice lakes. It is often difficult to directly separate these targets by setting thresholds, which brings a lot of trouble to post-processing. ③ Methods based on image segmentation. Since the surface of the ice lake is homogeneous, the ice lake information can be extracted based on this feature. Representative methods include CV models [5] and object-oriented methods [6]. This type of method has high segmentation accuracy, but it requires multiple attempts to determine the best segmentation parameters. In addition, there is still a part that is similar to the surface of the ice lake, such as clouds, glaciers, shadows, etc. The surface of these materials is also homogeneous, and this type of target also needs to be removed through post-processing.

[0005] However, the above-mentioned glacial lake extraction method fails to accurately extract the water area in the remote sensing image of the glacial lake. Summary of the invention

[0006] The present invention provides a water body region extraction method, device, computer-readable storage medium and electronic equipment, which can more accurately segment the area representing the lake in the remote sensing image.

[0007] In a first aspect, the present invention provides a method for extracting a water body region, comprising:

[0008] Determine a remote sensing image of a glacial lake region, wherein the glacial lake region includes a glacier and lakes surrounding the glacier, and the remote sensing image is an image obtained by long-distance detection of the glacial lake region;

[0009] Extracting spectral features from the remote sensing image to determine a spectral feature image of the glacial lake area, wherein the spectral feature image is used to highlight the area of ​​the lake corresponding to the remote sensing image;

[0010] Extracting a water body region in the remote sensing image based on the spectral feature image and determining a water body image, wherein the water body image indicates a region in the remote sensing image that may represent the lake;

[0011] Encoding the water body image to extract water body features and determine a water body feature map, wherein the water body feature map indicates information of the lake included in the water body image;

[0012] The water body feature map is decoded to segment the water body area in the remote sensing image, and a water body segmentation image corresponding to the remote sensing image is determined, wherein the water body segmentation image is used to distinguish the area corresponding to the lake in the remote sensing image from other areas.

[0013] In a second aspect, the present invention provides a water body region extraction device, comprising:

[0014] A remote sensing image determination module is used to determine a remote sensing image of a glacial lake area, wherein the glacial lake area includes a glacier and lakes surrounding the glacier, and the remote sensing image is an image obtained by long-distance detection of the glacial lake area;

[0015] A spectral image determination module, used for extracting spectral features from the remote sensing image to determine a spectral feature image of the glacial lake area, wherein the spectral feature image is used to highlight the area corresponding to the lake in the remote sensing image;

[0016] A water body extraction module, for extracting a water body region in the remote sensing image based on the spectral feature image, and determining a water body image, wherein the water body image indicates an area in the remote sensing image that may represent the lake;

[0017] An encoding module, used for encoding the water body image to extract water body features and determine a water body feature map, wherein the water body feature map indicates information of the lake included in the water body image;

[0018] A decoding module is used to decode the water body feature map to segment the water body area in the remote sensing image, determine the water body segmentation image corresponding to the remote sensing image, and the water body segmentation image is used to distinguish the area corresponding to the lake in the remote sensing image from other areas.

[0019] In a third aspect, the present invention provides a computer-readable storage medium, comprising execution instructions. When a processor of an electronic device executes the execution instructions, the processor executes any method described in the first aspect.

[0020] In a fourth aspect, the present invention provides an electronic device, comprising a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor executes any method described in the first aspect.

[0021] The present invention provides a method, device, computer-readable storage medium and electronic device for extracting a water body region. The method determines a remote sensing image of an ice lake region, wherein the ice lake region includes glaciers and lakes around the glaciers, and the remote sensing image is an image obtained by long-distance detection of the ice lake region; then, spectral feature extraction is performed on the remote sensing image to determine a spectral feature image of the ice lake region, and the spectral feature image is used to highlight the region corresponding to the lake in the remote sensing image; then, based on the spectral feature image, the water body region in the remote sensing image is extracted, and the water body image is determined, and the water body image indicates the region in the remote sensing image that may represent the lake; then, the water body image is encoded to extract water body features, and a water body feature map is determined, and the water body feature map indicates the information of the lake included in the water body image; then, the water body feature map is decoded to segment the water body region in the remote sensing image, and a water body segmentation image corresponding to the remote sensing image is determined, and the water body segmentation image is used to distinguish the region corresponding to the lake in the remote sensing image from other regions. In summary, through the technical solution of the present invention, since the water body region in the image is highlighted, the water body region in the image can be more accurately located, and then the region representing the lake in the remote sensing image can be more accurately segmented.

[0022] The further effects of the above-mentioned non-conventional preferred manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the existing technical solutions, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1A schematic flow chart of a method for training a water body extraction model provided by an embodiment of the present invention;

[0025] Figure 2a Schematic diagram of the structure of the water body extraction model provided in the embodiment of the present invention Figure 1 ;

[0026] Figure 2b A second structural diagram of a water body extraction model provided by an embodiment of the present invention;

[0027] Figure 2c Schematic diagram of the structure of the water body extraction model provided in the embodiment of the present invention Figure 3 ;

[0028] Figure 2d Schematic diagram of the structure of the water body extraction model provided in the embodiment of the present invention Figure 4 ;

[0029] Figure 3 A schematic diagram of the image processing flow of the first model provided in an embodiment of the present invention;

[0030] Figure 4 A schematic flow chart of a water body region extraction method provided by an embodiment of the present invention;

[0031] Figure 5 A schematic diagram of the structure of a water body region extraction device provided by an embodiment of the present invention;

[0032] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] like Figure 1 As shown in FIG. 1 , a training flow diagram of a water body region extraction model provided in an embodiment of the present invention is shown. Figures 2a to 2d As shown, the water body extraction model includes a first model, a second model and a third model, and the training process of the water body extraction model includes the following steps:

[0035] Step 101: determine a training set, where the training set includes a plurality of remote sensing images of a plurality of glacial lake areas and annotated images corresponding to the plurality of remote sensing images, wherein the annotated images annotate the lakes in the remote sensing images and the areas outside the lakes.

[0036] As a feasible implementation, the size of the remote sensing image is 256×256×7 pixels.

[0037] As a feasible implementation method, the labeled image is a binary image. For example, the remote sensing image can be a black and white image, where white represents the lake and black represents the area outside the lake.

[0038] It should be noted that the glacial lake area includes glaciers and lakes around glaciers.

[0039] Step 102: For each image in the plurality of remote sensing images, extract the spectral feature of the remote sensing image and determine the spectral feature image corresponding to the remote sensing image.

[0040] It should be noted that the spectral feature image has the same size as the remote sensing image, and is used to highlight the water area in the remote sensing image, so that the model can notice the water area in the image. It is worth noting that the spectral feature image can represent the spectral information used to distinguish the water body from other objects in the image.

[0041] For example, since the water index can highlight the water area in the remote sensing image and suppress the background area, the spectral feature image can be an image of the remote sensing image processed by the water index. In order to better characterize the glacial lake information in the remote sensing image, the water index selects NDWI (Normalized Difference Water Index).

[0042] Step 103: For each of the multiple remote sensing images, substitute the remote sensing image and the corresponding spectral feature image into the first model to determine a water body image output by the first model, where the water body image indicates an area in the remote sensing image that may represent a lake.

[0043] As a feasible implementation method, the first model adopts the attention mechanism.

[0044] In one example, the first model includes a key weight matrix and a query weight matrix. Correspondingly, the first model is used to determine the key graph corresponding to the remote sensing image based on the key weight matrix; determine the query graph corresponding to the remote sensing image based on the query weight matrix; then, the first model calculates the key graph and the query graph to determine the weight graph, and the weight graph indicates the water body attention weight of each pixel in the spectral feature image corresponding to the remote sensing image; then, the first model calculates the weight graph and the spectral feature image corresponding to the remote sensing image to determine the attention graph, and the attention graph indicates the attention size of each pixel in the remote sensing image, which can also be understood as the water body attention weight; thereafter, the first model calculates the attention graph and the remote sensing image, pays attention to and extracts the water body area in the remote sensing image, and determines the water body image. The obtained water body image can roughly represent the area representing the lake that may exist in the remote sensing image.

[0045] For example, Figure 3 As shown, the water body image can be determined by the following steps:

[0046] Step 1: Use a 1×1 convolution kernel to process the input remote sensing image to obtain the key image of the remote sensing image.

[0047] Step 2: Use a 1×1 convolution kernel to process the input remote sensing image to obtain the query graph of the remote sensing image.

[0048] Step 3: Perform matrix product on the key graph and the query graph to obtain a new weighted graph with the spatial location information of the water body.

[0049] Step 4: Use the spectral characteristic image corresponding to the remote sensing image (the image processed by the water body index NDWI) as the value map (value).

[0050] Step 5: Perform matrix product of the weight map obtained in step 3 and the value map obtained in step 4 to obtain the attention map.

[0051] Step 6, perform matrix product of the attention map obtained in step 5 and the input remote sensing image to obtain a water body image, which can be understood as a water body attention image.

[0052] Step 104: For each image in the plurality of remote sensing images, substitute the water body image corresponding to the remote sensing image into the second model for encoding to extract water body features, and determine a water body feature map output by the second model.

[0053] As a feasible implementation manner, the second model includes M downsampling blocks, and the water body characteristic map includes the outputs of the M downsampling blocks respectively.

[0054] like Figure 2a to Figure 2dAs shown, for the ith downsampling block among the M downsampling blocks, when i=1, the input of the ith downsampling block is the water body image, and when i is greater than 1, the input of the ith downsampling block is the output of the i-1th downsampling block.

[0055] In one example, the downsampling block includes two convolutional layers and a downsampling layer connected in series, and the output of the downsampling layer is the output of the downsampling block. In order to ensure that the downsampling block can extract smaller lakes in the remote sensing image, the downsampling layer is replaced by convolution, making full use of the image information and reducing the possibility of information loss. For example, the size of the convolution kernel of the two convolutional layers is 3*3, and the size of the convolution kernel of the downsampling layer is also 3*3.

[0056] It should be noted that the size of the feature maps output by each of the M downsampling blocks is different, so that the water body feature map can represent water body characteristics at multiple scales.

[0057] It should be noted that this embodiment does not intend to limit the number of downsampling blocks, which needs to be determined in combination with actual needs, such as M=5.

[0058] It should also be noted that the above third model is only an example and does not constitute a specific limitation. The structure of the third model can be determined in combination with actual needs.

[0059] Step 105: For each image in the plurality of remote sensing images, substitute the water body feature map corresponding to the remote sensing image into the third model for decoding to segment the water body area in the remote sensing image, and determine the water body segmentation image corresponding to the remote sensing image output by the third model.

[0060] It should be noted that the water body segmentation image, the annotated image and the remote sensing image have the same size, and the water body segmentation image is a binary image.

[0061] As a feasible implementation, the third model includes a decoding unit and a water body classification unit, and the water body segmentation image is the output of the water body classification unit. The input of the decoding unit is the output of each of the M downsampling blocks, which is used to decode the water body feature map; the input of the water body classification unit is the output of the decoding unit, or the output of the decoding unit and the output of each of at least one of the M downsampling blocks.

[0062] The decoding unit includes N upsampling blocks. The upsampling block is used to decode the input to determine a water body probability image, which indicates the water body probability of each pixel in the remote sensing image. It is worth noting that when the size of the water body probability image is smaller than the size of the remote sensing image, one pixel in the water body probability image can represent the water body probability shared by multiple pixels in the remote sensing image.

[0063] In one example, in order to ensure that the upsampling block can restore the smaller lakes in the remote sensing image, the upsampling layer is replaced by convolution to fully restore the image information and reduce the possibility of information loss. For example, the size of the convolution kernel of the two convolution layers is 3*3, and the size of the convolution kernel of the upsampling layer is also 3*3.

[0064] In one example, for the j-th upsampling block among the N upsampling blocks, when j=1, the input of the j-th upsampling block is the output of a connected downsampling block or the output of each of the multiple connected downsampling blocks is spliced ​​or fused; when j is greater than 1, the input of the j-th upsampling block is the output of the connected j-1-th upsampling block or the output of the connected j-1-th upsampling block is fused or spliced ​​with the output of at least one connected downsampling block; correspondingly, the output of the decoding unit is the output of the upsampling block located at the last position, or the output of each of the N upsampling blocks.

[0065] For example, Figure 2a As shown, the second model includes 4 downsampling blocks, and the decoding unit includes 4 upsampling blocks. For the jth upsampling block among the 4 upsampling blocks, when j=1, the input of the upsampling block is the feature map output by the connected 4th downsampling block; when j>1, the input of the jth upsampling block is the output of the j-1th upsampling block and the output of the connected 5-jth downsampling block. Correspondingly, the input of the water classification unit is the output of the 4th upsampling block.

[0066] For example, Figure 2b and Figure 2c As shown in , the second model includes 5 downsampling blocks, and the decoding unit includes 4 upsampling blocks. For the jth upsampling block among the 4 upsampling blocks, when j=1, the input of the upsampling block is the feature map after the output of the connected 4th and 5th downsampling blocks are fused or spliced; when j>1, the input of the jth upsampling block is the result of the splicing or fusion of the output of the j-1th upsampling block and the output of the connected 5-jth downsampling block. Figure 2b As shown in , the input of the water body classification unit is the output of the 4th upsampling block. Figure 2c As shown, the input of the water body classification unit is the result of splicing or fusion of the outputs of the 1st to 4th upsampling blocks.

[0067] The sizes of the multiple images input to the jth upsampling block are the same, so they can be directly fused or concatenated. Fusion can be understood as addition, or averaging after addition.

[0068] In one example, for the j-th upsampling block among N upsampling blocks, the input of the j-th upsampling block is the output of a connected downsampling block or the result of splicing or fusion of the outputs of multiple connected downsampling blocks, and the input of the water body classification unit is the output of each of the N upsampling blocks.

[0069] For example, Figure 2d As shown, the second model includes 4 upsampling and downsampling blocks, and the decoding unit includes 4 upsampling blocks. For the jth upsampling block among the 4 upsampling blocks, the input of the jth upsampling block is the feature map output by the connected 5th-jth downsampling blocks. Correspondingly, the input of the water body classification unit is the output of each of the 4 upsampling blocks, which is used to fuse the outputs of each of the N upsampling blocks to perform water body classification and output a water body segmentation image.

[0070] It should be noted that since the probability range of the water body probability image is 0-1, it is not a binary image, so it is necessary to classify the water body to obtain a binary image.

[0071] It should also be noted that the third model is merely an example and does not constitute a specific limitation, and the structure of the third model can be determined in combination with actual needs. For example, the second model and the third model can be U-net models.

[0072] Step 106: For each sample in the multiple remote sensing images, the water body segmentation image and the annotated image corresponding to the remote sensing image are substituted into the discriminant model for judgment to determine the judgment result of the water body segmentation image, and the judgment result indicates whether the water body segmentation image is an annotated image or an image generated by the water body extraction model.

[0073] As a feasible implementation method, Figure 2a to Figure 2d As shown in the figure, the discriminant model includes a water region extraction layer, ResNet, two fully connected layers and a nonlinear activation layer. The water region extraction layer is used to extract the water region in the remote sensing image as output based on the water segmentation image, and to extract the water region in the remote sensing image as output based on the labeled image. ResNet is used to extract the water features of the image output by the image processing layer, and the two fully connected layers and the nonlinear activation layer are used to classify the water body based on the output of ResNet.

[0074] It should be noted that the structure of the above-mentioned discriminant model is only an example and does not constitute a specific limitation. The structure of the discriminant model can be determined in combination with actual needs.

[0075] Step 107: Based on the labeled images, water body segmentation images, and judgment results of the water body segmentation images corresponding to the multiple remote sensing images, the discrimination model and the water body extraction model are trained.

[0076] The loss of the discriminant model and the water body extraction model includes two parts, one is the first loss of the discriminant model discrimination process, and the other is the second loss between the generated result (water body segmentation image) and the true result (labeled image), for example, the L2 norm loss.

[0077] In the actual training process, the discriminant model is first trained based on the first loss. When the discriminant model has a good performance, the water body extraction model is trained based on the second loss. The training is rotated in sequence to finally obtain a trained water body extraction model.

[0078] It should be noted that training a water body extraction model based on a discriminant model is merely an example and does not constitute a specific limitation. The training method of the water body extraction model can be determined in combination with actual needs. For example, the water body extraction model can be trained directly based on the annotated images and water body segmentation images corresponding to multiple remote sensing images.

[0079] Through the above technical solution, it can be seen that the beneficial effects of this embodiment are:

[0080] On the one hand, the accuracy of the water body extraction model can be improved by training the water body extraction model through the discriminant model.

[0081] On the other hand, the water body extraction model proposed in this embodiment adopts convolution method for sampling, which reduces the information loss of smaller lakes and can more accurately locate the area representing the lake in the image, which is more suitable for the extraction research of lakes around glaciers at different scales.

[0082] On the other hand, this embodiment focuses on the water area in the remote sensing image, thereby performing targeted processing on the water area in the remote sensing image, and can more accurately locate the area representing the lake in the image, thereby ensuring the reference value of the extraction result of the area representing the lake in the remote sensing image.

[0083] On the other hand, no auxiliary data is required. Traditional ice lake extraction methods generally only consider the spectral characteristics or spatial characteristics of ice lakes (homogenization of water surface), while ignoring other characteristics. This leads to the fact that ice lake extraction often requires other auxiliary data to remove interference from targets with similar characteristics to ice lakes, such as using DEM data to remove mountain shadows. The water body extraction model provided in this embodiment uses a convolution operation to extract the convolution features of ice lakes in the image, so that ice lake extraction can be effectively performed without the need for other auxiliary data, thereby realizing the automatic extraction of ice lakes. In addition, this embodiment can directly, quickly and accurately locate the ice lake area through the study of a large number of samples, and realize the extraction of ice lakes without the need for pre-processing or post-processing operations on remote sensing images, such as pixel value conversion, digital elevation model generation of slope maps and mountain shading maps, and denoising. The method proposed in the present invention does not require any pre-processing or post-processing.

[0084] Furthermore, the inventors Figure 2c The effect of the model shown is verified. The specific experimental process is as follows.

[0085] 1. Experimental conditions

[0086] The present invention is that the central processing unit is The model is implemented using Python programming on an i5-9400F 2.9GHz CPU, GTX 1660T 6G GPU, 16G memory, and WINDOWS 10 operating system. The model also involves a deep learning framework, and the deep learning framework used in this experiment is tensorflow 1.14. The data used in the experiment are all collected from Landsat (satellite) images, and all images are 256×256×7 in size.

[0087] 2. Experimental content

[0088] The calculation method for experimental accuracy evaluation is the F1 score. The specific calculation method of the F1 score is as follows:

[0089] Accuracy = correctly extracted glacial lake pixels / all extracted pixels

[0090] Recall rate = correctly extracted ice lake pixels / all ice lake pixels

[0091] F1 score = 2 × precision × recall / (precision + recall)

[0092] The size of the input image is set to 256 × 256 × 7. Regarding the training parameters, the batch size is set to 8, the epoch is set to 100, the dropout is set to 0.5 to prevent overfitting, the optimizer selects AdamOptimizer, and the learning rate is set to 0.0001.

[0093] To verify the effectiveness of the embodiment of the present invention, we compared the models in two papers that also used deep learning to extract ice lakes. The models used in these papers are U-net. This model is relatively simple, while the model proposed in the present invention not only introduces a water body attention module, but also uses a generative adversarial network structure. Through two sets of loss functions, the generator is optimized to perform ice lake segmentation. The accuracy comparison results of the final model are as follows:

[0094] Model Name F1 score U-net 83.53％ The model proposed in this example 92.17％

[0095] Table 1

[0096] As can be seen from Table 1, compared with U-net, the model proposed in the embodiment of the present invention has higher accuracy. Due to the introduction of water body attention and generative adversarial network, the ice lake area can be more accurately located in the model, so it has higher accuracy and is more suitable for large-scale ice lake extraction research.

[0097] like Figure 4 As shown, a method provided by an embodiment of the present invention is provided. The method provided by an embodiment of the present invention can be applied to an electronic device, and can be specifically applied to a server or a general computer. In this embodiment, the method specifically includes the following steps:

[0098] Step 401: determine a remote sensing image of an ice lake area, where the ice lake area includes glaciers and lakes surrounding the glaciers, and the remote sensing image is an image obtained by long-distance detection of the ice lake area.

[0099] Step 402: extract spectral features from the remote sensing image to determine the spectral feature image of the glacial lake area. The spectral feature image is used to highlight the area corresponding to the lake in the remote sensing image.

[0100] In this step, by highlighting the water area in the remote sensing image, the water area in the remote sensing image can be better located to be extracted, which facilitates the subsequent water body segmentation.

[0101] As a feasible implementation method, the water index value of each pixel in the remote sensing image is determined by the calculation formula of the water index, thereby obtaining a spectral characteristic image. For details, please refer to the above text and will not be elaborated here.

[0102] Step 403: extract the water body area in the remote sensing image based on the spectral feature image and determine the water body image, which indicates the area in the remote sensing image that may represent a lake.

[0103] In this step, by highlighting the water area in the remote sensing image, the water area in the remote sensing image can be better located to be extracted, thereby obtaining a water image that is convenient for subsequent water body segmentation.

[0104] As a feasible implementation method, the water body image can be determined by the following implementation method:

[0105] Based on the remote sensing image, the water body attention weights of different areas in the spectral feature image are determined; based on the water body attention weights of different areas in the spectral feature image and the spectral feature image, the water body attention weights of different areas in the remote sensing image are determined; based on the water body attention weights of different areas in the remote sensing image, the water body area in the remote sensing image is extracted to determine the water body image.

[0106] In one example, the water body attention weight can be determined in the following implementation method:

[0107] The key graph corresponding to the remote sensing image is determined based on the key weight matrix; the query graph corresponding to the remote sensing image is determined based on the query weight matrix; attention calculation is performed based on the key graph and the query graph to determine the water body attention weights of different areas in the spectral feature image.

[0108] In practical applications, the spectral characteristic graph and the remote sensing image are input into the first model, and the first model is determined to output the water body image.

[0109] For details, please refer to the above text and will not be repeated here.

[0110] Step 404: Encode the water body image to extract water body features and determine a water body feature map, which indicates information about the lake included in the water body image.

[0111] In this step, attention is paid to the information of the area in the water body image that may represent the lake, so as to better characterize the situation of the lake through the water body feature map, which is convenient for the subsequent water body segmentation.

[0112] As a feasible implementation method, the water body image is substituted into the second model to extract water body features of different scales, so as to determine the water body feature map output by the second model.

[0113] In one example, a water body feature map includes multiple feature maps with the same image size, and these feature maps are used to describe information about a lake included in the remote sensing image.

[0114] Please refer to the above for details, which will not be repeated here.

[0115] Step 405: decode the water body feature map to segment the water body area in the remote sensing image, determine the water body segmentation image corresponding to the remote sensing image, and the water body segmentation image is used to distinguish the area corresponding to the lake in the remote sensing image from other areas.

[0116] In this step, the water body feature map is decoded to restore the water body area in the remote sensing image, thereby achieving segmentation of the water body area and obtaining a water body segmentation image of the same size as the remote sensing image.

[0117] As a feasible implementation method, the water body image is substituted into the third model to extract water body features, thereby determining the water body feature map output by the second model.

[0118] Please refer to the above for details, which will not be repeated here.

[0119] Through the above technical solution, it can be seen that the beneficial effects of this embodiment are:

[0120] Since the water area in the image is highlighted, the water area in the image can be located more accurately, and then the area representing the lake in the remote sensing image can be segmented more accurately.

[0121] Based on the same concept as the embodiment of the method of the present invention, please refer to Figure 5 The embodiment of the present invention further provides a water body region extraction device, comprising:

[0122] A remote sensing image determination module 501 is used to determine a remote sensing image of a glacial lake area, wherein the glacial lake area includes a glacier and lakes surrounding the glacier, and the remote sensing image is an image obtained by long-distance detection of the glacial lake area;

[0123] A spectral image determination module 502 is used to extract spectral features from the remote sensing image to determine a spectral feature image of the ice lake area, wherein the spectral feature image is used to highlight the area corresponding to the lake in the remote sensing image;

[0124] A water body extraction module 503 is used to extract a water body region in the remote sensing image based on the spectral feature image and determine a water body image, wherein the water body image indicates a region in the remote sensing image that may represent the lake;

[0125] An encoding module 504 is used to encode the water body image to extract water body features and determine a water body feature map, wherein the water body feature map indicates information of the lake included in the water body image;

[0126] The decoding module 505 is used to decode the water body feature map to segment the water body area in the remote sensing image, determine the water body segmentation image corresponding to the remote sensing image, and the water body segmentation image is used to distinguish the area corresponding to the lake in the remote sensing image from other areas.

[0127] As a feasible implementation, the water body extraction module 503 includes: a first weight calculation unit, a second weight calculation unit and an extraction unit; wherein,

[0128] The first weight calculation unit is used to determine the water body attention weights of different areas in the spectral feature image based on the remote sensing image;

[0129] The second weight calculation unit is used to determine the water body attention weights of different areas in the remote sensing image based on the water body attention weights of different areas in the spectral feature image and the spectral feature image;

[0130] The extraction unit is used to extract the water body area in the remote sensing image based on the water body attention weights of different areas in the remote sensing image to determine the water body image.

[0131] As a feasible implementation, the first weight calculation unit includes: a key determination subunit, a query determination subunit and a weight calculation subunit; wherein,

[0132] The key determination subunit is used to determine the key graph corresponding to the remote sensing image based on the key weight matrix;

[0133] The query determination subunit is used to determine the query graph corresponding to the remote sensing image based on the query weight matrix;

[0134] The weight calculation subunit is used to perform attention calculation based on the key graph and the query graph to determine the water body attention weights of different areas in the spectral feature image.

[0135] As a feasible implementation manner, the water body feature map includes a plurality of feature maps of different sizes, and the plurality of feature maps are all used to represent the information of the lake included in the water body image.

[0136] As a feasible implementation, the decoding module 505 includes: a first decoding unit and a first classification unit; wherein,

[0137] The first decoding unit is used to decode the multiple feature maps in sequence to determine a water body probability image obtained by the last decoding and having the same size as the remote sensing image, wherein the water body probability image indicates the water body probabilities of different regions in the remote sensing image;

[0138] The first classification unit is used to perform water body classification on the water body probability image obtained by the last decoding, and determine the water body segmentation image corresponding to the remote sensing image.

[0139] As a feasible implementation, the decoding module 505 includes: a second decoding unit and a second classification unit; wherein,

[0140] The second decoding unit is used to decode the multiple feature maps respectively, determine the water body probability images corresponding to the multiple feature maps and having the same size as the remote sensing image, and the water body probability images indicate the water body probabilities of different areas in the remote sensing image;

[0141] The second classification unit is used to classify the water body after fusing the water body probability images corresponding to the multiple feature maps, and determine the water body segmentation image corresponding to the remote sensing image.

[0142] As a feasible implementation method, the device further includes: a model determination module; wherein,

[0143] The model determination module is used to determine a water body extraction model, and the water body extraction model includes a first model, a second model and a third model; wherein the first model is used to extract the water body area in the remote sensing image based on the spectral feature map and determine the water body image; the second model is used to encode the water body image to extract water body characteristics and determine the water body characteristic map; the third model is used to decode the water body characteristic map to segment the water body area in the remote sensing image and determine the water body segmentation image corresponding to the remote sensing image.

[0144] As a feasible implementation method, the device further includes: a labeling image determination module, a discrimination module and a training module; wherein,

[0145] The annotated image determination module is used to determine an annotated image corresponding to the remote sensing image, wherein the annotated image is used to distinguish an area corresponding to the lake from other areas in the remote sensing image;

[0146] The discrimination module is used to substitute the water body segmentation image and the annotated image into a discrimination model for discrimination, and determine a discrimination result of the water body segmentation image, wherein the discrimination result indicates whether the water body segmentation image is an annotated image or an image generated by the water body extraction model;

[0147] The model training module is used to train the water body extraction model and the discrimination model based on the judgment result, the water body segmentation image and the annotated image.

[0148] As a feasible implementation, the second model includes M downsampling blocks connected in series, and the water body feature map includes feature maps output by each of the M downsampling blocks; for the i-th downsampling block among the M downsampling blocks, it is used to retain information of lakes with smaller areas. When i=1, the input of the i-th downsampling block is the water body image, and when i is greater than 1, the input of the i-th downsampling block is the output of the i-1-th downsampling block;

[0149] The third model includes a decoding unit and a water body classification unit, and the water body segmentation image is the output of the water body classification unit; the input of the decoding unit is the output of each of the M downsampling blocks, which is used to decode the water body feature map, and the input of the water body classification unit is the output of the decoding unit, or the output of the decoding unit and the output of each of at least one of the M downsampling blocks.

[0150] Figure 6It is a structural diagram of an electronic device provided by an embodiment of the present invention. At the hardware level, the electronic device includes a processor 601 and a memory 602 storing execution instructions, and optionally also includes an internal bus 603 and a network interface 604. Among them, the memory 602 may include a memory 6021, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory 6022 (non-volatile memory), such as at least one disk storage, etc.; the processor 601, the network interface 604 and the memory 602 can be interconnected through an internal bus 603, and the internal bus 603 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc.; the internal bus 603 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus. Of course, the electronic device may also include hardware required for other services. When the processor 601 executes the execution instruction stored in the memory 602, the processor 601 executes the method in any embodiment of the present invention, and is at least used to execute the following steps: Figure 1 or Figure 4 The method shown.

[0151] In a possible implementation, the processor reads the corresponding execution instructions from the non-volatile memory into the memory and then runs them, and can also obtain the corresponding execution instructions from other devices to form a water body region extraction device at the logical level. The processor executes the execution instructions stored in the memory to implement a water body region extraction method provided in any embodiment of the present invention through the execution of the execution instructions.

[0152] The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present invention can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0153] The embodiment of the present invention further provides a computer-readable storage medium, including an execution instruction. When a processor of an electronic device executes the execution instruction, the processor executes the method provided in any embodiment of the present invention. The electronic device may be specifically as follows: Figure 6 The electronic device shown; the execution instruction is a computer program corresponding to a water body area extraction device.

[0154] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods or computer program products. Therefore, the present invention may be implemented in the form of a complete hardware embodiment, a complete software embodiment, or a combination of software and hardware.

[0155] The various embodiments of the present invention are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0156] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0157] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for extracting a water body region, characterized in that: include: Determine a remote sensing image of a glacial lake region, wherein the glacial lake region includes a glacier and lakes surrounding the glacier, and the remote sensing image is an image obtained by long-distance detection of the glacial lake region; Extracting spectral features from the remote sensing image to determine a spectral feature image of the glacial lake area, wherein the spectral feature image is used to highlight the area of ​​the lake corresponding to the remote sensing image; Extracting a water body region in the remote sensing image based on the spectral feature image and determining a water body image, wherein the water body image indicates a region in the remote sensing image that may represent the lake; Encoding the water body image to extract water body features and determine a water body feature map, wherein the water body feature map indicates information of the lake included in the water body image; Decoding the water body feature map to segment the water body area in the remote sensing image, and determining a water body segmentation image corresponding to the remote sensing image, wherein the water body segmentation image is used to distinguish the area corresponding to the lake in the remote sensing image from other areas; The step of extracting the water body region in the remote sensing image based on the spectral feature map and determining the water body image comprises: Based on the remote sensing image, determining water body attention weights of different areas in the spectral feature image; Determining water body attention weights of different areas in the remote sensing image based on the water body attention weights of different areas in the spectral feature image and the spectral feature image; Extracting a water body region in the remote sensing image based on water body attention weights of different regions in the remote sensing image to determine a water body image; The water body feature map includes a plurality of feature maps of different sizes, and the plurality of feature maps are all used to represent information of the lake included in the water body image; The decoding of the water body feature map to segment the water body area in the remote sensing image and determining the water body segmentation image corresponding to the remote sensing image includes: Decoding the multiple feature maps in sequence, determining a water body probability image obtained by the last decoding and having the same size as the remote sensing image, wherein the water body probability image indicates the water body probabilities of different regions in the remote sensing image; The water body probability image obtained by the last decoding is subjected to water body classification to determine a water body segmentation image corresponding to the remote sensing image.

2. The method according to claim 1, characterized in that: The step of determining water attention weights of different regions in the spectral feature image based on the remote sensing image includes: Determine a key graph corresponding to the remote sensing image based on a key weight matrix; Determining a query graph corresponding to the remote sensing image based on a query weight matrix; Attention calculation is performed based on the key graph and the query graph to determine water body attention weights in different areas of the spectral feature image.

3. The method according to claim 1, characterized in that The decoding of the water body feature map to segment the water body area in the remote sensing image and determining the water body segmentation image corresponding to the remote sensing image includes: Decoding the plurality of feature maps respectively, determining water body probability images corresponding to the plurality of feature maps and having the same size as the remote sensing image, the water body probability images indicating the water body probabilities of different regions in the remote sensing image; The water body probability images corresponding to the multiple feature maps are fused to perform water body classification, and a water body segmentation image corresponding to the remote sensing image is determined.

4. The method according to claim 1, characterized in that: The method further comprises: Determine a water body extraction model, the water body extraction model includes a first model, a second model and a third model; wherein the first model is used to extract the water body area in the remote sensing image based on the spectral feature map, and determine the water body image; the second model is used to encode the water body image to extract water body characteristics and determine the water body feature map; the third model is used to decode the water body feature map to segment the water body area in the remote sensing image, and determine the water body segmentation image corresponding to the remote sensing image.

5. The method according to claim 4, characterized in that The method further comprises: Determine a labeled image corresponding to the remote sensing image, wherein the labeled image is used to distinguish an area corresponding to the lake from other areas in the remote sensing image; Substituting the water body segmentation image and the annotated image into a discrimination model for judgment, and determining a judgment result of the water body segmentation image, wherein the judgment result indicates whether the water body segmentation image is an annotated image or an image generated by the water body extraction model; The water body extraction model and the discrimination model are trained based on the judgment result, the water body segmentation image and the annotated image.

6. The method according to claim 5, characterized in that The second model includes M downsampling blocks connected in series, and the water body feature map includes feature maps output by each of the M downsampling blocks; for the i-th downsampling block among the M downsampling blocks, it is used to retain information of lakes with smaller areas. When i=1, the input of the i-th downsampling block is the water body image, and when i is greater than 1, the input of the i-th downsampling block is the output of the i-1-th downsampling block; The third model includes a decoding unit and a water body classification unit, and the water body segmentation image is the output of the water body classification unit; the input of the decoding unit is the output of each of the M downsampling blocks, which is used to decode the water body feature map, and the input of the water body classification unit is the output of the decoding unit, or the output of the decoding unit and the output of each of at least one of the M downsampling blocks.

7. A water area extraction device, characterized in that: include: A remote sensing image determination module is used to determine a remote sensing image of a glacial lake area, wherein the glacial lake area includes a glacier and lakes surrounding the glacier, and the remote sensing image is an image obtained by long-distance detection of the glacial lake area; A spectral image determination module, used for extracting spectral features from the remote sensing image to determine a spectral feature image of the glacial lake area, wherein the spectral feature image is used to highlight the area corresponding to the lake in the remote sensing image; A water body extraction module, for extracting a water body region in the remote sensing image based on the spectral feature image, and determining a water body image, wherein the water body image indicates an area in the remote sensing image that may represent the lake; An encoding module, used for encoding the water body image to extract water body features and determine a water body feature map, wherein the water body feature map indicates information of the lake included in the water body image; A decoding module, used for decoding the water body feature map to segment the water body area in the remote sensing image, and determining a water body segmentation image corresponding to the remote sensing image, wherein the water body segmentation image is used to distinguish the area corresponding to the lake in the remote sensing image from other areas; The water body extraction module is used to determine the water body attention weights of different areas in the spectral feature image based on the remote sensing image; Determining water body attention weights of different areas in the remote sensing image based on the water body attention weights of different areas in the spectral feature image and the spectral feature image; Extracting a water body region in the remote sensing image based on water body attention weights of different regions in the remote sensing image to determine a water body image; The water body feature map includes a plurality of feature maps of different sizes, and the plurality of feature maps are all used to represent information of the lake included in the water body image; The decoding module is used to decode the multiple feature maps in sequence, determine a water body probability image obtained by the last decoding that is consistent with the size of the remote sensing image, and the water body probability image indicates the water body probability of different areas in the remote sensing image; perform water body classification on the water body probability image obtained by the last decoding, and determine a water body segmentation image corresponding to the remote sensing image.

Images