A glacial lake extraction method integrating spectral features and multi-scale spatial features

Abstract

The present invention provides an ice lake extraction method that integrates spectral features and multi-scale spatial features, which mainly solves the technical problem that the existing ice lake extraction method requires post-processing and auxiliary data, and cannot realize the automatic extraction of ice lakes using only remote sensing images. The present invention is based on a convolutional neural network, and uses a VGG16 network pre-trained on ImageNet, then takes out the shallow, middle and deep convolution features, and performs an upsampling operation on the image features of each layer, and connects the upsampling result with the features of the shallower layer, repeats the above process, and the last convolution layer obtained contains ice lake features of different scales, and performs a softmax operation on the result of the shallowest layer to obtain the probability that each pixel category belongs to an ice lake pixel, and then obtains an ice lake model, and finally obtains a network model for ice lake extraction by training the model, and completes the automatic extraction of ice lakes by inputting remote sensing images.

Description

Technical Field

[0001] The present invention relates to the fields of remote sensing image processing and computer vision, and in particular to a glacial lake extraction method integrating spectral features and multi-scale spatial features. Background Art

[0002] Glacial lakes are a sensitive factor related to climate change. Monitoring the status of glacial lakes can provide an important basis for the development of global climate change, have guiding value for the prediction and early warning of glacial lake outburst disasters, and provide assistance for ecological planning in plateau areas. Since glacial lakes develop around glaciers on the plateau where few people go, remote sensing methods are often used to monitor glacial lakes. Remote sensing images contain rich ground feature information, but many ground feature information (such as clouds, mountain shadows, melting glaciers, etc.) show spectral characteristics or spatial characteristics similar to glacial lakes. Therefore, when using traditional methods to extract glacial lakes, pre-processing or post-processing operations are often required to remove the influence of interference factors.

[0003] At present, glacial lake extraction methods can be roughly divided into the following categories:

[0004] ① Method based on manual assistance: that is, manually vectorize the boundary of the glacial lake to obtain the information of the glacial lake. This method has high accuracy, but requires the labeler to have certain professional knowledge, is time-consuming and labor-intensive, and is only suitable for small-scale glacial lake extraction.

[0005] ②Pixel-based method: Since the reflectivity of ice lake pixels is higher in the blue light band and lower in the near-infrared band, each pixel in the image is used as a target to determine whether the pixel conforms to the spectral distribution law of ice lake pixels, and a threshold is set to distinguish ice lake pixels from background pixels. Representative methods include threshold segmentation method, random forest, water body index, etc. The problem with this type of method is that it only uses the spectral characteristics of the ice lake and ignores the spatial characteristics of the image itself, resulting in incomplete extracted ice lake boundaries and being easily affected by noise. Certain auxiliary data (such as elevation data DEM) and post-processing are required to optimize the extraction results.

[0006] ③ Region-based methods: Since the surface of the glacial lake is homogeneous, some methods consider capturing the homogeneous area of ​​the glacial lake, and representative methods include object-oriented methods, improved CV models, etc. However, this type of method only uses the spatial characteristics of the glacial lake target, and it is difficult to directly remove the interference of objects with the same surface homogeneity (such as glaciers). At the same time, the segmentation scale also requires multiple experimental settings, and still requires certain auxiliary data (such as elevation data DEM) and post-processing work to optimize the extraction results.

[0007] ④ Feature-based methods: Since ice lakes not only have spectral and spatial features, but also contain many advanced features such as texture and roughness, this type of method uses a convolutional neural network to mine the advanced features of ice lakes in images and use the features to accurately extract ice lakes. The representative method is the U-Net network, but this type of method is still in the development stage and lacks a deep learning network designed specifically for ice lakes, and requires the production of a large number of training samples.

[0008] In summary, most traditional glacial lake extraction methods require auxiliary data or additional processing to remove interference from other factors, such as mountain shadows, clouds, melting glaciers, etc. It is currently difficult to achieve automatic extraction of glacial lakes using only remote sensing images. Summary of the invention

[0009] The present invention discloses a glacial lake extraction method integrating spectral features and multi-scale spatial features, which mainly solves the technical problem that the existing glacial lake extraction method requires post-processing and auxiliary data, and cannot realize automatic extraction of glacial lakes using only remote sensing images.

[0010] The invention discloses a method for extracting an ice lake by integrating spectral features and multi-scale spatial features, which is special in that it comprises the following steps:

[0011] Step 1: Input the glacial lake remote sensing image into the pre-trained VGG16 network to obtain convolutional layers of different scales; and simultaneously obtain the NDWI spectral feature map of the glacial lake remote sensing image; the convolutional layer includes a deep convolutional layer, a middle convolutional layer and a shallow convolutional layer, and each convolutional layer includes spatial features of glacial lakes of different scales;

[0012] Step 2: extract the deep convolution layer in step 1, perform convolution calculation and upsampling twice on the convolution features of the layer, and obtain a first feature layer with the same convolution feature size as the middle convolution layer in step 1;

[0013] Step 3: extract the middle convolutional layer in step 1, link the convolutional features of the first feature layer obtained in step 2 to the middle convolutional layer, perform convolution calculation and upsampling twice on the convolutional features of the linked middle convolutional layer, and obtain a second feature layer with the same convolutional feature size as the shallow convolutional layer in step 1;

[0014] Step 4, extracting the shallow convolution layer in step 1, linking the convolution features of the second feature layer obtained in step 3 and the NDWI spectral feature map of the water body index obtained in step 1 to the shallow convolution layer, performing two convolution processes on the convolution features of the shallow convolution layer added with the link, inputting the processing results into the softmax layer, and outputting the ice lake area map; the ice lake area map is used to determine the exact position of the ice lake boundary;

[0015] Step 5, combining the spatial characteristics of the glacial lake at different scales obtained in step 1 and the NDWI spectral characteristic map, the first characteristic layer obtained in step 2, the second characteristic layer obtained in step 3, and the glacial lake regional map obtained in step 4 to form a glacial lake model;

[0016] Step 6, using the Landsat-8 dataset to train the glacial lake model constructed in step 5, thereby obtaining an glacial lake extraction network model;

[0017] Step 7, the glacial lake remote sensing image in step 1 is input again into the glacial lake extraction network model obtained in step 6, so as to extract accurate glacial lake information.

[0018] Furthermore, in step 1, the VGG16 network is pre-trained using ImageNet; the pixel size of the image received by the pre-trained VGG16 network is 448×448.

[0019] Furthermore, the convolutional features of the deep convolutional layer include target category information, and the convolutional feature size is 28×28;

[0020] The convolutional features of the middle convolutional layer include the location information of the target, and the convolutional feature size is 112×112;

[0021] The convolution feature of the shallow convolution layer includes the boundary information of the target, and its convolution feature size is 448×448.

[0022] Furthermore, the calculation formula of the water body index NDWI spectral characteristic diagram is:

[0023]

[0024] Among them, ρ Green Represents the apparent reflectance of the top layer of the atmosphere in the green light band, 0<ρ Green <1;ρ NIR Represents the apparent reflectance of the top layer of the atmosphere in the near-infrared band, 0<ρ NIR <1.

[0025] Furthermore, in step 2, the convolutional feature size of the first feature layer is 112×112.

[0026] Furthermore, in step 3, the convolutional feature size of the second feature layer is 448×448.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. The present invention provides an ice lake extraction method that integrates spectral features and multi-scale spatial features. A convolutional neural network is used to extract the spatial features of ice lakes at different scales. At the same time, combined with the NDWI spectral features of the ice lake, the ice lake area can be obtained more accurately without unnecessary pre-processing or post-processing, which greatly improves the efficiency of ice lake extraction, thereby realizing the automatic extraction of ice lakes.

[0029] 2. The present invention provides an ice lake extraction method that integrates spectral features and multi-scale spatial features. Combined with the spatial features of the ice lake, it can not only better restore the boundary information of the ice lake, but also adopts the normalized water body index and introduces the spectral features of the ice lake, which can further enhance the noise resistance of the model and achieve better extraction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a schematic diagram of the ice lake extraction process of an embodiment of an ice lake extraction method that integrates spectral features and multi-scale spatial features of the present invention. DETAILED DESCRIPTION

[0031] The present invention proposes a method for extracting ice lakes by integrating multi-scale information. The method aims to utilize the spatial features, spectral features and potential high-level visual features (such as texture, roughness, etc.) of ice lakes in remote sensing images to accurately extract the ice lake area in the image; at the same time, considering that ice lakes are small targets, scale information and boundary information are very important. Therefore, based on a convolutional neural network, the present invention designs a method for extracting ice lakes by integrating spectral features and multi-scale spatial features, so as to realize the automatic extraction of ice lakes in remote sensing images.

[0032] The following is a detailed description of an ice lake extraction method that integrates spectral features and multi-scale spatial features of the present invention in conjunction with the accompanying drawings and specific embodiments.

[0033] like Figure 1 As shown, a method for extracting ice lakes by integrating spectral features and multi-scale spatial features includes the following steps:

[0034] Step 1: Input the glacial lake remote sensing image into the pre-trained VGG16 network to obtain convolutional layers of different scales; at the same time, obtain the water index NDWI spectral feature map of the glacial lake remote sensing image.

[0035] Specifically, the ice lake remote sensing image is input into the VGG16 network pre-trained on ImageNet. This embodiment chooses to pre-train the network on ImageNet. The network pre-trained by ImageNet has good feature extraction capabilities, which can improve the accuracy of subsequent ice lake feature extraction. In the VGG16 network, five convolutional layers of ice lake features of different scales are arranged from shallow to deep, namely the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer. This embodiment only uses the first convolutional layer, the third convolutional layer and the fifth convolutional layer. The first convolutional layer belongs to the shallow convolutional layer. The convolutional features of this layer include the boundary information of the target, and its convolutional feature size is 448×448. The third convolutional layer belongs to the middle convolutional layer. The convolutional features of this layer include the location information of the target, and its convolutional feature size is 112×112. The fifth convolutional layer belongs to the deep convolutional layer. The convolutional features of this layer include target category information, and its convolutional feature size is 28×28. The selection of shallow convolutional layers, middle convolutional layers, and deep convolutional layers can not only improve the efficiency of extraction, but the combination of the three convolutional layers also contains complete information such as the location and boundary of the ice lake, further improving the accuracy of ice lake extraction.

[0036] In addition, the normalized water index NDWI spectral characteristic map of the glacial lake remote sensing image is obtained by calculation in the network. The introduction of the spectral characteristics of the glacial lake can further enhance the noise resistance of the model and improve the extraction effect of the glacial lake. The specific calculation formula of the normalized water index NDWI spectral characteristic map is:

[0037]

[0038] Among them, ρ Green Represents the apparent reflectance of the top layer of the atmosphere in the green light band, 0<ρ Green <1;ρ NIR Represents the apparent reflectance of the top layer of the atmosphere in the near-infrared band, 0<ρ NIR <1.

[0039] In this embodiment, the glacial lake remote sensing image is an image collected from Landsat-8 images. The pixel size of the image received by the pre-trained VGG16 network is 448×448. If the input image size is not 448×448, the pre-trained VGG16 network will automatically adjust it to this size before receiving and processing it.

[0040] Step 2: extract the convolution features of the deep convolution layer in step 1, perform convolution calculation and upsampling twice on the convolution features of this layer, and obtain a first feature layer with the same size as the convolution features of the middle convolution layer in step 1.

[0041] Specifically, the deep convolution layer obtained in step 1 is extracted. The convolution features of this convolution layer belong to the deep features, which mainly contain target category information, such as category information of different regions such as glaciers, ice lakes, mountains, and vegetation. The convolution features of the deep convolution layer can be used to determine whether the target category is an ice lake. After convolution calculation and two upsampling processes on the convolution features of the deep convolution layer, the first feature layer with the same convolution feature size as the middle convolution layer in step 1 is obtained. The convolution features of this feature layer include information on whether the target category is an ice lake. The convolution feature size of the first feature layer is 112×112.

[0042] Step 3: extract the convolutional features of the middle convolutional layer in step 1, link the convolutional features of the first feature layer obtained in step 2 to the middle convolutional layer, perform convolution calculation and upsampling twice on the convolutional features of the linked middle convolutional layer, and obtain a second feature layer with the same size as the convolutional features of the shallow convolutional layer in step 1.

[0043] Specifically, the middle convolution layer obtained in step 1 is extracted. The convolution feature of this convolution layer belongs to the middle feature, which contains certain pixel category information and location information, that is, whether the pixel is a pixel category of an ice lake, and the approximate location of the ice lake area. Then the convolution feature of the first feature layer obtained in step 2 is linked to the middle convolution layer, and the convolution calculation and two upsampling processes are performed on the convolution feature of the linked middle convolution layer to obtain the second feature layer with the same size as the convolution feature of the shallow convolution layer in step 1. The convolution feature of the second feature layer mainly contains the approximate location information of the ice lake area, and the convolution feature size of the second feature layer is 448×448.

[0044] Step 4: extract the convolution features of the shallow convolution layer in step 1, link the convolution features of the second feature layer obtained in step 3 and the NDWI spectral feature map of the water body index obtained in step 1 to the shallow convolution layer, perform two convolution processes on the convolution features of the shallow convolution layer with the link, input the processing results into the softmax layer, and output the ice lake area map.

[0045] Specifically, the shallow convolution layer in step 1 is extracted. The convolution features of this convolution layer belong to the shallow features, which mainly contain the edge details of the ice lake and can accurately locate the boundary of the ice lake. The convolution features of the second feature layer obtained in step 3 and the NDWI spectral feature map of the water body index obtained in step 1 are linked to the shallow convolution layer. Then, the convolution features of the shallow convolution layer added with the link are convolved twice, and the processing results are input into the softmax layer for classification processing to obtain the ice lake area map, which is used to determine the exact location of the ice lake boundary.

[0046] Step 5, combining the spatial characteristics of the glacial lake at different scales obtained in step 1 and the NDWI spectral characteristic map, the first characteristic layer obtained in step 2, the second characteristic layer obtained in step 3, and the glacial lake regional map obtained in step 4 to form a glacial lake model;

[0047] Step 6: Use the Landsat-8 dataset to train the glacial lake model constructed in step 5, and then obtain the glacial lake extraction network model.

[0048] The Landsat-8 dataset is used to train the ice lake model. The loss function of the training process is determined by cross entropy. Here, cross entropy refers to the loss between the extracted ice lake area map and the real ice lake area map. The smaller the cross entropy value, the smaller the loss between the extracted ice lake area map and the real ice lake area map, which means the better the effect of the training model. In addition, the accuracy of the training model can be improved by increasing the number of training times or increasing the dataset.

[0049] Step 7, input the ice lake remote sensing image in step 1 into the ice lake extraction network model trained in step 6, so as to realize the automatic extraction of ice lake information, and the extracted information is accurate and reliable without unnecessary pre-processing or post-processing. Other ice lake remote sensing images can also be used as input to extract ice lake information with higher accuracy. In order to obtain the accuracy of ice lake extraction, the F1 score can be used for evaluation. After actual evaluation, the ice lake extraction method used in this embodiment has an extraction accuracy of more than 90%, which is a significant improvement over the existing ice lake extraction accuracy.

[0050] In addition, this extraction method can be further used in large-area glacial lake mapping to improve the accuracy and completeness of glacial lake extraction.

[0051] The effect of the present invention can be further illustrated by the following experiments.

[0052] 1. Experimental conditions

[0053] The present invention is that the central processing unit is The model is implemented using Python programming on an i5-9400F 2.9GHz CPU, GTX 1660T 6G GPU, 16G memory, and WINDOWS 10 operating system. The model also involves a deep learning framework, and the deep learning framework used in this experiment is tensorflow 1.14. The data used in the experiment are all collected from Landsat-8 images, and the size of all images is 256×256×7 (i.e., the length, width, and number of bands of the image).

[0054] 2. Experimental content

[0055] The present invention uses the F1 score to evaluate the accuracy of the final ice lake extraction. The specific calculation method of the F1 score is as follows:

[0056] Accuracy = correctly extracted glacial lake pixels / all extracted pixels in the extraction result

[0057] Recall rate = correctly extracted ice lake pixels / all ice lake pixels contained in the true result

[0058] F1 score = 2 × precision × recall / (precision + recall)

[0059] The size of the input image is 256 × 256 × 7. Regarding the training parameters, the batch size is set to 8, the epoch is set to 100, the dropout is set to 0.5 to prevent overfitting, the optimizer selects AdamOptimizer, and the learning rate is set to 0.0005.

[0060] To verify the effectiveness of the present invention, the extraction results of the present invention were compared with those of other glacial lake extraction methods, including a global-local iterative segmentation algorithm (Li, J.; Sheng, Y. An automated scheme for glacial lake dynamics mapping using Landsat imagery and Digital Elevation Models: A Case Study in the Himalayas. Int. J. Remote Sens. 2012, 33, 5194–5213.), a random forest segmentation algorithm (Wangchuk S.; Bolch T. Mapping of glacial lakes using Sentinel-1 and Sentinel-2 data and a random forest classifier: Strengths and challenges. Science of Remote Sensing, 2020, 2.), and a glacial lake extraction method using a U-net model (Qayyum N.; Ghuffar S.; Ahmad HM; Yousaf A.; Shahid I. Glacial Lakes Mapping Using Multi Satellite PlanetScope Imagery and Deep Learning. ISPRS International Journal of Geo-Information, 2020, 9.). A global-local iterative segmentation algorithm, that is, first use the water body index NDWI to perform a rough extraction of the glacial lake, then establish a buffer for the rough extraction results, and use the bimodal threshold segmentation algorithm in the buffer to further extract the glacial lake information. The random forest algorithm requires manual selection of glacial lake pixels and non-glacial lake pixels, and try to ensure that the number of the two is equal, and then construct the spectral vector and category label of each pixel as training data. The trained model can further identify the glacial lake pixels in the image. The U-net model is a deep learning model that mainly includes two processes: encoding (i.e., downsampling) and decoding (i.e., upsampling), and connects feature maps of different scales through jump connections. At present, there is work that uses U-net for the extraction of glacial lakes. The present invention takes Landsat-8 image data as an example to obtain images of the entire eastern Himalayas and crop the glacial lakes therein. The present embodiment involves 22 images, from which 2000 images of 256×256 size are randomly cropped for model training and verification. The accuracy comparison results of different models are shown in Table 1:

[0061] Table 1 Comparison of the accuracy of typical model glacial lake extraction

[0062]

[0063] As can be seen from Table 1, compared with the traditional ice lake extraction model and the deep learning model U-net, the model proposed in the present invention introduces a fusion mechanism of spectral features and multi-scale spatial features, integrating ice lake features and spectral features at different scales, so it can more accurately extract the edge information of the ice lake and obtain higher precision. This shows that the method proposed in the present invention is more suitable for ice lake extraction research at a large scale.

[0064] Since the convolutional neural network can better capture the spatial characteristics of the target, in order to highlight the glacial lake target, the present invention also uses the normalized water index NDWI and introduces the spectral characteristics of the glacial lake to enhance the noise resistance of the model and the accuracy of glacial lake extraction.

[0065] In general, the present invention is a glacial lake extraction method that integrates spectral features and multi-scale spatial features. It can accurately extract the edge features of glacial lakes and is suitable for the rapid and automatic extraction of glacial lake information in large-area remote sensing images.

Claims

1. A method for extracting ice lakes by integrating spectral features and multi-scale spatial features, characterized in that: The following steps are involved: Step 1: Input the glacial lake remote sensing image into the pre-trained VGG16 network to obtain convolutional layers of different scales; and at the same time, obtain the NDWI spectral feature map of the glacial lake remote sensing image; the convolutional layers of different scales include deep convolutional layers, middle convolutional layers and shallow convolutional layers, and each convolutional layer includes spatial features of glacial lakes of different scales; Step 2: extract the deep convolution layer in step 1, perform convolution calculation and upsampling twice on the convolution features of the layer, and obtain a first feature layer with the same convolution feature size as the middle convolution layer in step 1; Step 3: extract the middle convolutional layer in step 1, link the convolutional features of the first feature layer obtained in step 2 to the middle convolutional layer, perform convolution calculation and upsampling twice on the convolutional features of the linked middle convolutional layer, and obtain a second feature layer with the same convolutional feature size as the shallow convolutional layer in step 1; Step 4, extracting the shallow convolution layer in step 1, linking the convolution features of the second feature layer obtained in step 3 and the water body index NDWI spectral feature map obtained in step 1 to the shallow convolution layer, performing two convolution processes on the convolution features of the linked shallow convolution layer, inputting the processing results into the softmax layer, and outputting the ice lake area map; the ice lake area map is used to determine the exact position of the ice lake boundary; Step 5, combining the spatial characteristics of the glacial lake at different scales obtained in step 1 and the NDWI spectral characteristic map, the first characteristic layer obtained in step 2, the second characteristic layer obtained in step 3, and the glacial lake regional map obtained in step 4 to form a glacial lake model; Step 6, using the Landsat-8 dataset to train the glacial lake model constructed in step 5, thereby obtaining an glacial lake extraction network model; In step 7, any glacial lake remote sensing image is input into the glacial lake extraction network model obtained in step 6 to extract accurate glacial lake information.

2. The ice lake extraction method integrating spectral features and multi-scale spatial features according to claim 1, characterized in that: In step 1, the pre-trained VGG16 network refers to a VGG16 network pre-trained using ImageNet; the pixel size of the image received by the pre-trained VGG16 network is 448×448.

3. The ice lake extraction method integrating spectral features and multi-scale spatial features according to claim 2 is characterized in that: The convolutional features of the deep convolutional layer include target category information, and the convolutional feature size is 28×28; The convolutional features of the middle convolutional layer include the location information of the target, and the convolutional feature size is 112×112; The convolution feature of the shallow convolution layer includes the boundary information of the target, and its convolution feature size is 448×448.

4. The ice lake extraction method integrating spectral features and multi-scale spatial features according to claim 3 is characterized in that: The calculation formula of the water body index NDWI spectral characteristic diagram is: Among them, ρ Green Represents the apparent reflectance of the top layer of the atmosphere in the green light band, 0<ρ Green <1;ρ NIR Represents the apparent reflectance of the top layer of the atmosphere in the near-infrared band, 0<ρ NIR <1.

5. The ice lake extraction method integrating spectral features and multi-scale spatial features according to claim 4 is characterized in that: In step 2, the convolutional feature size of the first feature layer is 112×112.

6. The ice lake extraction method integrating spectral features and multi-scale spatial features according to claim 5, characterized in that: In step 3, the convolutional feature size of the second feature layer is 448×448.

Images