Method for extracting shallow lake algal blooms and aquatic plants based on deep learning model

Abstract

The present application relates to a kind of shallow lake algal bloom and aquatic plant extraction method based on deep learning model, comprising: obtaining Landsat and Sentinel-2 image covering shallow lake;Using optical image to make the water bloom and aquatic plant sample of training and testing deep learning model;Spectrum feature is constructed, and the importance of the top five features is selected using random forest model to participate in the construction of deep learning model;Deep learning model is constructed, trained and tested, and the trained deep learning model is used to classify the water bloom and aquatic plant of shallow lake, and the specific classification is four categories of water body, water bloom, submerged plant and floating leaf emergent plant.The present application can automatically and accurately obtain the spatial distribution of shallow lake water bloom and aquatic plant, and provide important technical support for dynamic monitoring of lake water bloom and aquatic plant.

Description

TECHNICAL FIELD

[0001] The present application relates to a shallow lake water bloom and aquatic plant extraction method based on a deep learning model, belonging to the technical field of remote sensing application. BACKGROUND

[0002] Aquatic plants are plants that can grow in water, generally divided into floating leaf emergent plants and submerged plants. They play a crucial role in lake ecosystems, not only helping to improve lake transparency, but also effectively inhibiting the overgrowth of harmful algae, thereby purifying water quality. In addition, aquatic plants also provide food sources and habitats for aquatic animals, which is of great significance to maintaining lake biodiversity and ecological balance. However, due to the combined effects of climate change and human activities, the eutrophication problem of shallow lakes is becoming increasingly serious, which leads to the accelerated extinction of aquatic plants. Given the decisive influence of aquatic plants on lake ecosystems, accurately obtaining the spatio-temporal distribution of water bloom and aquatic plants plays a very important role in restoring lake ecosystems.

[0003] Currently, the monitoring methods of water bloom and aquatic plants mainly include field investigation method and remote sensing monitoring method. The field investigation method has high accuracy, but it is labor-intensive and has poor timeliness; while the remote sensing technology has the advantages of short revisit time and can monitor spatial distribution, and it has gradually become an effective means for monitoring water bloom and aquatic plants.

[0004] The methods of monitoring water bloom and aquatic plants by remote sensing mainly include index method and supervised classification method. The index method is widely used in remote sensing monitoring of water bloom and aquatic plants due to its simple principle and easy implementation. Supervised classification method has high accuracy and automation degree, but it has certain requirements for data and quality of training samples. However, the existing methods cannot well meet the accuracy requirements in monitoring water bloom and aquatic plants. On the one hand, due to the influence of lake water environment on aquatic plants, the spectral characteristics of aquatic plants are small, which makes it difficult to distinguish different types of aquatic plants. On the other hand, the spectral characteristics of water bloom and aquatic plants are very similar, which greatly interferes with the distinction between the two. In this case, using new intelligent algorithms for distinction has become a research hotspot.

[0005] In recent years, the emergence of deep learning methods has brought new ideas and methods for the extraction of water bloom and aquatic plants. However, when facing inland shallow lakes, the situation becomes more complex. These lakes are often severely eutrophicated, and not only grow a variety of aquatic plants, but also often accompanied by the appearance of water bloom. This coexistence of water bloom and aquatic plants brings higher challenges to the extraction accuracy of deep learning models. The accuracy of most algorithms in extracting water bloom and aquatic plants is not ideal. SUMMARY

[0006] The technical problem solved by the present application is that the accuracy of most current algorithms in extracting water bloom and aquatic plants is not ideal, and a shallow lake water bloom and aquatic plant extraction method based on a deep learning model is proposed, which uses Landsat images as data sources, designs a deep learning model for water bloom and aquatic plant extraction, and applies the network to shallow lakes with serious eutrophication to achieve accurate classification of water bloom and aquatic plants.

[0007] To solve the above technical problems, the technical solution of the present application is a shallow lake water bloom and aquatic plant extraction method based on a deep learning model, comprising the following steps:

[0008] Step 1: Obtain Landsat reflectance images and Sentinel-2 images of the study area within a specified time range, and the imaging time of the Landsat reflectance images and the corresponding Sentinel-2 images should not differ by more than one day.

[0009] Step 2: Visually interpret and sample the Landsat reflectance images with the aid of Sentinel-2 high-resolution images to accurately produce training samples and test samples, and the training samples and test samples correspond to the categories in the visual interpretation results.

[0010] Step 3: Construct feature indexes for extracting water bloom and aquatic plants, and input the constructed feature indexes and each band of the Landsat reflectance images into the random forest model to obtain the importance of the feature indexes and sort them, select the top five feature indexes, and construct feature images of the Landsat reflectance images based on the five feature indexes for the training samples.

[0011] Step 4: Construct a deep learning model for water bloom and aquatic plant extraction, including an encoder, a pyramid pooling module ASPP, and a decoder.

[0012] Step 5: Divide the feature images of the training samples and the corresponding visual interpretation results (labels) into training set and validation set in proportion, input them into the constructed deep learning model for training, then iterate multiple times, and save the training model with the highest accuracy to obtain suitable training hyperparameters.

[0013] Step 6: Test and evaluate the trained deep learning model based on the test samples.

[0014] Step 7: Input the feature images of the Landsat reflectance images of each period of the study area into the trained deep learning model for processing to obtain the spatial distribution map of water bloom and aquatic plants in each period.

[0015] Further, in the third step, the top five feature indexes are normalized difference water index MNDWI, green band G, short wave infrared band SWIR, floating algae index FAI and enhanced vegetation index EVI, wherein,

[0016]

[0017] In the formula, NIR is, R is a red band, NIR is a near-infrared band, B is a blue band, and NIR is the wavelength of the near-infrared band, and R is the wavelength of the red band.

[0018] Further, in the deep learning model, the encoder comprises a convolutional layer and a ConvNeXt network with four layers, the feature image input into the encoder is subjected to convolution and normalization to obtain a feature map and sent to the ConvNeXt network, each layer of the ConvNeXt network sequentially performs convolution and pooling operation on the feature map, and the feature image is input into an ASPP module for processing, the ASPP module obtains a multi-scale feature map through multi-scale strategy on the input feature map through different dilation rates of the hole convolution, then the multi-scale feature map is fused to obtain a fused feature map and sent to a decoder; the decoder adds a Scse attention module, the decoder iteratively processes the fused feature map for three times, each iteration includes bilinear interpolation upsampling and fusion with the feature map of the corresponding layer in the encoder, and enhancement processing is performed by using the Scse attention module, and then convolution operation is performed again, after iteration is completed, bilinear interpolation upsampling is performed again to the size of the original input image, and convolution processing is performed, to generate a single-band output image.

[0019] The present application mainly faces eutrophic shallow lakes, can automatically and accurately obtain the spatio-temporal distribution of water bloom and aquatic plants, provides important technical support for dynamic monitoring of lake water bloom and aquatic plants, and further provides important reference for lake eutrophication treatment and water environment restoration. BRIEF DESCRIPTION OF DRAWINGS

[0020] The present application will be further described below in combination with the drawings.

[0021] Figure 1 is the geographical position of the lake and the false color Landsat image.

[0022] Figure 2 is the flowchart of the present application.

[0023] Figure 3 is the schematic diagram of data enhancement.

[0024] Figure 4 is the network structure of the deep learning model AAENet for water bloom and aquatic plant extraction.

[0025] Figure 5 is the network structure of the ASPP module.

[0026] Figure 6 is the training loss and accuracy curve during model training. (a) Training loss. (b) Accuracy curve.

[0027] Figure 7 is the Landsat false color image and the corresponding classification results of the three test lakes. (a) Landsat false color image. (b) AAENet, (c) UNet, (d) Deeplab v3, (e) PSPNet, and (f) VBI algorithm classification results.

[0028] Figure 8 is the five-feature image of Taihu region on May 27, 2017 and the result map of model output. DETAILED DESCRIPTION

[0029] The present application will be described in detail below with reference to the accompanying drawings, so that the steps and effects of the present application are more clear.

[0030] Figure 1 is the location of the three study lakes and the Landsat image map, including Lake Chaohu, Lake Taihu, and Lake Changlang. These lakes have serious eutrophication problems. Data source: Landsat reflectance image data and Landsat images obtained through the Google Earth Engine cloud platform.

[0031] As shown in Figure 2 is the flowchart of the shallow lake algal bloom and aquatic plant extraction method based on the deep learning model of the embodiment of the present application. The method of the embodiment specifically includes the following steps:

[0032] First step, image preparation: filter out Landsat 8 / 9 reflectance image data from 2013 to 2023 on the Google Earth Engine cloud platform. These images not only cover the three study lakes, but also ensure that the cloud cover is less than 30% to ensure the quality of the data. Then, filter out Sentinel-2 images on the Google Earth Engine cloud platform without cloud coverage, high imaging quality, and the imaging time difference with the imaging time of the target Landsat image is not more than one day.

[0033] Second step, making training and test samples: visual interpretation of Landsat images with the aid of Sentinel-2 high-resolution images to accurately make training samples and test samples, which correspond to the categories in the visual interpretation results. The visual interpretation results have four categories: water body, algal bloom, submerged plants and floating leaf emergent plants, which are respectively corresponding to 1, 2, 3 and 4, and are saved in the form of labels. The image size of the training sample is 256x256, and the test sample is shapefile point data. The training samples and test samples come from three eutrophic shallow lakes, multiple imaging periods and multiple scenes. In order to obtain sufficient training samples, data enhancement processing is performed on the training samples made. Figure 3 That is, the data enhancement operation.

[0034] Third step, feature construction and selection: a series of feature indexes for extracting algal bloom and aquatic plants are constructed, and the constructed feature indexes and Landsat reflectivity bands are input into the random forest model to obtain the importance of the features. The five most important features are selected to train the deep learning model; the formula and importance of a series of features input into the random forest model are shown in Table 1.

[0035] Table 1

[0036]

[0037] Based on the importance ranking in Table 1, the present application selects MNDWI, G, SWIR, FAI and EVI five features to input into the deep learning model, wherein, λ NIR is the wavelength of the near-infrared band, and λ R is the wavelength of the red light band.

[0038] Fourth step, constructing a deep learning model (AAENet) for extracting algal bloom and aquatic plants: Figure 4 is the convolutional neural network structure diagram for extracting algal bloom and aquatic plants, including an encoder, a pyramid pooling module ASPP and a decoder. In the model, the encoder contains a convolutional layer and a four-layer ConvNeXt network, which can achieve the best balance between precision and computational complexity. The feature image input into the encoder is subjected to convolution and normalization to obtain a feature map and sent to the ConvNeXt network, each layer of the ConvNeXt network performs convolution and pooling operation on the feature map in turn, and the feature image is input into the ASPP module for processing, the ASPP module obtains multi-scale feature maps through different dilation rates of the input feature map by using multi-scale strategy, so as to extract deep semantic features, then the multi-scale feature maps are fused to obtain a fused feature map and sent to the decoder, Figure 5The network structure of the ASPP module. The decoding part restores the small size feature map to the original size, thereby obtaining the algal bloom and aquatic plant segmentation map. In this embodiment, the Scse attention module is added to the decoder. The decoder performs 3 iterations of processing on the fused feature map. Each iteration includes bilinear interpolation upsampling and fusion with the feature map of the corresponding layer in the encoder, and enhancement processing is performed using the Scse attention module, and then convolution operation is performed. After the iteration is completed, bilinear interpolation upsampling is performed again to the size of the original input image, and convolution processing is performed to generate a single-band output image. In the decoding process, the bilinear interpolation output feature map is input into the Scse attention module, thereby further improving the performance of the model.

[0039] The ConvNeXt network proposed in 2022 is an innovative deep learning model that combines the design elements of convolutional neural networks while drawing on the strengths of the Transformer architecture, aiming to explore the performance limits of convolutional neural networks. ConvNext can better handle translation, rotation, and scaling transformations in images when processing complex image tasks, improving the robustness and generalization ability of the model. ConvNext has shown excellent performance in image classification and semantic segmentation due to its efficiency and accuracy. Therefore, this paper uses the ConvNext network as the feature extraction network.

[0040] The ASPP module uses a multi-scale strategy to perform dilated convolution on the input feature map with different expansion rates, and then fuses these multi-scale feature maps. This design not only effectively restores the details of the image, but also significantly improves the model's ability to capture deep semantic features, thereby maintaining high resolution while significantly enhancing the accuracy and robustness of semantic segmentation. Given the relatively small size of the feature map output by the ConvNext encoder, in order to more effectively extract deep semantic features, the invention adjusts the dilated convolution expansion rate of the ASPP module to 1, 2, 3, and 6. This smaller expansion rate setting reduces the parameters of the deep learning model while still maintaining the edge details of key information such as algal blooms and aquatic plants, thereby further optimizing the performance of the model.

[0041] Attention mechanisms play an important role in deep learning models, enabling the model to focus and enhance important features, thereby significantly improving the accuracy and efficiency of the model. This mechanism has been widely used in semantic segmentation tasks. The invention introduces the Scse attention mechanism to further improve the accuracy of model prediction. The Scse module does not change the dimension size of the feature map, but significantly improves the model's ability to fine-tune the edges of algal blooms and aquatic plants.

[0042] The five-band image with an image size of 256x256 is input into the AAENet model. The image is first processed by four ConvNext blocks to obtain a feature map with an image size of 768x8x8. Then, the obtained feature map with an image size of 768x8x8 is input into the ASPP module, which effectively extracts deep semantic features through multi-scale operations without changing the size of the feature map. Then, the model uses three iterations of upsampling processing. Each iteration includes: (1) bilinear interpolation upsampling of the feature map; (2) connecting the upsampling feature map with the feature map output by the corresponding level of the ConvNext block to fuse features at different levels; (3) using the Scse module to enhance the features after upsampling; (4) further integrating the features through convolution operations. After three iterations, the size of the feature map is restored to 96x64x64, which not only restores the spatial resolution but also significantly enhances the expression ability of the features. Finally, to generate the final output of the model, the restored feature map is bilinearly interpolated and upsampled to the size of the original input image, i.e., 96x256x256, and then processed through a convolution layer to generate a single-band output image with an image size of 256x256.

[0043] The detailed steps are described below in combination with image size changes. The model processing process is as follows:

[0044] 1. The input to the AAENet model is a 5x256x256 feature map (the feature map is 5 bands, with a row and column of 256x256 pixels). The 5 bands here are selected according to "feature selection".

[0045] 2. After "4x4 convolution, normalization", the size of the feature map becomes 96x64x64.

[0046] 3. After processing by four ConvNext blocks, the size of the feature map becomes 96x64x64, 192x32x32, 384x16x16, and 768x8x8. The ConvNext block contains two Conv operations.

[0047] 4. The feature map with an image size of 768x8x8 is input into the ASPP module, and the output feature map size is still 768x8x8.

[0048] 5. Three times of upsampling processing include: bilinear interpolation, fusion with the feature map output by the corresponding ConvNext block, Scse module, and two convolution operations.

[0049] The size of the first up-sampled input feature map is 768x8x8; the size of the feature map after bilinear interpolation is 384x16x16; the size of the feature map after fusion with the corresponding ConvNext block output is 768x16x16; the Scse module does not change the size of the feature map, and the size of the feature map is still 768x16x16; the size of the feature map after two convolutions is 384x16x16.

[0050] The feature map size changes in the second up-sampling process are 192x32x32, 384x32x32, 384x32x32, and 192x32x32.

[0051] The feature map size changes in the third up-sampling process are 96x64x64, 192x64x64, 192x64x64, and 96x64x64.

[0052] 6. Finally, the feature map with a size of 96x64x64 is bilinearly interpolated and up-sampled, and the feature map size changes to 96x256x256. Then, a convolutional layer is used for processing to generate a single-band output image with a size of 256x256.

[0053] Step 5: Training the deep learning model: randomly divide the feature images and corresponding visual interpretation results of the training samples into a training set and a validation set according to a ratio of 80% and 20% (the training samples and corresponding visual interpretation results produced in Step 2 can be divided first, and the model is trained based on the feature images of the training samples). The training set and the validation set are input into the deep learning model to train. Then, multiple iteration experiments are performed, and the training model with the highest accuracy is saved, thereby obtaining appropriate training hyperparameters. That is, the number of iteration rounds is set to 50, the batch size is set to 16, AdamW is selected as the optimizer, the initial learning rate is set to 10 -4 , the cross-entropy loss function and Dice Loss are combined for model training.

[0054]

[0055] Loss = 0.5 x Loss CE + 0.5 x Loss Dice

[0056] wherein, Loss CE , Loss Dice and Loss represent the cross-entropy loss function, Dice loss and the loss function used by the model, respectively, y i and p irespectively, n represents the total number of samples. The model converges after multiple rounds of iterative training, and the trained model exhibits excellent performance on the validation set. The model training curve is shown in FIG. 8. Figure 6

[0057] Step 6: Test the performance of the deep learning model: Select 3 typical eutrophic shallow lakes as test lakes, input the feature images of the selected 3 lakes into the deep learning model trained in step 5. Use accuracy, F1 score and frequency-weighted intersection over union (FWIoU) as three indicators to quantitatively evaluate the performance of the model. Compare the classification results of the trained deep learning model AAENet (the present application) with those of four other methods in the three lakes. The calculation formulas of the three indicators are as follows:

[0058]

[0059] wherein, acc, F1 and FWIoU represent accuracy, F1 score and frequency-weighted intersection over union, respectively; precision i and recall i represent the precision and recall of class i, respectively; TP i represents the number of pixels predicted by the model as class i and the true class is also i; FP i represents the number of pixels predicted by the model as class i and the true class is not i; FN i represents the number of pixels predicted by the model as not class i and the true class is i; TN i represents the number of pixels predicted by the model as not class i and the true class is also not i; N represents the number of all classes. The other four methods include UNet, Deeplab v3, PSPNet deep learning method and VBI algorithm. The extraction accuracy comparison of the five methods is shown in Table 2 and Figure 7

[0060] Table 2

[0061]

[0062] The results show that, compared with the other four methods, the AAENet deep learning model of the present application has the best effect and the highest accuracy in extracting water bloom and aquatic plants, and the target edge information is complete. Not only the water bloom and aquatic plants in large lakes (Taihu Lake and Chaohu Lake) are accurately extracted, but even those in small and medium-sized lakes (Changdang Lake) are completely extracted. The extraction accuracy of the three test lakes is all above 86%. This indicates that the AAENet model not only can accurately extract water bloom and aquatic plants in large lakes (Taihu Lake and Chaohu Lake), but also has good performance in small and medium-sized lakes (Changdang Lake).

[0063] ​​In the seventh step, the spatial and temporal distribution of algal blooms and aquatic plants in typical shallow lakes is obtained: the long-term characteristic images of the selected typical eutrophic shallow lakes are input into the deep learning model trained in the fifth step, so as to obtain the long-term spatial distribution map of algal blooms and aquatic plants. Figure 8 The results of the five characteristic images and the model output of the Taihu region on May 27, 2017 are shown.

[0064] In addition to the above embodiments, the present application can have other implementation manners. Any technical solution formed by equivalent replacement or equivalent transformation falls within the protection scope required by the present application.

Claims

1. A method for extracting shallow lake water bloom and aquatic plants based on a deep learning model, comprising the following steps: Step 1: Obtain Landsat reflectance images and Sentinel-2 images of the study area within a specified time range, and the imaging time of the Landsat reflectance images and the corresponding Sentinel-2 images differs by no more than one day; Step 2: Visually interpret and sample the Landsat reflectance images with the aid of the Sentinel-2 high-resolution images to accurately produce training samples and test samples, which correspond to the categories in the visual interpretation results; Step 3: Construct feature indexes for extracting water bloom and aquatic plants, and input the constructed feature indexes and each band of the Landsat reflectance images into a random forest model to obtain the importance of the feature indexes and sort them, select the top five feature indexes, and construct feature images of the Landsat reflectance images based on the five feature indexes for the training samples; Step 4: Construct a deep learning model for extracting water bloom and aquatic plants, including an encoder, a pyramid pooling module ASPP, and a decoder; Step 5: Divide the feature images of the training samples and the corresponding visual interpretation results into a training set and a validation set in proportion, input them into the constructed deep learning model for training, then iterate multiple times, and save the training model with the highest accuracy to obtain suitable training hyperparameters; Step 6: Test and evaluate the trained deep learning model based on the test samples; Step 7: Input the feature images of the Landsat reflectance images of each period of the study area into the trained deep learning model for processing to obtain the spatial distribution maps of water bloom and aquatic plants in each period; In the deep learning model, the encoder includes a convolutional layer and a ConvNeXt network with four layers, the feature images input into the encoder are subjected to convolution and normalization to obtain feature maps and sent to the ConvNeXt network, each layer of the ConvNeXt network sequentially performs convolution and pooling operations on the feature maps, and the feature images are input into the ASPP module for processing, the ASPP module obtains multi-scale feature maps through multi-scale strategies and different dilation rates of the input feature maps, then fuses the multi-scale feature maps to obtain a fused feature map and sends it to the decoder; the decoder adds a Scse attention module, the decoder iteratively processes the fused feature map three times, each iteration includes bilinear interpolation upsampling and fusion with the feature maps of the corresponding layers in the encoder, and enhancement processing is performed using the Scse attention module, then convolution operation is performed again, after iteration is completed, bilinear interpolation upsampling is performed again to the size of the original input image, and convolution processing is performed to generate a single-band output image.

2. The method for extracting algal blooms and aquatic plants in shallow lakes based on a deep learning model according to claim 1, characterized in that: In the first step, Landsat reflectance images and Sentinel-2 images were obtained from the Google Earth Engine cloud platform, the Landsat reflectance images covered the study lakes and the cloud coverage was less than 30%, and the Sentinel-2 images had no cloud coverage. 3.The method of claim 1, wherein the method is based on a deep learning model. In the second step, the training samples and test samples were derived from a plurality of shallow lakes with serious eutrophication. 4.The method of claim 1, wherein the method is based on a deep learning model. In the third step, the top five feature indexes were the normalized difference water index MNDWI, the green band G, the short-wave infrared band SWIR, the phytoplankton index FAI and the enhanced vegetation index EVI, wherein, ； ； ； where R is the red wavelength band, NIR is the near infrared wavelength band, B is the blue wavelength band, λ NIR is the wavelength of the near infrared wavelength band, and λ R is the wavelength of the red wavelength band. 5.The method of claim 1, wherein the method is based on a deep learning model. The dilation rate of the ASPP module is adjusted to 1, 2, 3 and 6. 6.The method of claim 1, wherein the method is based on a deep learning model. In the fifth step, the optimal hyperparameters of the iteration number of 50, the batch size of 16, AdamW as the optimizer, the initial learning rate of 10 -4 , and the combination of cross-entropy loss function and Dice Loss are obtained. 7.The method of claim 1, wherein the method is based on a deep learning model. In the sixth step, a plurality of typical eutrophic shallow lakes were selected as test samples, the feature images of the selected lakes were input into the deep learning model trained in the fifth step, and the performance of the model was quantitatively evaluated by comparing the recognition results with those of existing methods for extracting water bloom and aquatic plants using three indexes of accuracy, F1 score and frequency-weighted intersection over union FWIoU.

8. The method for extracting algal blooms and aquatic plants in shallow lakes based on a deep learning model according to claim 7, characterized in that: The existing methods for extracting water bloom and aquatic plants include UNet algorithm, Deeplab v3 algorithm, PSPNet algorithm and VBI algorithm. 9.The method of claim 1, wherein the method is based on a deep learning model. The categories of the visual interpretation results include water body, water bloom, submerged plants and floating leaf emergent plants.

Images