A cloud, snow and lake image recognition method for plateau mountains

Abstract

The present invention discloses a method for recognizing images of snowy lakes in plateau mountains, and specifically relates to the technical field of image recognition of snowy lakes. The present invention adopts support vector machines for segmentation, which greatly reduces the influence of different spectra of the same object and the same spectra of different objects in remote sensing images, can reduce the difficulty of segmentation and improve the classification accuracy, and is easier to annotate the images of snowy lakes so that they can meet the standards of segmentation data sets; the FPN deep learning model and the MobileNet‑v2 convolutional neural network are combined to extract features of different scales and improve the accuracy of prediction results. Because of the use of deep separable convolution, the number of model parameters is reduced and the deployment of equipment is facilitated; and the MobileNet‑v2 convolutional neural network introduces a linear bottleneck structure, a width multi-scale structure and an inverted residual structure to reduce the computational cost of the model, better retain information and improve the performance and nonlinear representation ability of the recognition method.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud snow lake image recognition, and in particular to a cloud snow lake image recognition method for plateau mountains. Background Art

[0002] Remote sensing images are an effective way to obtain geographic environment information and land resources. Optical remote sensing satellites have high resolution, rich content, and clear details for earth observation, which can be used for tasks such as monitoring land, forests, and the atmosphere. However, in the process of remote sensing satellite earth observation, the earth's surface is often blocked by clouds and covered by snow, making it impossible to monitor a large amount of landform information and surface features. At the same time, in the study of climate-human-environment interaction, clouds, snow, and lakes play a vital role in studying climate change, investigating hydrological resources, and issuing snow disaster warnings. Therefore, it is necessary to segment clouds, snow, and lakes in remote sensing images. Since clouds, snow, and ice lakes have similar reflectivity characteristics in the visible and near-infrared bands, they are easily confused when they coexist. Clouds are brighter and colder than most land surfaces. Existing cloud masking algorithms are generally effective, but they perform poorly in snow-covered areas and ice lake areas. The 1.38μm band in Sentinel-2 is used to detect thin cirrus clouds, but it is difficult to detect thin cirrus clouds on high-albedo surfaces such as snow, so it is challenging to distinguish the three in the segmentation task.

[0003] As a subfield of machine learning, deep learning has made significant progress in image processing in recent years, which has greatly promoted the detection of remote sensing clouds, snow and lakes. Deep learning can automatically capture the texture, shape and contextual information of images during training, and the results are often better than traditional methods. Although neural networks perform well in segmentation tasks, they are also very dependent on data. The quantity and quality of data sets will directly affect the effect of segmentation. Therefore, how to produce high-quality data sets is the difficulty of current segmentation, and visual interpretation is usually used for manual annotation. However, remote sensing images have large images, multiple bands, complex textures, and rich types of objects. If only visual interpretation is used for manual annotation, the workload is too large. Therefore, relying on some traditional algorithms to assist in annotation can increase the annotation speed and reduce the workload. At present, the main cloud, snow and lake auxiliary annotation methods include band threshold method and machine learning method. Band threshold method is widely used because of its advantages of simple algorithm and fast speed. Its basic idea is to perform spectral analysis and feature channel selection based on the reflectivity difference and brightness temperature difference of different objects, but it depends largely on the reflectivity of image bands and predefined thresholds, and lacks flexibility and robustness in complex situations. Machine learning algorithms use statistical principles to automatically segment target objects, which greatly reduces the high reliance of traditional detection methods on the spectral range of satellite image data. However, machine learning models usually require manual design of image features, making it difficult for the model to efficiently extract higher-level information from remote sensing images. When faced with complex scenes, such methods will face the problem of reduced accuracy. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method for recognizing cloud, snow and lake images in plateau mountains, which solves the problems of heavy workload and slow labeling speed of visual interpretation method for labeling clouds, snow and lakes, low accuracy of band threshold method and difficulty in feature selection of machine learning algorithm.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0006] A method for recognizing cloud, snow and lake images in plateau mountains is provided, which comprises the following steps:

[0007] S1. Obtain remote sensing image data of cloud, snow and lakes in plateau mountains;

[0008] S2, preprocessing the remote sensing image data to obtain false color image data;

[0009] S3. Input the false color image data into the cloud and snow lake image recognition model to obtain the recognition result, thereby completing the recognition of the cloud and snow lake image.

[0010] Further, step S2 includes the following steps:

[0011] S2-1, perform atmospheric correction and radiation calibration on the remote sensing image data to obtain the corresponding atmospheric reflectance data;

[0012] S2-2, standardizing the atmospheric reflectivity data to obtain standardized atmospheric reflectivity data;

[0013] S2-3, performing spectral analysis on the standardized atmospheric reflectance data to obtain corresponding analysis results;

[0014] S2-4, based on the analysis results, the remote sensing image data is resampled using a bilinear interpolation method to obtain resampled remote sensing image data;

[0015] S2-5, performing band synthesis on the resampled remote sensing image data and eliminating true color image data to obtain false color image data.

[0016] Furthermore, the false color image data uses false color images corresponding to SWIR, Green, and Blue bands.

[0017] Further, the cloud and snow lake image recognition model in step S3 includes a MobileNet-v2 convolutional neural network and an FPN deep learning model connected in series; the MobileNet-v2 convolutional neural network includes a depth-by-depth convolutional layer and a point-by-point convolutional layer connected in series; the FPN deep learning model includes an encoder and a decoder connected in series; the encoder includes a first convolutional layer and a Bottleneck module connected in series; the Bottleneck module includes a first Bottleneck unit, a second Bottleneck unit and a third Bottleneck unit connected in series, and the first Bottleneck unit, the second Bottleneck unit and the third Bottleneck unit all include a Bottleneck1 layer and a Bottleneck2 layer connected in series; the Bottleneck1 layer includes a second convolutional layer, a first Dwise layer and a linear convolutional layer connected in series; the Bottleneck2 layer includes a third convolutional layer, a second Dwise layer and a linear convolutional layer connected in series;

[0018] The decoder includes a first sampling module, a second sampling module and a third sampling module in parallel, and the first sampling module and the second sampling module, and the second sampling module and the third sampling module are connected through an upsampling layer; the first sampling module, the second sampling module and the third sampling module each include two fourth convolutional layers and an upsampling layer connected in series.

[0019] Furthermore, the first convolutional layer is activated by the Relu function, and the convolution kernel size is 3×3 and the stride is 2; the second and third convolutional layers are both activated by the Relu function, and the convolution kernel size is 1×1; the first Dwise layer is activated by the Relu function, and the convolution kernel size is 3×3 and the stride is 2; the second Dwise layer is activated by the Relu function, and the convolution kernel size is 3×3; the convolution kernel size of each linear convolutional layer is 1×1; each fourth convolutional layer is activated by the Relu function, and the convolution kernel size is 3×3 and the stride is 1.

[0020] Furthermore, the training process of the cloud and snow lake image recognition model in step S3 includes the following steps:

[0021] S3-1, obtaining training remote sensing image data and processing it in the same manner as steps S1 to S2 to obtain training false color image data;

[0022] S3-2, using SVM to segment the training false color image data to obtain initial training segmentation data;

[0023] S3-3, randomly crop and enhance the training segmentation data set to obtain the final training segmentation data;

[0024] S3-4, input the final training segmentation data into the depth-by-depth convolution layer to obtain the first feature map data;

[0025] S3-5, inputting the first feature map data into the point-by-point convolution layer to obtain second feature map data;

[0026] S3-6, inputting the second feature map data into the first convolutional layer and the first Bottleneck unit to obtain third feature map data;

[0027] S3-7, inputting the third characteristic map data into the second Bottleneck unit to obtain fourth characteristic map data;

[0028] S3-8, inputting the fourth characteristic map data into the third Bottleneck unit to obtain fifth characteristic map data;

[0029] S3-9, inputting the fifth feature map data into the first sampling module to obtain a first prediction result;

[0030] S3-10, splicing the fourth feature map data and the output data of the first fourth convolutional layer of the first sampling module and inputting the concatenated data into the second sampling module to obtain a second prediction result;

[0031] S3-11, splicing the third feature map data and the output data of the first fourth convolutional layer of the second sampling module and inputting the concatenated data into the third sampling module to obtain a third prediction result;

[0032] S3-12, adding the first prediction result, the second prediction result and the third prediction result pixel by pixel and processing them using the Sigmoid function to obtain a final prediction result;

[0033] S3-13, calculate the final prediction result and obtain the cross entropy loss function;

[0034] S3-14. Adjust the parameters of the cloud and snow lake image recognition model based on the cross entropy loss function until the optimal weight of the cloud and snow lake image recognition model is obtained, and complete the training of the cloud and snow lake image recognition model.

[0035] Further, step S3-2 includes the following steps:

[0036] S3-2-1, using Labelme to divide the training false color image data into blocks, obtaining the corresponding divided training false color image data and retaining the corresponding position information;

[0037] S3-2-2, using SVM to segment the training false color image data after each block, to obtain the corresponding training segmentation results;

[0038] S3-2-3, correcting each training segmentation result to obtain a corresponding corrected training segmentation result;

[0039] S3-2-4, according to the location information, stitching the corrected training segmentation results to generate corresponding labels; wherein the labels include cloud and snow labels and lake labels;

[0040] The initial training segmentation data includes the training remote sensing image data and the labels of the segmentation dataset.

[0041] Further, step S3-14 includes the following steps:

[0042] S3-14-1. Based on the cross entropy loss function, the Adam optimizer is used to adjust the parameters of the cloud and snow lake image recognition model to obtain the corresponding weights;

[0043] S3-14-2, repeat steps S3-4 to S3-13, and use the same method as step S3-14-1 to update all weights;

[0044] S3-14-3. Apply all weights to the cloud and snow lake image recognition model;

[0045] S3-14-4, obtain verification remote sensing image data; use the same method as steps S1 to S3 to process the verification remote sensing image data and evenly crop it to obtain verification segmentation data; S3-14-5, input the verification segmentation data into cloud and snow lake image recognition models with different weights to obtain corresponding test results;

[0046] S3-14-6. Calculate the MIoU value of each test result, take the weight corresponding to the highest MIoU value as the optimal weight, and apply it to the Yunxue Lake image recognition model to complete the training of the Yunxue Lake image recognition model.

[0047] The beneficial effects of the present invention are:

[0048] 1. This method uses support vector machine (SVM) for segmentation, which greatly reduces the number of remote sensing images with the same object but different spectra and different objects but the same spectra, which can reduce the segmentation difficulty and improve the classification accuracy. It is easier to annotate the cloud, snow and lake images so that they can meet the standards of the segmentation data set;

[0049] 2. This method combines the FPN deep learning model and the MobileNet-v2 convolutional neural network to extract features of different scales and improve the accuracy of the prediction results. The use of deep separable convolution reduces the number of model parameters and facilitates device deployment. The MobileNet-v2 convolutional neural network introduces a linear bottleneck structure, a wide multi-scale structure, and an inverted residual structure to reduce the computational cost of the model, better retain information, and improve the performance of the recognition method and its nonlinear representation capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a method flow chart;

[0051] Figure 2 This is a schematic diagram of the structure of the cloud and snow lake image recognition model;

[0052] Figure 3 It is the MobileNet-v2 convolutional neural network;

[0053] Figure 4 Schematic diagram of data processing for the cloud and snow lake image recognition model. DETAILED DESCRIPTION

[0054] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0055] like Figure 1 As shown, a cloud snow lake image recognition method for plateau mountains includes the following steps:

[0056] S1. Obtain remote sensing image data of cloud, snow and lakes in plateau mountains;

[0057] S2, preprocessing the remote sensing image data to obtain false color image data;

[0058] Step S2 includes the following steps:

[0059] S2-1, perform atmospheric correction and radiation calibration on the remote sensing image data to obtain the corresponding atmospheric reflectance data;

[0060] S2-2, standardizing the atmospheric reflectivity data to obtain standardized atmospheric reflectivity data;

[0061] The purpose of atmospheric correction is to eliminate the influence of factors such as atmosphere and illumination on the reflection of objects, and obtain formal physical model parameters such as object reflectivity, emissivity, and surface temperature, including eliminating the influence of water vapor, oxygen, carbon dioxide, methane, and ozone in the atmosphere on the reflection of objects; eliminating the influence of atmospheric molecules and aerosol scattering. Radiometric calibration is the process of converting the digital quantization value (DN) of an image into a physical quantity such as radiation brightness value or reflectivity or surface temperature. Since the value range of RGB images is an integer between 0-255, and the values ​​of atmospheric reflectivity are all floating-point numbers between 0-1, they cannot be directly visualized. Therefore, the atmospheric reflectivity is scaled as a whole, and its value is enlarged to between 0-255 and rounded.

[0062] S2-3, perform spectral analysis on the standardized atmospheric reflectance data to obtain corresponding analysis results, that is, analyze the resolution of remote sensing image data in different bands;

[0063] S2-4. Based on the analysis results, the remote sensing image data is resampled using a bilinear interpolation method to obtain resampled remote sensing image data; the resampling method includes but is not limited to a bilinear interpolation method, and the corresponding resampling method is selected based on the analysis results.

[0064] S2-5, performing band synthesis on the resampled remote sensing image data and eliminating true color image data to obtain false color image data.

[0065] In true color images, clouds and snow appear white, and the spectral difference is not obvious, which is not conducive to segmentation. In the false color images synthesized by the SWIR, Green, and Blue bands, the clouds are still grayish white, while the snow appears sky blue. The two are effectively distinguished in the spectrum. Therefore, the false color image data uses the false color images corresponding to the SWIR, Green, and Blue bands.

[0066] S3. Input the false color image data into the cloud and snow lake image recognition model to obtain the recognition result, thereby completing the recognition of the cloud and snow lake image.

[0067] like Figure 2 and Figure 3 As shown, the cloud and snow lake image recognition model in step S3 includes a MobileNet-v2 convolutional neural network and an FPN deep learning model connected in series; the MobileNet-v2 convolutional neural network includes a depth-by-depth convolutional layer and a point-by-point convolutional layer connected in series; the FPN deep learning model includes an encoder and a decoder connected in series; the encoder includes a first convolutional layer and a Bottleneck module connected in series; the Bottleneck module includes a first Bottleneck unit, a second Bottleneck unit and a third Bottleneck unit connected in series, and the first Bottleneck unit, the second Bottleneck unit and the third Bottleneck unit all include a Bottleneck1 layer and a Bottleneck2 layer connected in series; the Bottleneck1 layer includes a second convolutional layer, a first Dwise layer and a linear convolutional layer connected in series, which are used to reduce the scale of the feature map to half of the original scale and perform feature extraction; the Bottleneck2 layer includes a third convolutional layer, a second Dwise layer and a linear convolutional layer connected in series, which are used to double the number of channels of the feature map;

[0068] The decoder includes a first sampling module, a second sampling module and a third sampling module in parallel, and the first sampling module and the second sampling module, and the second sampling module and the third sampling module are connected through an upsampling layer; the first sampling module, the second sampling module and the third sampling module each include two fourth convolutional layers and an upsampling layer connected in series.

[0069] The first convolutional layer is activated by the Relu function, and the convolution kernel size is 3×3 and the stride is 2; the second and third convolutional layers are both activated by the Relu function, and the convolution kernel size is 1×1; the first Dwise layer is activated by the Relu function, and the convolution kernel size is 3×3 and the stride is 2; the second Dwise layer is activated by the Relu function, and the convolution kernel size is 3×3; the convolution kernel size of each linear convolutional layer is 1×1; each fourth convolutional layer is activated by the Relu function, and the convolution kernel size is 3×3 and the stride is 1.

[0070] The FPN deep learning model is a feature pyramid network, which is a pyramid-shaped network composed of different scale features extracted by the deep convolutional neural network (DCNN). It can be divided into feature extraction, upsampling, feature fusion and multi-scale feature output. Its input is image data of size 512×512. The entire network structure of the FPN deep learning model can be divided into two parts: bottom-up (Bottom-Up) and top-down (Top-Down). Bottom-Up is the feature extraction process, corresponding to the Encoder part in Unet. Top-Down samples the deepest features through layers of upsampling to the resolution size corresponding to the output of Bottom-Up, and then outputs the feature map after fusion. The fusion method is to add the corresponding positions.

[0071] MobileNetV2 is a lightweight convolutional neural network architecture designed specifically for scenarios with limited computing resources on mobile and embedded devices. It uses depthwise separable convolutions to reduce the number of model parameters and the amount of computation, decomposes standard convolutions into depthwise convolutions and pointwise convolutions to reduce computational complexity, and introduces a linear bottleneck structure to map input channels to lower-dimensional intermediate representations and then map them back to the target dimension, which helps reduce the computational cost of the model; it uses an inverted residual structure in which the connection between input and output is direct rather than through an added layer. This structure is able to better retain information and improve the performance of the model. A width multi-scale structure is introduced to allow the model to be scaled at different width factors to provide a trade-off between different performance and model sizes. Using the ReLU6 activation function, negative values ​​are truncated to zero and positive values ​​are truncated to 6 to improve the nonlinear representation capabilities of the model.

[0072] The training process of the cloud and snow lake image recognition model in step S3 includes the following steps:

[0073] S3-1, obtaining training remote sensing image data and processing it in the same manner as steps S1 to S2 to obtain training false color image data;

[0074] S3-2, using SVM to segment the training false color image data to obtain initial training segmentation data;

[0075] Since the phenomenon of different spectra for the same object and the same spectra for different objects is obvious in remote sensing images, which is manifested in the similar spectral characteristics of clouds and underlying materials, and low grayscale values ​​of shadows and lakes, etc., this brings great difficulties to segmentation. If SVM is used directly to classify remote sensing images, the accuracy is very low. Therefore, firstly, the target objects in the remote sensing images are manually divided into blocks using labelme software, that is, the target objects are cropped with rectangular frames to reduce the spatial range of the target objects, so as to reduce the impact of the background on the spectrum of the target objects. Then, SVM is used to classify the target objects in each block, so that step S3-2 includes the following steps:

[0076] S3-2-1, using Labelme to divide the training false color image data into blocks, obtaining the corresponding divided training false color image data and retaining the corresponding position information;

[0077] S3-2-2, using SVM to segment the training false color image data after each block, to obtain the corresponding training segmentation results;

[0078] Among them, the specific processing process of SVM is as follows:

[0079] 1. Data preparation: collect data sets and preprocess the data, including feature selection, feature scaling, and processing missing values; divide the data sets into training sets and test sets, and usually use cross-validation to evaluate the performance of the model;

[0080] 2. Select kernel function: Select the appropriate kernel function according to the nature of the data; kernel functions include linear kernel, polynomial kernel, and Gaussian kernel;

[0081] 3. Training model: Use the training set to train the SVM model; during the training process, the SVM will find an optimal hyperplane to maximize the margin and correctly classify the training samples. The complexity of the model can be controlled by adjusting the regularization parameters to avoid overfitting.

[0082] 4. Model evaluation: Use the test set to evaluate the performance of the trained model. Usually, indicators such as accuracy, precision, recall, and F1 score are used to evaluate the performance of the classification model. Methods such as cross-validation can be used to more reliably evaluate the performance of the model.

[0083] 5. Parameter tuning: According to the performance of the model on the test set, adjust the parameters of the model to improve performance. This includes adjusting the parameters of the kernel function, regularization parameters, etc.

[0084] 6. Model optimization and adjustment: The model can be further optimized, such as by feature engineering, adjusting the kernel function, etc. to improve the performance of the model. Ensemble learning methods such as random forests and gradient boosting trees can be used in combination with SVM to improve classification performance. Since SVM is a supervised classification algorithm, sample data is required to train SVM. Therefore, an appropriate amount of sample points are collected for each block to train SVM, and then the entire block is predicted to obtain the segmentation result.

[0085] S3-2-3, correcting each training segmentation result to obtain a corresponding corrected training segmentation result;

[0086] S3-2-4, according to the location information, stitching the corrected training segmentation results to generate corresponding labels; wherein the labels include cloud and snow labels and lake labels;

[0087] The initial training segmentation data includes the training remote sensing image data and the labels of the segmentation dataset.

[0088] Although SVM can accurately segment clouds, snow and lakes, its workload is much greater than that of neural networks and it cannot perform large-scale segmentation. Therefore, SVM is used to label a small number of samples for neural networks to learn, and the trained neural network can perform large-scale and fast segmentation of clouds, snow and lakes.

[0089] S3-3. Since the size of the remote sensing image is too large, directly passing it into the neural network will result in insufficient computer video memory, so the training segmentation data set is randomly cropped and data enhanced to obtain the final training segmentation data with a size of 512×512;

[0090] Among them, random cropping will generate random points on the image, and the point will be used as the center point of the cropping result for cropping, further expanding the data set, which helps to improve the accuracy of the training model; data enhancement is usually a process of artificially increasing the amount of data by generating new data points from existing data, including perturbing the data in different directions or using a deep learning model to generate new data points in the latent space of the original data to expand the new data set. The data enhancement methods used in this method include: rotation, rotating the image at a certain angle, which can increase samples at different angles; flipping, flipping the image horizontally or vertically to generate mirror samples; scaling, changing the size of the image, which can generate samples of different scales; translation, translating the image on the plane to increase samples with position changes; shearing, cropping in a certain area of ​​the image, which can highlight a certain part of the image features; color transformation, adjusting the brightness, contrast, hue, etc. of the image, and increasing the samples of the image under different lighting conditions; adding noise, adding random noise to the image to make the model more robust; combined transformation, combining multiple transformations and applying them to generate richer samples. S3-4, input the final training segmentation data into the depth-by-depth convolution layer to obtain the first feature map data;

[0091] S3-5, inputting the first feature map data into the point-by-point convolution layer to obtain second feature map data;

[0092] S3-6, inputting the second feature map data into the first convolutional layer and the first Bottleneck unit to obtain third feature map data;

[0093] like Figure 4 As shown in the figure, the size of the third feature map data is 128×128×64; the Bottleneck1 layer is used to reduce the feature map scale to half of the original scale for feature extraction. First, the input of Bottleneck1 (i.e., 256×256×32 feature map) is expanded through a 1×1 convolution to expand the number of channels of the feature map, and then the feature map scale is reduced to half of the original scale through a depth-separable convolution with a convolution kernel size of 3×3 and a step size of 2, and finally the number of feature map channels is reduced through a 1×1 convolution to complete the construction of the inverse residual module; the Bottleneck2 layer is used to double the number of channels of the output of Bottleneck1. First, the input (i.e., the output of Bottleneck1) is expanded through a 1×1 convolution to expand the number of channels of the feature map, and then the feature is extracted through a depth-separable convolution with a convolution kernel size of 3×3 and a step size of 1, and then the number of feature map channels is reduced through a 1×1 convolution, and finally the output feature map is spliced ​​with the input using a residual connection to double the number of feature map channels.

[0094] S3-7, inputting the third feature map data into the second Bottleneck unit to obtain fourth feature map data; wherein the size of the fourth feature map data is 64×64×128;

[0095] S3-8, inputting the fourth feature map data into the third Bottleneck unit to obtain fifth feature map data; wherein the size of the fifth feature map data is 32×32×256;

[0096] S3-9, inputting the fifth feature map data into the first sampling module to obtain a first prediction result; wherein the size of the first prediction result is 512×512×4;

[0097] S3-10, splicing the fourth feature map data and the output data of the first fourth convolutional layer of the first sampling module and inputting the concatenated data into the second sampling module to obtain a second prediction result; wherein the size of the second prediction result is 512×512×4;

[0098] S3-11, splicing the third feature map data and the output data of the first fourth convolutional layer of the second sampling module and inputting the concatenated data into the third sampling module to obtain a third prediction result; wherein the size of the third prediction result is 512×512×4;

[0099] S3-12, adding the first prediction result, the second prediction result and the third prediction result pixel by pixel and processing them using the Sigmoid function to obtain a final prediction result;

[0100] S3-13, calculate the final prediction result and obtain the cross entropy loss function;

[0101] S3-14. Adjust the parameters of the cloud and snow lake image recognition model based on the cross entropy loss function until the optimal weight of the cloud and snow lake image recognition model is obtained, and complete the training of the cloud and snow lake image recognition model.

[0102] S3-14-1. Based on the cross entropy loss function, the Adam optimizer is used to adjust the parameters of the cloud and snow lake image recognition model to obtain the corresponding weights;

[0103] S3-14-2, repeat steps S3-4 to S3-13, and use the same method as step S3-14-1 to update all weights;

[0104] S3-14-3. Apply all weights to the cloud and snow lake image recognition model;

[0105] S3-14-4, obtain verification remote sensing image data; use the same method as steps S1 to S3 to process the verification remote sensing image data and evenly crop it to obtain verification segmentation data; S3-14-5, input the verification segmentation data into cloud and snow lake image recognition models with different weights to obtain corresponding test results;

[0106] S3-14-6. Calculate the MIoU value of each test result, take the weight corresponding to the highest MIoU value as the optimal weight, and apply it to the Yunxue Lake image recognition model to complete the training of the Yunxue Lake image recognition model.

[0107] The calculation formula of MIoU value is:

[0108]

[0109] Where M represents the number of categories, ∑(·) represents the summation function, and pii, pji, and pij represent the number of predicted j in category i, the number of predicted i in category j, and the number of predicted j in category i, respectively.

[0110] In one embodiment of the present invention, the Sentinel-2 multispectral L2C remote sensing image product is selected for experiment. Because Sentinel-2 has 13 bands, the pixel size of each band is 10, 20 or 60 meters, etc. Among them, the blue (B2), green (B3), red (B4) and near infrared (B8) bands have a resolution of 10 meters; the ground sampling distance of the red end (B5), near infrared NIR (B6, B7 and B8A) and short wave infrared SWIR (B11 and B12) is 20 meters; the pixel size of the coastal atmospheric aerosol (B1) and cirrus band (B10) is 60 meters. Since the band resolutions are not exactly the same, it is impossible to directly perform band synthesis, so the bilinear interpolation method is currently selected for resampling to unify the resolution of all bands to 10m. This method is used to process Sentinel-2 multispectral L2C remote sensing image products to obtain segmented data sets, and the training set, validation set and test set are divided into a ratio of 3:1:1; the training set is used to train the neural network, determine the parameters such as the weight and bias of the model, let the model learn the key features of the data for prediction, and use random cropping and data enhancement; the validation set is used to find the epoch with the best training effect as the final prediction model, and regular grid cropping is used; the test set is used to evaluate the effect of the model, and regular grid cropping is used; regular grid cropping is to crop the remote sensing image in a regular order from left to right and from top to bottom. Compared with random cropping, standard grid cropping can splice the prediction effect according to the sequence number, so as to see the overall prediction effect.

[0111] The overall accuracy OA, category average pixel accuracy MPA, category average pixel recall MPR, and average intersection over union ratio MIoU are used as evaluation indicators. The LinkNet model, U-Net model, PAN model, and Deeplabv3+ model are used to compare with the network model of this method. The training set, validation set, and test set are input into each model for processing to obtain the corresponding evaluation index values.

[0112] Table 1 Model accuracy evaluation

[0113]

[0114] As shown in Table 1, the overall accuracy OA of the network model of this method is 98.41, which is 2.21, 0.88, 0.56, and 0.54 higher than the overall accuracy OA of the LinkNet model, U-Net model, PAN model, and Deeplabv3+ model, respectively; the category average pixel accuracy MPA of the network model of this method is 88.51, which is 6.74, 3.69, 2.12, and 4.70 higher than the category average pixel accuracy MPA of the LinkNet model, U-Net model, PAN model, and Deeplabv3+ model, respectively. 2.09; the category average pixel recall rate MPR of the network model of this method is 96.14, which is 0.57 and 0.55 higher than the category average pixel recall rate MPR of the PAN model and Deeplabv3+ model respectively; the average intersection-over-union ratio MIoU of the network model of this method is 86.06, which is 6.45, 3.3, 2.57 and 2.34 higher than the average intersection-over-union ratio MIoU of the LinkNet model, U-Net model, PAN model and Deeplabv3+ model respectively; the above experimental data show the excellence of the model of this method.

[0115] In summary, the present invention uses support vector machine (SVM) for segmentation, which greatly reduces the images of the same object with different spectra and different objects with the same spectra in remote sensing images, can reduce the difficulty of segmentation and improve the classification accuracy, and is easier to annotate the images of cloud, snow and lakes so that they can meet the standards of segmentation data sets; the FPN deep learning model and the MobileNet-v2 convolutional neural network are combined to extract features of different scales and improve the accuracy of prediction results. In addition, because of the use of deep separable convolution, the number of model parameters is reduced and the deployment of equipment is facilitated; and the MobileNet-v2 convolutional neural network introduces a linear bottleneck structure, a width multi-scale structure and an inverted residual structure to reduce the computational cost of the model, better retain information and improve the performance of the recognition method and the nonlinear representation ability.

Claims

1. A cloud, snow and lake image recognition method for plateau mountains, characterized by: The following steps are involved: S1. Obtain remote sensing image data of cloud, snow and lakes in plateau mountains; S2, preprocessing the remote sensing image data to obtain false color image data; S3, inputting the false color image data into the cloud and snow lake image recognition model to obtain the recognition result, thereby completing the recognition of the cloud and snow lake image; The cloud and snow lake image recognition model in step S3 includes a MobileNet-v2 convolutional neural network and an FPN deep learning model connected in series; the MobileNet-v2 convolutional neural network includes a depth-by-depth convolutional layer and a point-by-point convolutional layer connected in series; the FPN deep learning model includes an encoder and a decoder connected in series; the encoder includes a first convolutional layer and a Bottleneck module connected in series; the Bottleneck module includes a first Bottleneck unit, a second Bottleneck unit and a third Bottleneck unit connected in series, and the first Bottleneck unit, the second Bottleneck unit and the third Bottleneck unit all include a Bottleneck1 layer and a Bottleneck2 layer connected in series; the Bottleneck1 layer includes a second convolutional layer, a first Dwise layer and a linear convolutional layer connected in series; the Bottleneck2 layer includes a third convolutional layer, a second Dwise layer and a linear convolutional layer connected in series; The decoder includes a first sampling module, a second sampling module and a third sampling module in parallel, and the first sampling module and the second sampling module, and the second sampling module and the third sampling module are connected through an upsampling layer; the first sampling module, the second sampling module and the third sampling module each include two fourth convolutional layers and an upsampling layer connected in series; The first convolutional layer is activated by the Relu function, and the convolution kernel size is The second and third convolutional layers are activated by Relu function, and the convolution kernel size is ; The first Dwise layer is activated by the Relu function, and the convolution kernel size is and the step size is 2; the second Dwise layer uses the Relu function for activation, and the convolution kernel size is ; The convolution kernel size of each linear convolution layer is ; Each of the fourth convolutional layers is activated by the Relu function, and the convolution kernel size is and the step size is 1; The training process of the cloud and snow lake image recognition model in step S3 includes the following steps: S3-1, obtaining training remote sensing image data and processing it in the same manner as steps S1 to S2 to obtain training false color image data; S3-2, using SVM to segment the training false color image data to obtain initial training segmentation data; S3-3, randomly crop and enhance the training segmentation data set to obtain the final training segmentation data; S3-4, input the final training segmentation data into the depth-by-depth convolution layer to obtain the first feature map data; S3-5, inputting the first feature map data into the point-by-point convolution layer to obtain second feature map data; S3-6, inputting the second feature map data into the first convolutional layer and the first Bottleneck unit to obtain third feature map data; S3-7, inputting the third characteristic map data into the second Bottleneck unit to obtain fourth characteristic map data; S3-8, inputting the fourth characteristic map data into the third Bottleneck unit to obtain fifth characteristic map data; S3-9, inputting the fifth feature map data into the first sampling module to obtain a first prediction result; S3-10, splicing the fourth feature map data and the output data of the first fourth convolutional layer of the first sampling module and inputting the concatenated data into the second sampling module to obtain a second prediction result; S3-11, splicing the third feature map data and the output data of the first fourth convolutional layer of the second sampling module and inputting the concatenated data into the third sampling module to obtain a third prediction result; S3-12, adding the first prediction result, the second prediction result and the third prediction result pixel by pixel and processing them using the Sigmoid function to obtain a final prediction result; S3-13, calculate the final prediction result and obtain the cross entropy loss function; S3-14. Adjust the parameters of the cloud and snow lake image recognition model based on the cross entropy loss function until the optimal weight of the cloud and snow lake image recognition model is obtained, and complete the training of the cloud and snow lake image recognition model.

2. The cloud, snow and lake image recognition method for plateau mountains according to claim 1 is characterized by: The step S2 comprises the following steps: S2-1, perform atmospheric correction and radiation calibration on the remote sensing image data to obtain the corresponding atmospheric reflectance data; S2-2, standardizing the atmospheric reflectivity data to obtain standardized atmospheric reflectivity data; S2-3, performing spectral analysis on the standardized atmospheric reflectance data to obtain corresponding analysis results; S2-4, based on the analysis results, the remote sensing image data is resampled using a bilinear interpolation method to obtain resampled remote sensing image data; S2-5, performing band synthesis on the resampled remote sensing image data and eliminating true color image data to obtain false color image data.

3. The cloud, snow and lake image recognition method for plateau mountains according to claim 1 is characterized by: The false color image data adopts false color images corresponding to SWIR, Green, and Blue bands.

4. The cloud, snow and lake image recognition method for plateau mountains according to claim 1 is characterized by: The step S3-2 comprises the following steps: S3-2-1, using Labelme to divide the training false color image data into blocks, obtaining the corresponding divided training false color image data and retaining the corresponding position information; S3-2-2, using SVM to segment the training false color image data after each block, to obtain the corresponding training segmentation results; S3-2-3, correcting each training segmentation result to obtain a corresponding corrected training segmentation result; S3-2-4, according to the location information, stitching the corrected training segmentation results to generate corresponding labels; wherein the labels include cloud and snow labels and lake labels; The initial training segmentation data includes training remote sensing image data and labels of the segmentation data set.

5. The cloud, snow and lake image recognition method for plateau mountains according to claim 4 is characterized by: The step S3-14 comprises the following steps: S3-14-1. Based on the cross entropy loss function, the Adam optimizer is used to adjust the parameters of the cloud and snow lake image recognition model to obtain the corresponding weights; S3-14-2, repeat steps S3-4 to S3-13, and use the same method as step S3-14-1 to update all weights; S3-14-3. Apply all weights to the cloud and snow lake image recognition model; S3-14-4, obtaining verification remote sensing image data; using the same method as steps S1 to S3 to process the verification remote sensing image data and evenly crop it to obtain verification segmentation data; S3-14-5. Input the verification segmentation data into the cloud and snow lake image recognition models with different weights to obtain the corresponding test results; S3-14-6. Calculate the MIoU value of each test result, take the weight corresponding to the highest MIoU value as the optimal weight, and apply it to the Yunxue Lake image recognition model to complete the training of the Yunxue Lake image recognition model.

Images