Alpine region artificial forest classification method and system based on multi-source remote sensing data and convolutional neural network

Abstract

The invention discloses an alpine region artificial forest classification method and system based on multi-source remote sensing data and a convolutional neural network, and the method comprises the following steps: selecting a sampling point in an alpine region, marking a land coverage type, obtaining the multi-source remote sensing image data at the sampling point, carrying out the preprocessing, and obtaining the needed data, forming a data set by the required data and the land coverage type labels; a three-channel convolutional neural network model is constructed, the three-channel convolutional neural network model comprises an input layer, a convolutional layer, a global average pooling layer, a connection layer, a full connection layer and an output layer, and the output layer maps the features to land coverage types; training a three-channel convolutional neural network model by using the data set; and acquiring data at the same to-be-classified sample point in the alpine region, inputting the data into the trained three-channel convolutional neural network model, predicting the artificial forest, and completing classification of the artificial forest in the alpine region. The method can be widely applied to accurate monitoring of artificial forests in alpine regions.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image vegetation classification, and in particular relates to a method and system for classifying artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks. Background Art

[0002] Afforestation in alpine regions is a challenging task. Due to the limited suitable forest areas, future afforestation work needs to be planned based on the distribution and characteristics of successfully planted trees. Remote sensing technology can provide high-precision spatial data to fully understand and monitor the actual distribution of plantations in alpine regions. However, the vegetation coverage of plantations in alpine regions is low and the distribution is scattered, making remote sensing-based identification extremely challenging. Existing studies have found that current remote sensing-based studies often underestimate the extent of existing forests with low coverage, especially in areas with scattered plantations. In addition, many large-scale planned afforestation areas currently already have trees distributed. Taking the Qinghai-Tibet Plateau as an example, previous studies have underestimated the effectiveness of existing ecological engineering and overestimated the carbon sequestration potential in low-coverage areas.

[0003] The current systematic underestimation of plantation areas is mainly due to the following two constraints: first, large-scale identification of plantations often fails to incorporate key features that distinguish plantations from natural forests and other vegetation, especially at fine scales (30m resolution); and traditional feature recognition models rely on expert knowledge to distinguish feature information for labeling and cannot automatically extract important features from remote sensing data.

[0004] Therefore, it is urgent to solve the above problems. Summary of the invention

[0005] Purpose of the invention: The first purpose of the present invention is to provide a classification method for artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks. The classification method can extract and fuse the vertical structure characteristics, long-term spectral characteristics and spatial spectral characteristics of the data, amplify the differences in time series data between artificial forests and natural forests, and can be widely used in the precise monitoring of artificial forests in high-cold areas.

[0006] The second object of the present invention is to provide a system for classifying artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks.

[0007] Technical solution: To achieve the above purpose, the present invention discloses a method for classifying artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks, comprising the following steps: (1) Sample points were selected in the alpine areas and the land cover types were marked. Multi-source remote sensing image data were obtained at the sample points. The multi-source remote sensing image data were preprocessed to obtain Landsat multispectral data, Sentinel-1 polarimetric band data, and Sentinel-2 multispectral data. The land cover types included artificial forests, natural forests, shrubs, grasslands, cultivated land, bare land, and other areas. (2) The Landsat multispectral data, Sentinel-1 polarization band data, Sentinel-2 multispectral data, and land cover type labels are used to form a data set, which is then divided into a training data set and a validation data set according to a certain ratio; (3) Construct a three-channel convolutional neural network model, which includes an input layer, a convolution layer, a global average pooling layer, a connection layer, a fully connected layer and an output layer. The input layer has three input channels, and the input data of the three input channels are Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data respectively. The convolution layer has three feature extraction channels, and each feature extraction channel is connected to a global average pooling layer. The outputs of the three global average pooling layers are the spectral features of the long time series, the vertical structure features and the spectral features of the space respectively. The connection layer concatenates the spectral features of the long time series, the vertical structure features and the spectral features of the space along the feature dimension. There are multiple fully connected layers after the connection layer. The last fully connected layer uses the softmax activation function. The output layer maps the features to the land cover type. (4) Inputting the training data set and the verification data set into the three-channel convolutional neural network model for training and verification to obtain a trained three-channel convolutional neural network model; (5) Obtain multi-source remote sensing image data from Landsat, Sentinel-1 and Sentinel-2 at the same sample point to be classified in the alpine region, perform preprocessing to obtain Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data, and input the Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data into the trained three-channel convolutional neural network model to predict the artificial forests and complete the classification of artificial forests in the alpine region.

[0008] Optionally, in step (1), sample points are first selected in the alpine region based on Google Earth images, field measurement data, terrain data, and forest inventory data, and the land cover types are marked. The original multi-source remote sensing image data at the sample points are collected using the Google Earth Engine platform, and the multi-source remote sensing image data are preprocessed to obtain Landsat multispectral data, Sentinel-1 polarization band data, and Sentinel-2 multispectral data.

[0009] Optionally, the specific steps for obtaining Landsat multispectral data in step (1) are as follows: collecting Landsat remote sensing image data at sample points during the vegetation growing season each year within a long time series, where the long time series refers to at least 20 consecutive years, and the vegetation growing season refers to June to September; processing the Landsat remote sensing image data to remove clouds, and retaining images with cloud coverage below 20%; selecting the median of three years of images before and after each year as the data for that year, to obtain Landsat multispectral data, where the Landsat multispectral data includes 10 bands, and the resolution of each band is 30 meters.

[0010] Optionally, the specific steps of obtaining the Sentinel-1 polarization band data in step (1) are as follows: using the Google Earth Engine platform to collect Sentinel-1 remote sensing image data at sample points in a vegetation growing season in a certain year in a high-cold area; performing radiation and atmospheric correction on the Sentinel-1 remote sensing image data, taking the median of each pixel in all images, and forming the Sentinel-1 polarization band data for one year. The Sentinel-1 polarization band data includes a VV band and a VH band, where VV refers to single co-polarization, vertical transmission / vertical reception, and VH refers to dual-frequency cross-polarization, vertical transmission / horizontal reception.

[0011] Optionally, the specific steps for obtaining the Sentinel-2 multispectral data in step (1) are as follows: using the Google EarthEngine platform to collect Sentinel-2 remote sensing image data at sample points in a vegetation growing season in a certain year in a high-cold area; performing radiation and atmospheric correction on the Sentinel-2 remote sensing image data, filtering the images to include only images with a turbid pixel percentage of less than 70%; and then taking the median of each pixel in all the images to obtain the Sentinel-2 multispectral data for one year, wherein the Sentinel-2 multispectral data includes 12 bands.

[0012] Optionally, in step (3), the input data of the first input channel is two-dimensional Landsat multispectral data, the dimension of the Landsat multispectral data is M×10, M is the total number of years of Landsat data, and 10 is the number of spectral bands; the input data of the second input channel is three-dimensional Sentinel-1 polarization band data, the dimension of the Sentinel-1 polarization band data is 3×3×2, 3×3 is the spatial size, and 2 is the number of polarization bands; the input data of the third input channel is three-dimensional Sentinel-2 multispectral data, the dimension of the Sentinel-2 multispectral data is 3×3×12, 3×3 is the spatial size, and 12 is the number of spectral bands; the convolution kernel size of the first feature extraction channel is 3, and the step size is 1; the convolution kernel size of the second feature extraction channel is (3,3), and the step size is 1; the convolution kernel size of the third feature extraction channel is (3,3), and the step size is 1.

[0013] Based on the same inventive concept, the present invention discloses a classification system for artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks, comprising: The data acquisition module is used to select sample points in alpine areas and mark land cover types, and obtain the original multi-source remote sensing image data at the sample points. The multi-source remote sensing image data is preprocessed to obtain Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data. The land cover types include artificial forests, natural forests, shrubs, grasslands, cultivated land, bare land and other areas. The data partitioning module is used to form a data set consisting of Landsat multispectral data, Sentinel-1 polarization band data, Sentinel-2 multispectral data and land cover type labels, and divide it into a training data set and a validation data set according to a certain ratio; A model building module is used to build a three-channel convolutional neural network model. The three-channel convolutional neural network model includes an input layer, a convolution layer, a global average pooling layer, a connection layer, a fully connected layer and an output layer. The input layer has three input channels. The input data of the three input channels are Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data respectively. The convolution layer has three feature extraction channels. Each feature extraction channel is connected to a global average pooling layer. The outputs of the three global average pooling layers are spectral features on long time series, vertical structure features and spectral features on space respectively. The connection layer splices the spectral features on long time series, vertical structure features and spectral features on space along the feature dimension. There are multiple fully connected layers after the connection layer. The last fully connected layer uses a softmax activation function. The output layer maps the features to land cover types. A model training module is used to input the training data set and the verification data set into the three-channel convolutional neural network model for training and verification, so as to obtain a trained three-channel convolutional neural network model; The artificial forest prediction module is used to obtain multi-source remote sensing image data of Landsat, Sentinel-1 and Sentinel-2 at the same sample point to be classified in the alpine area, and pre-process them to obtain Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data. The Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data are input into the trained three-channel convolutional neural network model to predict the artificial forest and complete the classification of artificial forests in the alpine area.

[0014] Optionally, in the data acquisition module, sample points are first selected in the alpine region and the land cover types are marked according to Google Earth images, field measurement data, terrain data and forest inventory data, the original multi-source remote sensing image data at the sample points are collected using the Google Earth Engine platform, and the multi-source remote sensing image data are preprocessed to obtain Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data.

[0015] Optionally, the specific steps of obtaining Landsat multispectral data in the data acquisition module are: collecting Landsat remote sensing image data at the sample point in the vegetation growing season of each year in a long time series, the long time series refers to at least 20 consecutive years, and the vegetation growing season refers to June to September; the Landsat remote sensing image data is processed to remove clouds, and images with cloud coverage below 20% are retained; the median of three years of images before and after each year is selected as the data of that year to obtain Landsat multispectral data, the Landsat multispectral data includes 10 bands, and the resolution of each band is 30 meters; The specific steps to obtain the Sentinel-1 polarization band data are as follows: use the Google Earth Engine platform to collect Sentinel-1 remote sensing image data at sample points in the vegetation growing season of a certain year in the alpine region; perform radiation and atmospheric correction on the Sentinel-1 remote sensing image data, take the median of each pixel in all images, and form the Sentinel-1 polarization band data for one year. The Sentinel-1 polarization band data includes VV band and VH band. VV refers to single co-polarization, vertical transmission / vertical reception, and VH refers to dual-frequency cross-polarization, vertical transmission / horizontal reception. The specific steps to obtain Sentinel-2 multispectral data are as follows: use the Google Earth Engine platform to collect Sentinel-2 remote sensing image data at sample points in the vegetation growing season of a certain year in the alpine region; perform radiation and atmospheric correction on the Sentinel-2 remote sensing image data, and filter the images to include only those with a turbid pixel percentage of less than 70%; then take the median of each pixel in all images to obtain one year of Sentinel-2 multispectral data. Sentinel-2 multispectral data includes 12 bands.

[0016] Optionally, the input data of the first input channel in the model building module is two-dimensional Landsat multispectral data, the dimension of the Landsat multispectral data is M×10, M is the total number of years of Landsat data, and 10 is the number of spectral bands; the input data of the second input channel is three-dimensional Sentinel-1 polarization band data, the dimension of the Sentinel-1 polarization band data is 3×3×2, 3×3 is the spatial size, and 2 is the number of polarization bands; the input data of the third input channel is three-dimensional Sentinel-2 multispectral data, the dimension of the Sentinel-2 multispectral data is 3×3×12, 3×3 is the spatial size, and 12 is the number of spectral bands; the convolution kernel size of the first feature extraction channel is 3, and the step size is 1; the convolution kernel size of the second feature extraction channel is (3,3), and the step size is 1; the convolution kernel size of the third feature extraction channel is (3,3), and the step size is 1.

[0017] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the present invention can extract and fuse the vertical structural features, long-term spectral features and spatial spectral features of the data, amplify the differences in time series data between artificial forests and natural forests, and can be widely used in the precise monitoring of artificial forests in high-cold areas; compared with traditional manual field surveys, the present invention greatly reduces the time, manpower and other costs and improves efficiency; compared with traditional land object classification methods, the present invention does not require prior feature extraction and screening, makes full use of the rich information of spectral images and saves the trouble of artificial pre-setting features, has higher classification accuracy, reduces mixed classification phenomenon, and has more accurate classification results; the three-channel convolutional neural network proposed in the present invention is used to complete the classification of artificial forests, which can not only achieve higher accuracy, but also improve the speed of training and prediction. The classification accuracy of artificial forests in the Qinghai-Tibet Plateau is 93%, which has practical value; the present invention can realize the classification of artificial forests in high-cold areas, is portable, and fills the current technical gap related to high-precision classification of artificial forests in high-cold areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 It is a comparison diagram of the effects of the present invention; Figure 3 It is a system schematic diagram of the present invention. DETAILED DESCRIPTION

[0019] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0020] Example 1: Figure 1 As shown, this embodiment discloses a method for classifying artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks, comprising the following steps: (1) Sample points are selected on the Qinghai-Tibet Plateau and the land cover types are marked. The original multi-source remote sensing image data at the sample points are obtained at the same time. The multi-source remote sensing image data are preprocessed to obtain Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data. The land cover types include artificial forests, natural forests, shrubs, grasslands, cultivated land, bare land and other areas (impermeable areas). The present invention combines ground survey data, uses the Google Earth Engine platform and visual interpretation to mark the land cover types in the study area; Firstly, we selected sample points on the Qinghai-Tibet Plateau and marked the land cover types based on Google Earth images, field measurement data, terrain data and forest inventory data. We used the Google Earth Engine platform to collect original multi-source remote sensing image data at the sample points. We then preprocessed the multi-source remote sensing image data to obtain Landsat multispectral data, Sentinel-1 polarimetric band data and Sentinel-2 multispectral data. The specific steps of obtaining Landsat multispectral data are as follows: collecting Landsat remote sensing image data at sample points in the vegetation growing season every year in a long time series, the long time series refers to at least 20 consecutive years, preferably 32 years, and the vegetation growing season refers to June to September; Landsat is a series of land satellites, and Landsat 5 images are used from 1991 to 2000, Landsat 7 images are used from 2001 to 2014, and Landsat 8 images are used from 2015 to 2022; Landsat remote sensing image data are processed to remove clouds, and images with cloud coverage less than 20% are retained; the median of three years of images before and after each year is selected as the data of that year to obtain Landsat multispectral data, and Landsat multispectral data includes 10 bands: Blue, Green, Red, NIR (near-infrared), SIR1 (shortwave infrared 1), SIR2 (shortwave infrared 2) NDVI, TCB, TCG and TCW, all with a resolution of 30 meters; Normalized Difference Vegetation Index NDVI is an evaluation index of vegetation coverage. The calculation formula of NDVI is:

[0021] TCB, TCG and TCW are three typical bands of the tasseled cap transformation, representing "brightness", "greenness" and "wetness" respectively; the bands after the tasseled cap transformation can be used to identify and monitor specific land features. Brightness is closely related to land cover type and land use, greenness is used to evaluate vegetation coverage, and wetness reflects soil moisture or vegetation water content factors; The specific steps to obtain Sentinel-1 polarimetric band data are: The Google Earth Engine platform is used to collect Sentinel-1 remote sensing image data at sample points on the Qinghai-Tibet Plateau from June 1, 2022 to September 30, 2022; the Sentinel-1 remote sensing image data is corrected for radiation and atmosphere, and the median of each pixel in all images is taken to form one year of Sentinel-1 polarization band data. The Sentinel-1 polarization band data includes VV band and VH band. VV refers to single co-polarization, vertical transmission / vertical reception, and VH refers to dual-frequency cross-polarization, vertical transmission / horizontal reception; The specific steps to obtain Sentinel-2 multispectral data are: The Google Earth Engine platform was used to collect Sentinel-2 remote sensing image data at the sample points over the Qinghai-Tibet Plateau from June 1, 2022 to September 30, 2022. The Sentinel-2 remote sensing image data were radiometrically and atmospherically corrected, and the images were filtered to include only images with a turbid pixel percentage of less than 70%. The median of each pixel in all images was then taken to obtain one year of Sentinel-2 multispectral data, which includes 12 bands: aerosols (B1), blue (B2), green (B3), red (B4), red edge 1-4 (B5-B7, B8A), infrared (B8), watervapor (B9), and short-wave infrared (B11, B12). (2) The Landsat multispectral data, Sentinel-1 polarization band data, Sentinel-2 multispectral data, and land cover type labels are used to form a data set, which is then divided into a training data set and a validation data set according to a certain ratio; The specific steps are as follows: in the obtained Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data, zero values ​​are deleted and the data format is converted, Landsat multispectral data, Sentinel-1 polarization band data, Sentinel-2 multispectral data and land cover type labels form a data set, and are divided into a training data set and a validation data set according to a certain ratio; (3) Construct a three-channel convolutional neural network model, which includes an input layer, a convolution layer, a global average pooling layer, a connection layer, a fully connected layer, and an output layer. The convolutional neural network model consists of multiple layers, including convolutional layers, pooling layers, and fully connected layers; the convolutional layers extract key features from the image through convolution operations, the pooling layers are used for dimensionality reduction, and the fully connected layers are responsible for classification tasks. The training process of the convolutional neural model uses the back-propagation algorithm for error propagation, while using gradient descent to minimize the specified loss function to optimize the model weights and biases. The present invention designs a three-channel convolutional neural network model to effectively combine the spatiotemporal dynamics in long-term Landsat multispectral data (one dimension: time dimension), Sentinel-2 multispectral data (two dimensions: spectral and spatial dimensions), and Sentinel-1 polarimetric band data (two dimensions: polarimetric band and spatial dimensions).

[0022] Input layer: The input layer has three input channels. The input data of the first input channel is two-dimensional Landsat multispectral data. The dimension of Landsat multispectral data is 32×10, where 32 is the total number of years of data and 10 is the number of spectral bands. The input data of the second input channel is three-dimensional Sentinel-1 polarization band data. The dimension of Sentinel-1 polarization band data is 3×3×2, where 3×3 is the spatial size and 2 is the number of polarization bands. The input data of the third input channel is three-dimensional Sentinel-2 multispectral data. The dimension of Sentinel-2 multispectral data is 3×3×12, where 3×3 is the spatial size and 12 is the number of spectral bands. Convolution layer: The convolution layer has three feature extraction channels. The number of convolution kernels of the first feature extraction channel is 32, the convolution kernel size is 3, the step size is 1, the number of convolution kernels of the first and second layers of the first feature extraction channel is 32, and the number of convolution kernels of the third layer of the first feature extraction channel is 64; the convolution kernel size of the second feature extraction channel is (3,3), the step size is 1, the number of convolution kernels of the first and second layers of the second feature extraction channel is 32, and the number of convolution kernels of the third layer of the second feature extraction channel is 64; the convolution kernel size of the third feature extraction channel is (3,3), the step size is 1, the number of convolution kernels of the first and second layers of the third feature extraction channel is 32, and the number of convolution kernels of the third layer of the third feature extraction channel is 64; The three feature extraction channels all adopt a design with a convolution kernel size of 3 and a step size of 1, aiming to achieve fine feature extraction through appropriate parameter selection and avoid losing important information. The Landsat multispectral data is input with one-dimensional convolution, and the Sentinel-2 multispectral data and Sentinel-1 polarization band data are input with two-dimensional convolution respectively. For each input data, the three-channel convolutional neural network model uses multiple convolutional layers and maximum pooling layers to extract features. The edge padding method in each convolution operation is the same, that is, the size of the output feature map is the same as the input. The activation function used after each convolution layer is ReLU. The specific parameters of the model convolution layer are shown in Table 1.

[0023] Table 1 Model convolution layer parameters

[0024] Global Average Pooling Layer: When fusing spatial polarimetric information with multispectral time series and spatial spectral information, the number of features obtained from these three data sources is uneven, and a simple superposition method that does not consider the balance between them may lead to inefficiency. Therefore, a global average pooling layer (GAP) is used at the end of each feature advance channel to reduce the feature dimension, which can reduce the number of parameters, reduce the complexity of the model, prevent overfitting, and make the model insensitive to the spatial position of the input, thereby enhancing the generalization ability of the model. Landsat multispectral data is input into the first input channel, corresponding to the spectral features in the output time series, Sentinel-1 polarization band data is input into the second input channel, corresponding to the vertical structure features, and Sentinel-2 multispectral data is input into the third input channel, corresponding to the spectral features in the output space; Connection layer: The connection layer combines the temporal spectral features, vertical structural features, and spatial spectral features along the feature dimension; Fully connected layer: Use multiple fully connected layers and ReLU activation functions, use Dropout to reduce overfitting, and randomly set 20% of the neuron outputs to 0 during each training to force the network to learn more robust features. Repeat the fully connected layer and Dropout mode 4 times, adding 120 neurons each time, and gradually increase the Dropout ratio to 0.2, 0.3, 0.4, and 0.5, respectively, which helps the network learn more complex features and reduces the risk of overfitting through Dropout. Output layer: The last fully connected layer uses a softmax activation function to map features to land cover types; (4) The training data set is input into the three-channel convolutional neural network model for training, and the overall accuracy OA is set as the evaluation index. The Adam optimizer is used to optimize the model parameters, the learning rate is 0.0001, 128 samples are used for training in each iteration, and the batch size is 128; at the end of each iteration, the verification data set is input into the three-channel convolutional neural network model for prediction. The three-channel convolutional neural network model will output the corresponding loss function value and overall accuracy. The iteration is stopped when the loss function value drops to a stable level. The number of iterations is 1200, and the trained three-channel convolutional neural network model is obtained; the loss function is the focal loss function, the focal factor gamma is 2, and the balance factor alpha is 0.25; (5) The trained three-channel convolutional neural network model was applied to predict the 30-meter resolution artificial forest map in the eastern Qinghai-Tibet Plateau. Multi-source remote sensing image data of Landsat, Sentinel-1 and Sentinel-2 were obtained at the same sample point to be classified on the Qinghai-Tibet Plateau. The Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data were preprocessed and input into the trained three-channel convolutional neural network model to predict the artificial forests and complete the classification mapping of artificial forests on the Qinghai-Tibet Plateau.

[0025] In order to verify the performance of the three-channel convolutional neural network model, this embodiment conducts ablation experiments and uncertainty analysis. The performance of the three-channel convolutional neural network model is evaluated, that is, the confusion matrix, overall accuracy (OA), Kappa coefficient, precision (Precision), recall rate (Recall), F1 score, false positive error (CE), missed positive error (OE), user accuracy (UA), producer accuracy (PA) of the model are calculated, and the three-channel convolutional neural network model is compared with the single-channel and dual-channel input scenarios. For all single-channel and dual-channel input scenarios, except for the reduction in the number of input channels, the model structure is the same as the three-channel convolutional neural network model. The experimental results are as follows: Figure 2 shown.

[0026] The results show that the three-channel convolutional neural network model shows excellent performance in predicting plantations, with Precision = 0.93, Recall = 0.90, and F1 = 0.92. By utilizing the complementary information from each data source, the three-channel convolutional neural network model captures more comprehensive vegetation features, thereby improving the accuracy of drawing plantation maps, verifying the effectiveness of the present invention.

[0027] The present invention uses a three-channel convolutional neural network model to extract time series, spatial texture, spectrum and vertical structure feature information from remote sensing images, realizes multi-source remote sensing data fusion, and obtains a high-resolution and relatively accurate distribution map of artificial forests in high-cold areas. The artificial forest classification of the present invention has high accuracy and small error, providing a scientific basis for clarifying the effectiveness of afforestation projects and evaluating where afforestation can be implemented.

[0028] Example 2: Figure 3 As shown, this embodiment discloses a system for classifying artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks, including: The data acquisition module is used to select sample points on the Qinghai-Tibet Plateau and mark the land cover types, and simultaneously obtain the original multi-source remote sensing image data at the sample points, and pre-process the multi-source remote sensing image data to obtain Landsat multi-spectral data, Sentinel-1 polarization band data, and Sentinel-2 multi-spectral data. The land cover types include artificial forests, natural forests, shrubs, grasslands, cultivated land, bare land, and other areas (impermeable areas). The present invention combines ground survey data, uses the Google Earth Engine platform and visual interpretation, and marks the land cover types in the study area; Firstly, we selected sample points on the Qinghai-Tibet Plateau and marked the land cover types based on Google Earth images, field measurement data, terrain data and forest inventory data. We used the Google Earth Engine platform to collect original multi-source remote sensing image data at the sample points. We then preprocessed the multi-source remote sensing image data to obtain Landsat multispectral data, Sentinel-1 polarimetric band data and Sentinel-2 multispectral data. The specific steps of obtaining Landsat multispectral data are as follows: collecting Landsat remote sensing image data at sample points in the vegetation growing season every year in a long time series, the long time series refers to at least 20 consecutive years, preferably 32 years, and the vegetation growing season refers to June to September; Landsat is a series of land satellites, Landsat 5 images are used from 1991 to 2000, Landsat 7 images are used from 2001 to 2014, and Landsat 8 images are used from 2015 to 2022; Landsat remote sensing image data are processed to remove clouds, and images with cloud coverage less than 20% are retained; the median of three years of images before and after each year is selected as the data of that year to obtain Landsat multispectral data, and Landsat multispectral data includes 10 bands: Blue, Green, Red, NIR (near-infrared), SIR1 (shortwave infrared 1), SIR2 (shortwave infrared 2) NDVI, TCB, TCG and TCW, all with a resolution of 30 meters; Normalized Difference Vegetation Index NDVI is an evaluation index of vegetation coverage. The calculation formula of NDVI is:

[0029] TCB, TCG and TCW are three typical bands of the tasseled cap transformation, representing "brightness", "greenness" and "wetness" respectively; the bands after the tasseled cap transformation can be used to identify and monitor specific land features. Brightness is closely related to land cover type and land use, greenness is used to evaluate vegetation coverage, and wetness reflects soil moisture or vegetation water content factors; The specific steps to obtain Sentinel-1 polarimetric band data are: The Google Earth Engine platform is used to collect Sentinel-1 remote sensing image data at sample points on the Qinghai-Tibet Plateau from June 1, 2022 to September 30, 2022; the Sentinel-1 remote sensing image data is corrected for radiation and atmosphere, and the median of each pixel in all images is taken to form one year of Sentinel-1 polarization band data. The Sentinel-1 polarization band data includes VV band and VH band. VV refers to single co-polarization, vertical transmission / vertical reception, and VH refers to dual-frequency cross-polarization, vertical transmission / horizontal reception; The specific steps to obtain Sentinel-2 multispectral data are: The Google Earth Engine platform was used to collect Sentinel-2 remote sensing image data at the sample points over the Qinghai-Tibet Plateau from June 1, 2022 to September 30, 2022. The Sentinel-2 remote sensing image data were radiometrically and atmospherically corrected, and the images were filtered to include only images with a turbid pixel percentage of less than 70%. The median of each pixel in all images was then taken to obtain one year of Sentinel-2 multispectral data. The Sentinel-2 multispectral data includes 12 bands: aerosols (B1), blue (B2), green (B3), red (B4), red edge 1-4 (B5-B7, B8A), infrared (B8), watervapor (B9), and short-wave infrared (B11, B12). The data partitioning module is used to form a data set consisting of Landsat multispectral data, Sentinel-1 polarization band data, Sentinel-2 multispectral data and land cover type labels, and divide it into a training data set and a validation data set according to a certain ratio; The specific steps are as follows: in the obtained Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data, zero values ​​are deleted and the data format is converted, Landsat multispectral data, Sentinel-1 polarization band data, Sentinel-2 multispectral data and land cover type labels form a data set, and are divided into a training data set and a validation data set according to a certain ratio; The model building module is used to build a three-channel convolutional neural network model. The three-channel convolutional neural network model includes an input layer, a convolution layer, a global average pooling layer, a connection layer, a fully connected layer, and an output layer. The convolutional neural network model consists of multiple layers, including convolutional layers, pooling layers, and fully connected layers; the convolutional layers extract key features from the image through convolution operations, the pooling layers are used for dimensionality reduction, and the fully connected layers are responsible for classification tasks. The training process of the convolutional neural model uses the back propagation algorithm for error propagation, and uses gradient descent to minimize the specified loss function to optimize the model weights and biases. The present invention designs a three-channel convolutional neural network model to effectively combine the spatiotemporal dynamics in long-term Landsat multispectral data (one dimension: time dimension), Sentinel-2 multispectral data (two dimensions: spectral and spatial dimensions), and Sentinel-1 polarimetric band data (two dimensions: polarimetric band and spatial dimensions).

[0030] Input layer: The input layer has three input channels. The input data of the first input channel is two-dimensional Landsat multispectral data. The dimension of Landsat multispectral data is 32×10, where 32 is the total number of years of data and 10 is the number of spectral bands. The input data of the second input channel is three-dimensional Sentinel-1 polarization band data. The dimension of Sentinel-1 polarization band data is 3×3×2, where 3×3 is the spatial size and 2 is the number of polarization bands. The input data of the third input channel is three-dimensional Sentinel-2 multispectral data. The dimension of Sentinel-2 multispectral data is 3×3×12, where 3×3 is the spatial size and 12 is the number of spectral bands. Convolution layer: The convolution layer has three feature extraction channels. The number of convolution kernels of the first feature extraction channel is 32, the convolution kernel size is 3, the step size is 1, the number of convolution kernels of the first and second layers of the first feature extraction channel is 32, and the number of convolution kernels of the third layer of the first feature extraction channel is 64; the convolution kernel size of the second feature extraction channel is (3,3), the step size is 1, the number of convolution kernels of the first and second layers of the second feature extraction channel is 32, and the number of convolution kernels of the third layer of the second feature extraction channel is 64; the convolution kernel size of the third feature extraction channel is (3,3), the step size is 1, the number of convolution kernels of the first and second layers of the third feature extraction channel is 32, and the number of convolution kernels of the third layer of the third feature extraction channel is 64; The three feature extraction channels all use a convolution kernel size of 3 and a step size of 1, aiming to achieve fine feature extraction through appropriate parameter selection and avoid losing important information. The Landsat multispectral data is input using one-dimensional convolution, and the Sentinel-2 multispectral data and Sentinel-1 polarization band data are input using two-dimensional convolution respectively. For each type of input data, the three-channel convolutional neural network model uses multiple convolutional layers and maximum pooling layers to extract features. The edge padding method in each convolution operation is the same, that is, the size of the output feature map is the same as the input. The activation function used after each convolution layer is ReLU.

[0031] Global Average Pooling Layer: When fusing spatial polarimetric information with multispectral time series and spatial spectral information, the number of features obtained from these three data sources is uneven, and a simple superposition method that does not consider the balance between them may lead to inefficiency. Therefore, a global average pooling layer (GAP) is used at the end of each feature advance channel to reduce the feature dimension, which can reduce the number of parameters, reduce the complexity of the model, prevent overfitting, and make the model insensitive to the spatial position of the input, thereby enhancing the generalization ability of the model. Landsat multispectral data is input into the first input channel, corresponding to the spectral features in the output time series, Sentinel-1 polarization band data is input into the second input channel, corresponding to the vertical structure features, and Sentinel-2 multispectral data is input into the third input channel, corresponding to the spectral features in the output space; Connection layer: The connection layer combines the temporal spectral features, vertical structural features, and spatial spectral features along the feature dimension; Fully connected layer: Use multiple fully connected layers and ReLU activation functions, use Dropout to reduce overfitting, and randomly set 20% of the neuron outputs to 0 during each training to force the network to learn more robust features. Repeat the fully connected layer and Dropout mode 4 times, adding 120 neurons each time, and gradually increase the Dropout ratio to 0.2, 0.3, 0.4, and 0.5, respectively, which helps the network learn more complex features and reduces the risk of overfitting through Dropout. Output layer: The last fully connected layer uses a softmax activation function to map features to land cover types; The model training module is used to input the training data set into the three-channel convolutional neural network model for training, set the overall accuracy OA as the evaluation index, use the Adam optimizer to optimize the model parameters, the learning rate is 0.0001, and 128 samples are used for training in each iteration, and the batch size is 128; at the end of each iteration, the verification data set is input into the three-channel convolutional neural network model for prediction. The three-channel convolutional neural network model will output the corresponding loss function value and overall accuracy. When the loss function value drops to a stable level, the iteration is stopped. The number of iterations is 1200, and a trained three-channel convolutional neural network model is obtained; the loss function is the Focal Loss loss function, the focus factor gamma is 2, and the balance factor alpha is 0.25; The artificial forest prediction module is used to apply the trained three-channel convolutional neural network model to predict the artificial forest map with a resolution of 30 meters in the eastern part of the Qinghai-Tibet Plateau, obtain the multi-source remote sensing image data of Landsat, Sentinel-1 and Sentinel-2 at the same sample point to be classified on the Qinghai-Tibet Plateau, pre-process the Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data, input the Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data into the trained three-channel convolutional neural network model, predict the artificial forest, and complete the classification mapping of the artificial forest on the Qinghai-Tibet Plateau.

[0032] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for classifying artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks, characterized in that: The following steps are involved: (1) Sample points were selected in the alpine areas and the land cover types were marked. Multi-source remote sensing image data were obtained at the sample points. The multi-source remote sensing image data were preprocessed to obtain Landsat multispectral data, Sentinel-1 polarimetric band data, and Sentinel-2 multispectral data. The land cover types included artificial forests, natural forests, shrubs, grasslands, cultivated land, bare land, and other areas. (2) The Landsat multispectral data, Sentinel-1 polarization band data, Sentinel-2 multispectral data, and land cover type labels are used to form a data set, which is then divided into a training data set and a validation data set according to a certain ratio; (3) Construct a three-channel convolutional neural network model, which includes an input layer, a convolution layer, a global average pooling layer, a connection layer, a fully connected layer and an output layer. The input layer has three input channels, and the input data of the three input channels are Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data respectively. The convolution layer has three feature extraction channels, and each feature extraction channel is connected to a global average pooling layer. The outputs of the three global average pooling layers are the spectral features of the long time series, the vertical structure features and the spectral features of the space respectively. The connection layer concatenates the spectral features of the long time series, the vertical structure features and the spectral features of the space along the feature dimension. There are multiple fully connected layers after the connection layer. The last fully connected layer uses the softmax activation function. The output layer maps the features to the land cover type. (4) Inputting the training data set and the verification data set into the three-channel convolutional neural network model for training and verification to obtain a trained three-channel convolutional neural network model; (5) Obtain multi-source remote sensing image data from Landsat, Sentinel-1 and Sentinel-2 at the same sample point to be classified in the alpine region, perform preprocessing to obtain Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data, and input the Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data into the trained three-channel convolutional neural network model to predict the artificial forests and complete the classification of artificial forests in the alpine region.

2. The method for classifying artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks according to claim 1 is characterized in that: In the step (1), sample points are first selected in the alpine region according to Google Earth images, field measurement data, terrain data and forest inventory data, and the land cover types are marked. The original multi-source remote sensing image data at the sample points are collected using the Google Earth Engine platform, and the multi-source remote sensing image data are preprocessed to obtain Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data.

3. The method for classifying artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks according to claim 2 is characterized in that: The specific steps of obtaining the Landsat multispectral data in step (1) are as follows: collecting Landsat remote sensing image data at sample points during the vegetation growing season every year within a long time series, where the long time series refers to at least 20 consecutive years, and the vegetation growing season refers to June to September; processing the Landsat remote sensing image data to remove clouds, and retaining images with a cloud coverage rate of less than 20%; selecting the median of the three years before and after each year as the data for that year, to obtain the Landsat multispectral data, where the Landsat multispectral data includes 10 bands, and the resolution of each band is 30 meters.

4. The method for classifying artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks according to claim 2 is characterized in that: The specific steps of obtaining the Sentinel-1 polarization band data in step (1) are as follows: using the Google Earth Engine platform to collect Sentinel-1 remote sensing image data at sample points in a vegetation growing season in a certain year in a high-cold area; performing radiation and atmospheric correction on the Sentinel-1 remote sensing image data, taking the median of each pixel in all images, and forming Sentinel-1 polarization band data for one year. The Sentinel-1 polarization band data includes a VV band and a VH band, where VV refers to single co-polarization, vertical transmission / vertical reception, and VH refers to dual-frequency cross-polarization, vertical transmission / horizontal reception.

5. The method for classifying artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks according to claim 2 is characterized in that: The specific steps of obtaining the Sentinel-2 multispectral data in step (1) are as follows: using the Google Earth Engine platform to collect Sentinel-2 remote sensing image data at sample points in a vegetation growing season in a certain year in a high-cold area; performing radiation and atmospheric correction on the Sentinel-2 remote sensing image data, filtering the images to include only images with a turbid pixel percentage of less than 70%; and then taking the median of each pixel in all the images to obtain the Sentinel-2 multispectral data for one year, wherein the Sentinel-2 multispectral data includes 12 bands.

6. The method for classifying artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks according to claim 1 is characterized in that: In the step (3), the input data of the first input channel is two-dimensional Landsat multispectral data, the dimension of the Landsat multispectral data is M×10, M is the total number of years of Landsat data, and 10 is the number of spectral bands; the input data of the second input channel is three-dimensional Sentinel-1 polarization band data, the dimension of the Sentinel-1 polarization band data is 3×3×2, 3×3 is the spatial size, and 2 is the number of polarization bands; the input data of the third input channel is three-dimensional Sentinel-2 multispectral data, the dimension of the Sentinel-2 multispectral data is 3×3×12, 3×3 is the spatial size, and 12 is the number of spectral bands; The convolution kernel size of the first feature extraction channel is 3 and the stride is 1; The convolution kernel size of the second feature extraction channel is (3,3) and the stride is 1; The convolution kernel size of the third feature extraction channel is (3,3) and the stride is 1.

7. A classification system for artificial forests in high-cold areas based on multi-source remote sensing data and convolutional neural networks, characterized in that: include: The data acquisition module is used to select sample points in alpine areas and mark land cover types, and obtain the original multi-source remote sensing image data at the sample points. The multi-source remote sensing image data is preprocessed to obtain Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data. The land cover types include artificial forests, natural forests, shrubs, grasslands, cultivated land, bare land and other areas. The data partitioning module is used to form a data set consisting of Landsat multispectral data, Sentinel-1 polarization band data, Sentinel-2 multispectral data and land cover type labels, and divide it into a training data set and a validation data set according to a certain ratio; A model building module is used to build a three-channel convolutional neural network model. The three-channel convolutional neural network model includes an input layer, a convolution layer, a global average pooling layer, a connection layer, a fully connected layer and an output layer. The input layer has three input channels. The input data of the three input channels are Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data respectively. The convolution layer has three feature extraction channels. Each feature extraction channel is connected to a global average pooling layer. The outputs of the three global average pooling layers are spectral features on long time series, vertical structure features and spectral features on space respectively. The connection layer splices the spectral features on long time series, vertical structure features and spectral features on space along the feature dimension. There are multiple fully connected layers after the connection layer. The last fully connected layer uses a softmax activation function. The output layer maps the features to land cover types. A model training module is used to input the training data set and the verification data set into the three-channel convolutional neural network model for training and verification, so as to obtain a trained three-channel convolutional neural network model; The artificial forest prediction module is used to obtain multi-source remote sensing image data of Landsat, Sentinel-1 and Sentinel-2 at the same sample point to be classified in the alpine area, and pre-process them to obtain Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data. The Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data are input into the trained three-channel convolutional neural network model to predict the artificial forest and complete the classification of artificial forests in the alpine area.

8. The alpine forest plantation classification system based on multi-source remote sensing data and convolutional neural network according to claim 1 is characterized in that: In the data acquisition module, sample points are first selected in the alpine area and the land cover types are marked according to Google Earth images, field measurement data, terrain data and forest inventory data. The original multi-source remote sensing image data at the sample points are collected using the Google Earth Engine platform. The multi-source remote sensing image data are preprocessed to obtain Landsat multispectral data, Sentinel-1 polarization band data and Sentinel-2 multispectral data.

9. The alpine forest plantation classification system based on multi-source remote sensing data and convolutional neural network according to claim 8, characterized in that: The specific steps of obtaining Landsat multispectral data in the data acquisition module are as follows: collecting Landsat remote sensing image data at the sample point in the vegetation growth season of each year in a long time series, the long time series refers to at least 20 consecutive years, and the vegetation growth season refers to June to September; the Landsat remote sensing image data is processed to remove clouds, and images with cloud coverage below 20% are retained; the median of three years of images before and after each year is selected as the data of that year to obtain Landsat multispectral data, and the Landsat multispectral data includes 10 bands, and the resolution of each band is 30 meters; The specific steps to obtain the Sentinel-1 polarization band data are as follows: use the Google Earth Engine platform to collect Sentinel-1 remote sensing image data at sample points in the vegetation growing season of a certain year in the alpine region; perform radiation and atmospheric correction on the Sentinel-1 remote sensing image data, take the median of each pixel in all images, and form the Sentinel-1 polarization band data for one year. The Sentinel-1 polarization band data includes VV band and VH band. VV refers to single co-polarization, vertical transmission / vertical reception, and VH refers to dual-frequency cross-polarization, vertical transmission / horizontal reception. The specific steps to obtain Sentinel-2 multispectral data are as follows: use the Google Earth Engine platform to collect Sentinel-2 remote sensing image data at sample points in the vegetation growing season of a certain year in the alpine region; perform radiation and atmospheric correction on the Sentinel-2 remote sensing image data, and filter the images to include only those with a turbid pixel percentage of less than 70%; then take the median of each pixel in all images to obtain one year of Sentinel-2 multispectral data. Sentinel-2 multispectral data includes 12 bands.

10. The alpine forest plantation classification system based on multi-source remote sensing data and convolutional neural network according to claim 7, characterized in that: The input data of the first input channel in the model building module is two-dimensional Landsat multispectral data, the dimension of Landsat multispectral data is M×10, M is the total number of years of Landsat data, and 10 is the number of spectral bands; the input data of the second input channel is three-dimensional Sentinel-1 polarization band data, the dimension of Sentinel-1 polarization band data is 3×3×2, 3×3 is the spatial size, and 2 is the number of polarization bands; the input data of the third input channel is three-dimensional Sentinel-2 multispectral data, the dimension of Sentinel-2 multispectral data is 3×3×12, 3×3 is the spatial size, and 12 is the number of spectral bands; The convolution kernel size of the first feature extraction channel is 3 and the stride is 1; The convolution kernel size of the second feature extraction channel is (3,3) and the stride is 1; The convolution kernel size of the third feature extraction channel is (3,3) and the stride is 1.

Images