Hyperspectral image dominant tree species identification method based on deep learning

Abstract

The invention provides a hyperspectral image dominant tree species identification method based on deep learning, and the method comprises the following steps: 1), obtaining data: obtaining the hyperspectral image data of trees in a forest region through a hyperspectral image sensor carried by an unmanned plane; 2) data preprocessing: carrying out preprocessing operations such as geometric correction and atmospheric correction on the obtained hyperspectral image data, and carrying out tree feature extraction on the denoised data; and 3) data set construction: calibrating the extracted features to make a tree species identification data set, and increasing the number of training samples by adopting a data amplification method. And 4) training the model: training the training set by using the built DeepLab v3 + model to obtain a trained model. And 5) model testing: detecting and identifying the test set picture by using the generated model.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning, image recognition and hyperspectral remote sensing technology, and specifically to a hyperspectral image dominant tree species recognition method based on deep learning.

[0002] Forests are important terrestrial ecosystems and play an important role in maintaining ecological balance and ecological security. Traditional forest resource surveys are time-consuming and labor-intensive. The development of remote sensing technology provides technical support for forest resource surveys, forest tree species identification and classification, and forest health monitoring. The effective classification of forest tree species is of great significance to the study of forest species diversity, forest biomass, and forest carbon storage. Since hyperspectral remote sensing image data contains a large amount of spectral information and image information, it has been widely used in the identification and classification of tree species in different forest stands such as artificial forests and natural forests. However, in the past, machine learning algorithms were used to select a certain number of features from hyperspectral data for classification, which was time-consuming and had low accuracy. With the development of deep learning technology, many scholars have used deep learning networks to pre-train hyperspectral data, extract features, and optimize models to achieve rapid and accurate classification of tree species from hyperspectral image data. This paper takes the natural secondary coniferous mixed forest in Maoershan area of ​​Harbin City, Heilongjiang Province as the research object, and uses drone hyperspectral data combined with deep learning network models to classify dominant tree species. The research results can provide a reference for the accurate inversion of forest tree species diversity and forest biomass.

[0003] When identifying tree species in forest hyperspectral images obtained by platforms such as drones, it is difficult to ensure efficiency by relying solely on human eyes, and it is impossible to return specific tree distribution information in a timely manner when there are no location markers or poor vision. Therefore, a forest tree species identification method based on high-resolution remote sensing images is urgently needed to solve the above problems. Summary of the invention

[0004] The present invention provides a method for identifying dominant tree species in hyperspectral images based on deep learning. The method can quickly and accurately identify some tree species in a research plot, and can determine the dominant tree species in the research area through result analysis.

[0005] To achieve the above object, the present invention provides the following technical solution: a method for identifying dominant tree species in hyperspectral images based on deep learning, comprising the following steps:

[0006] Under good weather conditions, the hyperspectral imaging sensor was used to collect data over the study area on a DJI M300RTK drone. The hyperspectral imaging sensor was Pika L. The hyperspectral spatial resolution was 10 cm, the wavelength range was 400-1000 nm, the spectral resolution was 2.1 nm, and the pixel size was 5.86 um. The original data was cropped using the Envi platform, and the image size and band number were 824*1753*300.

[0007] Preprocess the acquired hyperspectral images. Hyperspectral images have rich spectral and spatial information. In addition to tree species information, they are also affected by clouds, radiation, etc. Direct calibration training will affect the recognition accuracy of the model. Therefore, the original data is first preprocessed by radiation calibration, geometric correction, atmospheric correction, etc., and a low-pass filtering algorithm is used for noise reduction.

[0008] The CARS algorithm was used to select the optimal variable subset, the number of cross-validation folds was 20, the number of Monte Carlo sampling was set to 50, and the standard for selecting the best variables was that the sample ratio of the training set and the test set selected for each run of the program was 4:1, and 78 characteristic wavelengths were screened out. Then the data after the initial selection of the band was PCA transformed, and the first three PCA components after PCA transformation were selected as the experimental data of this experiment. The cumulative variance contribution rate of the first three PCA transformation components was 99.84%. A shape file was created based on the ArcGIS platform, and a png format classification label was created based on the shape file. The label marked a total of six types of tree species, which were marked with seven different pixel values, including six types of tree species and one type of background. Finally, the training data was expanded to 10,000 training samples through the image amplification method, and a training model was built based on the DeepLab v3+ algorithm, and the data set was imported for training;

[0009] Define and initialize hyperparameters such as learning rate α, number of iterations epoch, batch size batch_size, etc. Initializing the learning rate is an important hyperparameter in deep learning, which determines whether the objective function can converge to the local minimum and when it converges to the minimum. If it is too large, it is easy to miss the optimal value, and if it is too small, it may obtain a local optimal solution and fail to reach the optimal value. Therefore, the learning rate is first set to 0.001 and set to dynamically decrease so that the model can converge better when it reaches the optimal point. The number of iterations is first set to 60, so that the overall trend of loss decreases and gradually floats at a lower value. Limited by the hardware environment, batch_size is set to 8, and the model starts training;

[0010] During the model training process, the DeepLab v3+ algorithm resizes the input image to 256*256 pixels while ensuring the aspect ratio of the input image. The feature layer is obtained through feature extraction using the backbone network Xception. The Depthwise separable convolution structure in Xception couples the correlation between channels in the convolution layer with the spatial correlation, performs 3*3 convolution on each group of inputs, and concatenates the results. This greatly reduces the training parameters while retaining the number of channels. In addition, bilinear sampling can resize the final classification result to the original size.

[0011] Furthermore, during the model training process, the classification accuracy and location accuracy are usually contradictory. In the model, conditional random fields (CRFs) are used to deal with the problem of non-smoothness in segmentation. Its energy function is The goal is to minimize the energy function, where θ i (x i )=-logP(x i ) This item ensures the accuracy of classification, the accuracy P(x i ) is closer to 1, the smaller the value of this item is, and Here k m (f i ,f j ) is a Gaussian function, which is mainly determined by the position and color of two pixels. The detail information can be enhanced through multi-layer fusion of CRFs;

[0012] During the training process, tfrecord is read to obtain training data and set model hyperparameters: training type, data category, pre-training weight, data weight ratio and visualization parameter setting, etc. At the same time, the loss function is calculated using the cross entropy loss, and the parameters of the entire network are updated to obtain the network model.

[0013] Model evaluation,After the training is completed, the test set images are loaded to evaluate the model, and a curve is drawn for the classification accuracy and loss value changes during the training process, and the model classification accuracy is obtained to be 96.35%.

[0014] Model application, load the trained DeepLab v3+ network model, select the model file saved in the last step, create a new demo.py file, give the test path and model path, set the loaded test image path and name, and perform recognition test;

[0015] Furthermore, the DeepLab v3+ network model described in the present invention is mainly divided into two parts;

[0016] Encoder, first use dilated convolution to prevent the resolution from being too low, then connect dilated convolutions with different dilation rates in series or in parallel, and use a global average pooling to obtain more upper and lower structural information. Finally, continuous downsampling can ensure that the model obtains a wider range of features. Dilated convolutions with different dilation rates also ensure resolution and a large receptive field.

[0017] The decoder first receives the result of the encoder and upsamples the result by 4 times, then integrates it with the convolution structure feature before downsampling in Xception, performs a 3*3 convolution on the result, and finally restores the output result to the input 256*256 pixel size by upsampling 4 times;

[0018] Furthermore, in the present invention, during the training process, the weight parameters are adjusted by calculating the loss function. The loss functions of the DeepLabv3+ network are: Cross Entropy Loss and Dice Loss, where Cross Entropy Loss is a common cross entropy loss. When the model performs tree species recognition, Softmax is called to classify pixels. Dice Loss is a set similarity measurement function. When making predictions, the weight file obtained by training is loaded, and the DeepLab v3+ network can segment the input image into corresponding areas, and the corresponding area positions are represented by different colors to obtain a visual prediction result.

[0019] Beneficial effects: The technical solution of this application has the following technical effects:

[0020] The present invention can classify the dominant forest tree species through the forest tree species hyperspectral data, and output the approximate location distribution and tree species types of the occurring forest tree species through picture information, which provides an important reference for relevant personnel to conduct forest resource surveys. It also plays an important role in the development and exploration of precious forest resources, the protection of forest ecological environment and the improvement of local economic level to a certain extent.

[0021] It should be appreciated that all combinations of the foregoing concepts, as well as additional concepts described in greater detail below, may be considered to be part of the inventive subject matter of the present disclosure, provided such concepts are not mutually inconsistent.

[0022] The foregoing and other aspects, embodiments and features of the present invention can be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the present invention, such as the features and / or beneficial effects of the exemplary embodiments, will be apparent from the following description or learned from the practice of the specific embodiments according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 The method and steps for tree species identification using DeepLab v3+ combined with hyperspectral remote sensing.

[0024] Figure 2 This is the model structure of DeepLab v3+.

[0025] Figure 3 To use the parameter changes during DeepLab v3+ model training and the prediction results of the trained model. DETAILED DESCRIPTION

[0026] In order to better understand the technical content of the present invention, specific embodiments are cited and described as follows in conjunction with the accompanying drawings. Various aspects of the present invention are described in this disclosure with reference to the accompanying drawings, in which many illustrative embodiments are shown. The embodiments of the present disclosure are not necessarily intended to include all aspects of the present invention. It should be understood that the various concepts and embodiments introduced above, as well as those described in more detail below, can be implemented in any of many ways, because the concepts and embodiments disclosed in the present invention are not limited to any implementation. In addition, some aspects disclosed in the present invention can be used alone or in any appropriate combination with other aspects disclosed in the present invention.

[0027] This embodiment provides a method for identifying dominant tree species in hyperspectral images based on deep learning, which includes hyperspectral data processing, data set construction, label production, data set expansion, DeepLab v3+ model construction, model training and tree species identification detection.

[0028] The present invention comprises the following steps:

[0029] S1: At 11:00 am on August 26, 2021, the weather conditions were good. The hyperspectral imaging sensor Pika L was mounted on the drone DJI M300RTK to collect data over the study area. The spatial resolution of the hyperspectral data is 10cm, the wavelength range is 400-1000nm, the spectral resolution is 2.1nm, and the pixel size is 5.86um. The acquired drone hyperspectral data was then cropped using the Envi platform. The pixel size of the image in the cropped area is 824plex*1753plex, and the number of bands is 300.

[0030] S2: Data preprocessing: Envi software is used to perform splicing, radiation calibration, geometric correction, atmospheric correction and other preprocessing on the acquired hyperspectral data, and noise reduction is performed based on the low-pass filter algorithm. ArcGIS software is used to create a shape file, the image is cropped according to the shape file, and then the image is corrected using the ArcGIS platform.

[0031] S3: Labeling, first conduct a field survey of tree species, and based on the field survey results, use visual interpretation to accurately map the drone hyperspectral data on ArcGIS software, mark the location of each tree species in the study area, and create a shape vector file, divide the tree species into 6 types, and form 7 categories (including background) of classification labels with different pixel values. Use the Python platform to label the label data, such as 1.png.

[0032] S4: The preprocessed data is further processed. The Python platform is used to perform preliminary band selection on the processed data through the CARS algorithm, and the first three PCA transformation component features with a cumulative variance contribution rate of 99.84% are selected to complete the dimensionality reduction extraction of hyperspectral data feature information.

[0033] S5: Dataset construction and expansion. The geographic information corresponding to the hyperspectral data and label data after feature extraction is the same. The hyperspectral data and label data after feature extraction are placed in two folders respectively, and the label file has the same name as the processed data file. Since deep learning requires a large number of forest tree species samples to learn features, in order to prevent the model from overfitting, data enhancement is used to expand the data set. This paper uses the PIL module in Python to perform transformation processing such as cropping, rotating, flipping, and adding noise on the existing data samples, including labeled samples and data samples, and divides the original processed data from 824*1753*3 size and data enhancement. At the same time, in order to meet the different needs of each process of model training, the data set is divided, and src, label, and val folders are established to store the hyperspectral data after data enhancement, labels, and the collection of hyperspectral data and corresponding labels. The sample size after data enhancement reaches 10,000 images, each image size is 256*256*3, and the training set and test set are trained at a ratio of 4:1.

[0034] S6: Model training, using Xception as the backbone network to build the DeepLab v3+ network structure, setting parameters such as learning rate, batch_size, and maximum number of iterations, using stochastic gradient descent (SGD) to optimize weights, fixing the learning rate to 0.1 and the momentum to 0.9 so that it can converge better when it is close to the optimal value. Due to hardware limitations, batch_size is set to 8 and the maximum number of iterations is set to 60, so that the loss value gradually decreases during training, and the model finally converges.

[0035] S7: Identify forest tree species. Use the trained DeepLab v3+ model to identify the preprocessed hyperspectral image data. The model will predict all hyperspectral images containing tree species information to form corresponding prediction images. Finally, the prediction images at each location are spliced ​​to form a tree species prediction map for the entire study area.

[0036] S8: For images with identified forest tree species, extract the stored file information, output the approximate distribution of forest tree species and tree species types, and provide reference for forest resource surveys.

[0037] Furthermore, the specific process of model training in step S6 is: using the DeepLab v3+ network to complete tree species classification. The core of DeepLab v3+ is the encoder and decoder, in which the encoder part uses dilated convolution to prevent the resolution from being too low. Secondly, dilated convolutions with different dilation rates are connected in series or in parallel, and a global average pooling is used to obtain more upper and lower structural information. Finally, continuous downsampling can ensure that the model obtains a wider range of features, and dilated convolutions with different dilation rates also ensure resolution and a large receptive field. The decoder part first receives the result of the encoder and upsamples the result by 4 times, then integrates (Concat) it with the convolution structure features before downsampling in Xception, performs a 3*3 convolution on the result, and finally restores the output result to the input 256*256 pixel size by upsampling by 4 times. The upsampling layer restores the segmented image to the size of the input, and can retrieve the image information in the pooling process in the decoder through learning, retaining the context information to the maximum extent, making the training results more accurate. Finally, a pixel-by-pixel classification layer is connected to send the feature map to the softmax classifier for classification.

[0038] The main error correction methods are clustering and non-maximum suppression (NMS). Clustering is to cluster the prediction results into different classes and select the result with greater confidence as the final result. However, clustering also has several problems: similar targets are easily clustered into one class, and the total number of target categories is unknown, so it is unknown how many classes they should be clustered into. Therefore, it is necessary to add the non-maximum suppression method to filter out prediction results with high overlap.

[0039] Further, the step S6 includes the following process:

[0040] S6-1: Build the DeepLab v3+ network to train the training set, define and initialize relevant parameters, use stochastic gradient descent (SGD) to optimize the weights, fix the learning rate to 0.1, the momentum to 0.9, the maximum number of iterations to 60, select Xception as the classification model, set the number of batch images to be processed batch_size to 8, and the input image size to 256×256 pixels.

[0041] S6-2: Use the image as the input of the neural network, use the Encoder layer to extract the feature map of the image, and then use the output of the feature map to perform deconvolution operation in the Decoder layer, and finally use the softmax classifier to complete the classification.

[0042] S6-3: After passing through the Encoder layer of DeepLab v3+, it is used to extract features and save pooling indexes. The same convolution operation is performed to extract features. The original size of the image is maintained after convolution. The Encoder part uses the first 13 layers of VGG16. Unlike VGG16, the Encoder part stores the max-pooling index in order to perform nonlinear upsampling in the Decoder part.

[0043] S6-4: Deconvolution operation and upsampling process of the Decoder layer of DeepLab v3+. The deconvolution here is no different from the convolution operation of the Encoder part. Deconvolution is used to reproduce the features of the image after classification, and upsampling restores the feature information to the original size of the image.

[0044] S6-5: The category to which each suggestion box belongs is calculated through the fully connected layer and the softmax function. In the present invention, it is mainly used to distinguish the tree species categories, and the probability vector is output at the same time. The position offset of each suggestion box is obtained by bounding box regression, and the loss function is calculated. At the same time, the entire network parameters are updated to obtain the training model.

[0045] In the steps S6-3 and S6-4, the Encoder part of DeepLab v3+ uses Xception, which achieves fast calculation and storage by modifying the Entry flow network structure, and replaces all max pooling layers with depthwise separable convolutions, so that feature extraction can be performed on input data at any resolution. The Decoder is deconvolution and upsampling, and the features are reproduced after classification through deconvolution, and the image is restored to its original size through upsampling. Finally, the output of the Encoder is sent to the softmax classifier to independently generate class probabilities for each pixel to complete the classification.

[0046] When the hyperspectral data of forest tree species are identified in step S7, based on the high accuracy of the DeepLab v3+ model, good results can still be obtained even when the source of individual forest tree species is tiny and the tree species are in the shadow area.

[0047] After identifying the tree species identification image, running the corresponding Python program can extract the information in the tree species image, output the approximate location and tree species type of the forest tree species, and provide a reference for forest resource surveys.

[0048] As described above, an embodiment of the forest tree species identification method based on DeepLab v3+ of the present invention is introduced. The present invention constructs a DeepLab v3+ network, pre-processes the data, completes the preliminary selection of data bands through the CARS algorithm, and then performs PCA transformation on the data after the preliminary selection of bands, selects the first three PCA transformation component data sets with a cumulative variance contribution rate of 99.84% to construct, trains the data set to obtain the DeepLab v3+ model, and performs forest tree species identification on the prediction set to solve the problems of low digitization degree and low recognition rate of tree species identification.

[0049] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. A person with ordinary knowledge in the technical field to which the present invention belongs may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the definition of the claims.

Claims

1. A method for identifying dominant tree species in hyperspectral images based on deep learning, characterized in that: The steps include: S1: Acquire data using a hyperspectral imaging sensor carried by a drone; S2: Perform pre-processing such as denoising and correction on the acquired hyperspectral images, extract the characteristic information of each dominant tree species after algorithm denoising, and use visual interpretation methods to calibrate the tree species information based on the original hyperspectral image information and field survey results; S3: Build a training model based on the DeepLab v3+ algorithm and import the dataset for training; S4: Use the trained model to perform recognition test on the test set data.

2. According to the method for identifying dominant tree species in hyperspectral images based on deep learning as described in claim 1, it is characterized in that: In step S2, in view of the influence of clouds and radiation, correction processing is first performed, then noise reduction processing is performed using a low-pass filtering algorithm, and then band selection is performed using the CARS algorithm. Finally, data feature extraction is completed using PCA transformation, and tree species information is calibrated to produce a hyperspectral image dominant tree species identification data set; data expansion (translation, inversion, rotation, etc.) is performed on the labeled data, and the src, label and val folders are used as indexes when the model reads files.

3. According to the method for identifying dominant tree species in hyperspectral images based on deep learning as described in claim 2, it is characterized in that: In step S3, DeepLab v3+ is used to train the training set, and hyperparameters such as learning rate are defined and initialized; the classification of dominant tree species is completed using the DeepLab v3+ network. The core of DeepLab v3+ is the encoder and decoder, in which the encoder part uses dilated convolution to prevent the resolution from being too low. Secondly, dilated convolutions with different expansion rates are connected in series or in parallel, and a global average pooling is used to obtain more upper and lower structural information. Finally, continuous downsampling can ensure that the model obtains a wider range of features, and dilated convolutions with different expansion rates also ensure resolution and a large receptive field. The decoder part first receives the result of the encoder and upsamples the result by 4 times, then integrates (Concat) it with the convolution structure features before downsampling in Xception, performs a 3*3 convolution on the result, and finally restores the output result to the input 256*256 pixel size by upsampling by 4 times. The upsampling layer restores the segmented image to the size of the input, and can retrieve the image information in the pooling process in the decoder through learning, retaining the context information to the maximum extent, making the training results more accurate. Finally, a pixel-by-pixel classification layer is connected to send the feature map to the softmax classifier for classification.

4. According to the method for identifying dominant tree species in hyperspectral images based on deep learning as described in claim 3, it is characterized in that: In the steps, during the training process, the weight parameters are adjusted by calculating the loss function. The loss functions of the DeepLab v3+ network are: Cross Entropy Loss and Dice Loss, where Cross Entropy Loss is a common cross entropy loss. When the model performs tree species recognition, Softmax is called to classify pixels. The DeepLab v3+ network can segment the input image into corresponding areas, and the corresponding area positions are represented by different colors to obtain a visual prediction result.

5. According to the method for identifying dominant tree species in hyperspectral images based on deep learning as described in claim 1, it is characterized in that: In step S4, the test image is input into the model, and the corresponding tree species areas are divided according to the parameter information in the image.

Images