A method for coniferous forest tree species classification based on a convolutional neural network

Abstract

The application discloses a kind of based on convolutional neural network's multispectral image coniferous forest tree species classification method, it is related to tree species identification technical field, the method includes steps: collecting tree species multispectral image, constructs tree species multispectral image dataset, and carries out weighted principal component analysis and data enhancement processing to the tree species multispectral image dataset;With U-Net model as architecture, utilize the way of combination of deep convolution and continuous expansion rate's hollow convolution, apply LeakyReLU activation function, and add multi-head cross attention mechanism on the jump path, construct tree species classification model;Using the processed dataset, optimize tree species classification model parameters by hybrid loss function and Adam optimizer;Based on the tree species classification model, coniferous forest tree species classification is carried out.The application utilizes the way of combination of deep convolution and continuous expansion rate's hollow convolution, improves the learning and classification ability to complex tree species characteristics.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of tree species identification, and particularly relates to a coniferous forest tree species classification method based on a convolutional neural network. BACKGROUND

[0002] Coniferous forests are an important component of forest ecosystems, and accurate classification of their tree species has profound significance for the rational use of forest resources, ecosystem monitoring, and biodiversity protection. However, traditional methods of classifying coniferous forest tree species rely mainly on manual field surveys and remote sensing classification techniques based on single features, which have many limitations. Manual field surveys are time-consuming, labor-intensive, and inefficient, and it is difficult to comprehensively monitor large areas of coniferous forest. Remote sensing classification methods based on single features, such as using only spectral features or texture features, often fail to fully utilize the rich information in multispectral images, resulting in low accuracy in classifying coniferous forest tree species in complex environments, and the classification results cannot meet actual needs.

[0003] Traditional classification methods based on machine learning often fail to fully exploit the complex relationships and deep feature information between different bands in multispectral images, and their classification performance is limited for coniferous forests, which have special spectral and structural characteristics. Moreover, the lack of effective feature fusion mechanisms limits the accuracy and generalization ability of tree species classification. SUMMARY

[0004] The present application aims to provide a coniferous forest tree species classification method based on a convolutional neural network to solve the problems in the prior art.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solution: a coniferous forest tree species classification method based on a convolutional neural network, the method comprising the steps of:

[0006] S1, collecting tree species multispectral images, constructing a tree species multispectral image dataset, and performing weighted principal component analysis and data enhancement processing on the tree species multispectral image dataset;

[0007] S2, using a U-Net model as the architecture, combining deep convolution with continuous dilation rate and applying LeakyReLU activation function, and adding a multi-head cross-attention mechanism on the skip path to construct a tree species classification model;

[0008] S3, using the processed dataset to train and optimize the tree species classification model using an Adam optimizer;

[0009] S4, classifying coniferous forest tree species based on the tree species classification model.

[0010] According to the above scheme, the weighted principal component analysis comprises:

[0011] The original pixel value matrix X of the tree species multispectral image is represented as x ij , wherein x ij represents the pixel value of the i-th pixel point on the j-th waveband; i = 1, 2, …, n; i represents the sample number, i.e., the pixel point, n represents the total number of samples, j = 1, 2, …, p; j represents the number of wavebands, and p represents the total number of wavebands.

[0012] The tree species multispectral image is subjected to standardization processing to eliminate the differences between different wavebands due to the dimension, and the formula is as follows:

[0013] ;

[0014] , wherein z ij represents the pixel value matrix after standardization processing; represents the mean value of the j-th waveband data; s j represents the standard deviation of the j-th waveband data.

[0015] A weight vector w = (w1, w2, …, w p ) is set, and a higher weight is given to the near-infrared waveband to effectively highlight the tree species information.

[0016] When a higher weight is given to the near-infrared waveband, the vegetation information can be effectively highlighted, because the near-infrared waveband is particularly sensitive to vegetation reflectivity; and coniferous tree species remain green all year round, and even in winter, they can maintain a high near-infrared reflectivity. The weighted principal component analysis with a higher weight given to the near-infrared waveband can reduce the influence of seasonal changes on the classification result, because the near-infrared waveband has less reflectivity change for tree species in different seasons, and can effectively distinguish coniferous forest tree species, even in the non-growing season.

[0017] Based on the standardization processing and the weight vector, a covariance matrix C is obtained; the covariance matrix reflects the correlation between different wavebands and the dispersion degree of each waveband data.

[0018] Based on the covariance matrix C, eigenvalues λ1, λ2, …, λ p and corresponding eigenvectors v1, v2, …, v p are obtained; wherein the eigenvalue represents the original data information contained in the corresponding principal component.

[0019] The eigenvalues are sorted in descending order based on the size, and the eigenvectors v1, v2, …, v m corresponding to the first m larger eigenvalues are selected; wherein m is a positive integer and m ≤ p.

[0020] Based on the standardized pixel value matrix z ij and the selected feature vector matrix V=(v1, v2, …, v m ), a weighted principal component matrix is obtained.

[0021] Each column of data of the weighted principal component matrix is a needle leaf tree species feature extracted after weighted principal component analysis.

[0022] According to the above scheme, the data enhancement processing includes:

[0023] The data enhancement processing includes rotation, horizontal flipping and scaling operations.

[0024] The rotation randomly selects a rotation angle and rotates each image in the tree species multispectral image data set one by one. Each rotation generates a new image.

[0025] The horizontal flipping randomly selects part of the images in the tree species multispectral image data set and mirrors the part of the images along the horizontal direction to generate new images.

[0026] The scaling sets a scaling ratio range, randomly selects a scaling ratio value within the scaling ratio range, and scales each image in the tree species multispectral image data set. Each scaling generates a new image.

[0027] The new image is added to the data set as derivative data of the original image.

[0028] According to the above scheme, the tree species classification model includes:

[0029] The tree species classification model uses a U-Net model as the architecture, and the U-Net model includes an encoder, a decoder, a skip connection, and a final convolutional layer.

[0030] The encoder is composed of a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, and a bottleneck convolutional block, which are used to downsample the input image and extract features.

[0031] The decoder is composed of a first upsampling block, a second upsampling block, a third upsampling block, and a fourth upsampling block, which are used to upsample the feature map output by the encoder to a larger spatial size.

[0032] The skip connection connects the corresponding blocks of the encoder and the decoder, splices the feature maps of the same size, effectively combines feature information of different scales, and improves the accuracy of segmentation. A multi-head cross-attention mechanism is added on the skip connection path.

[0033] The multi-head cross attention mechanism captures different feature subspaces by parallel computing multiple attention heads.

[0034] The final convolution layer reduces the number of feature map channels to the required output channel number using a 1x1 convolution kernel.

[0035] According to the above scheme, the first convolution block receives an input image and transmits the output feature map to the next convolution block; the bottleneck convolution block includes three layers of convolution operations and outputs a fifth feature map with the same size as the feature map output by the fourth convolution block;

[0036] Four max pooling layers are inserted between the first convolution block, the second convolution block, the third convolution block, the fourth convolution block and the bottleneck convolution block to reduce the spatial dimension of the feature map.

[0037] The max pooling layer uses a pooling window to slide on the feature map output by the convolution block, and selects the pixel with the maximum pixel value in the window as the value of the corresponding position of the output feature map each time to reduce the spatial dimension of the feature map.

[0038] The first convolution block, the second convolution block, the third convolution block and the fourth convolution block each include two layers of convolution operations, and after each layer of convolution operation, a LeakyReLU activation function is used for nonlinear transformation.

[0039] The first upsampling block, the second upsampling block, the third upsampling block and the fourth upsampling block each include two layers of convolution operations.

[0040] The first upsampling block performs feature splicing on the feature map output by the bottleneck convolution block and the feature map output by the fourth convolution block obtained through the jump connection to form the input of the first upsampling block and perform upsampling.

[0041] After the first upsampling block performs upsampling, the output image is transmitted to the next upsampling block, and the feature map of the corresponding block is obtained through the jump connection for feature splicing.

[0042] The fourth upsampling block transmits the output image to the final convolution layer.

[0043] Based on the final convolution layer, a classification result image is output.

[0044] According to the above scheme, the combination of deep convolution and dilated convolution with continuous expansion rate includes:

[0045] The two layers of convolution operations in the encoder and the decoder are a combination of deep convolution and dilated convolution with continuous expansion rate.

[0046] Based on the deep convolution, feature extraction is performed;

[0047] The feature extraction is performed by 3*3 convolution kernel sliding point by point at each position of the image, and the pixels in the surrounding 3*3 region are operated for each sliding position to capture the local features of the image, and the formula is as follows:

[0048] ;

[0049] Wherein, y a represents a pixel in the output feature map of the deep convolution, and the pixel position is a; x c represents the pixel value corresponding to the convolution kernel in the input image; w c represents the weight value of the convolution kernel; b represents the bias term; k represents the size of the convolution kernel; a and c represent the pixel positions of the output image and the input image respectively;

[0050] Based on the dilated convolution, the receptive field of the convolution kernel is increased by the dilation rate to capture larger range of features;

[0051] The receptive field of the convolution kernel is increased by the dilation rate, and in the sliding process of the 3*3 convolution kernel, the adjacent calculation positions skip pixels according to the interval of the dilation rate, covering a larger range of image area, so as to capture larger range of features, and the formula is as follows:

[0052] ;

[0053] Wherein, z a represents a pixel in the output feature map of the dilated convolution, and the pixel position is a; x rate(c) represents the pixel value sampled in the input image according to the dilation rate rate of the dilated convolution; rate(c) represents the sampling interval of the dilation rate;

[0054] The dilation rate of the dilated convolution in the first convolution block, the second convolution block, the third convolution block and the fourth convolution block is rate1, rate2, rate3 and rate4 respectively

[0055] Different dilation rates can make each convolution block obtain features of different scales and different emphases; by setting different dilation rates of dilated convolution in each convolution block, the model can learn more rich and representative features from simple local features to complex environment-related features; the accumulation and combination of these features enhance the extraction and analysis ability of the tree species classification model to tree species features, thereby improving the accuracy and generalization ability of the tree species classification model in the tree species classification task, which helps to accurately distinguish different tree species.

[0056] According to the above scheme, the multi-head cross attention mechanism added on the skip connection path comprises:

[0057] When the feature map output by the fourth convolutional block is transmitted to the upsampling block, the feature map output by the bottleneck convolutional block is spliced with the feature map to obtain a feature splicing map;

[0058] Based on the multi-head cross attention mechanism, the feature map output by the bottleneck convolutional block is taken as a query, and the feature map output by the fourth convolutional block is taken as a key and a value to obtain a multi-head cross attention output;

[0059] The multi-head cross attention output is combined with the feature splicing map to obtain a decoder feature map, which helps to enhance the feature information;

[0060] Based on the first upsampling block, the decoder feature map is subjected to the upsampling;

[0061] The use of the multi-head cross attention mechanism on the skip connection helps to integrate low-level local detail information from the encoder and high-level semantic information from the decoder, improves the fusion effect of the feature information, and enables the model to better understand and utilize information of different scales. For the subtle feature differences of different tree species in multispectral images, it can better capture and utilize them.

[0062] According to the above scheme, the LeakyReLU activation function performs nonlinear transformation, which comprises:

[0063] The LeakyReLU activation function sets the value of alpha to make the gradient in the negative half-axis region still exist, and the formula is as follows:

[0064] ;

[0065] Wherein, f(d) represents the LeakyReLU activation function, d represents the input value; represents a small constant alpha, which is used to control the slope of the activation function when d<0, i.e. negative slope;

[0066] Based on the Adam optimizer, the tree species classification model is optimized, and the optimal alpha value is selected as the negative slope.

[0067] According to the above scheme, the tree species classification model is trained and optimized by using the Adam optimizer, which comprises:

[0068] The processed data set is randomly divided into a training set, a validation set and a test set;

[0069] In each training period, the tree species classification model is trained based on the training set;

[0070] In the process of training the tree species classification model, the gradient is generated through a back propagation algorithm;

[0071] The Adam optimizer automatically adjusts the learning rate for each parameter based on the gradient, using the first and second moments of the gradient;

[0072] Based on the automatically adjusted learning rate for each parameter, the parameters of the tree species classification model are continuously updated in each training cycle, accelerating the training process and improving the stability of convergence;

[0073] Each parameter includes weight, bias, and negative slope alpha value of LeakyReLU activation function;

[0074] Setting the batch size and learning rate ensures stable and effective training;

[0075] After the training cycle ends, the performance of the tree species classification model is evaluated based on the validation set to help select hyperparameters; when the performance of the validation set no longer improves after several cycles, the early stopping mechanism is triggered to stop training and avoid overfitting;

[0076] Based on the gradient-based optimization algorithm, the alpha value is used as the optimized hyperparameter, and the validation set is used as the objective function for optimization;

[0077] Based on the test set, the final performance of the tree species classification model is evaluated to simulate the performance of the tree species classification model in actual data.

[0078] According to the above scheme, the tree species classification model outputs a classification result map with the same size as the input tree species multispectral image; the classification result map indicates the location of each tree species through color coding;

[0079] Based on the classification result map, the proportion of different tree species in the classification result map is counted and labeled for display.

[0080] Compared with the prior art, the beneficial effects of the present application are:

[0081] 1、The present application combines the characteristics of coniferous forests and uses weighted principal component analysis to stably extract tree species features and improve classification accuracy;

[0082] 2、The present application uses a combination of deep convolution and continuous expansion rate of the hollow convolution to improve the learning and classification ability of complex tree species features;

[0083] 3、The present application uses the Adam optimizer to automatically adjust the learning rate for each parameter, avoiding the problem of manually adjusting the global learning rate and improving the efficiency of the tree species classification model. BRIEF DESCRIPTION OF DRAWINGS

[0084] Fig. 1 A flow chart of steps of a method for coniferous forest tree species classification based on a convolutional neural network according to the present application;

[0085] Fig. 2 A structure diagram of a tree species classification model of a method for coniferous forest tree species classification based on a convolutional neural network according to the present application. DETAILED DESCRIPTION

[0086] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0087] Embodiment: As shown in the accompanying drawings, the present application provides a technical solution, a method for coniferous forest tree species classification based on a convolutional neural network, which comprises the following steps: Figs. 1-2

[0088] S1, collect tree species multispectral images, construct a tree species multispectral image dataset, and perform weighted principal component analysis and data enhancement processing on the tree species multispectral image dataset;

[0089] Specifically, a multispectral image containing a coniferous forest region is obtained by using a UAV, satellite remote sensing or aerial sensing equipment; wherein, different seasons, different geographical regions, different growth states and different types of coniferous forests are covered to comprehensively cover all possible situations; the resolution of each image is 1024x1024 pixels, there are a total of 150 images, and the collected images are labeled according to tree species types to construct a tree species multispectral image dataset;

[0090] The tree species multispectral image dataset is subjected to weighted principal component analysis. Specifically, first, the original pixel value matrix X=(x ij ) of the tree species multispectral image is obtained, wherein x ij represents the pixel value of the i-th pixel point on the j-th waveband; i=1, 2, …, n; i represents the sample number, i.e. the pixel point, n represents the total number of samples, j=1, 2, …, p; j represents the number of wavebands, and p represents the total number of wavebands; for example, the obtained multispectral image dataset contains 1000 samples, each sample has 9 wavebands, and the original pixel value matrix X=(x ij ) is obtained, wherein i=1, 2, …, 1000; j=1, 2, …, 9;

[0091] ​The multispectral image of the tree species is normalized to eliminate differences between different wave bands due to the dimension, and the formula is as follows:

[0092] ;

[0093] wherein z ij represents a pixel value matrix after normalization processing; represents the mean value of the jth wave band data; s j represents the standard deviation of the jth wave band data;

[0094] A weight vector w=(w1, w2, …, w p ) is set, and a higher weight is given to the near-infrared wave band to effectively highlight the tree species information; for example, the near-infrared wave band w NIR =0.2, and the other wave bands are 0.1.

[0095] When a higher weight is given to the near-infrared wave band, the vegetation information can be effectively highlighted, because the near-infrared wave band is particularly sensitive to vegetation reflectivity; and the coniferous tree species remains green all year round, and even in winter, it can maintain a high near-infrared reflectivity. The weighted principal component analysis with a higher weight given to the near-infrared wave band can reduce the influence of seasonal changes on the classification result, because the near-infrared wave band has less reflectivity change of the tree species in different seasons, and can effectively distinguish the coniferous forest tree species, even in the non-growing season.

[0096] Based on the normalization processing and the weight vector, a covariance matrix C is obtained; the covariance matrix reflects the correlation between different wave bands and the dispersion degree of each wave band data;

[0097] Based on the covariance matrix C, eigenvalues λ1, λ2, …, λ p and corresponding eigenvectors v1, v2, …, v p are obtained; wherein the eigenvalue represents the original data information contained in the corresponding principal component;

[0098] The eigenvectors v1, v2, …, v m corresponding to the first m larger eigenvalues are selected in descending order based on the size of the eigenvalue; wherein m is a positive integer and m≤p;

[0099] Based on the pixel value matrix z ij after normalization processing and the selected eigenvector matrix V=(v1, v2, …, v m ), a weighted principal component matrix is obtained.

[0100] The weighted principal component matrix, and each column of data is the feature of the coniferous forest tree species extracted after weighted principal component analysis.

[0101] The tree species multispectral image dataset is subjected to data enhancement processing; specifically, each image in the tree species multispectral image dataset is subjected to data enhancement processing one by one; the data enhancement processing includes rotation, horizontal flipping and scaling operations;

[0102] Specifically, the rotation randomly selects a rotation angle, rotates each image in the tree species multispectral image dataset one by one, and generates a new image each time it is rotated; for example, the random rotation angle θ is in the range of [-30°, 30°], for the first image, a random rotation angle θ = 15° is selected, and the image is rotated 15° clockwise to generate a new image; the horizontal flipping randomly selects part of the images in the tree species multispectral image dataset, and mirrors the part of the images along the horizontal direction to generate new images; for example, 50% of the images in the data set are randomly selected and mirrored along the horizontal direction to generate new images; the scaling sets a scaling ratio range, randomly selects a scaling ratio value within the scaling ratio range, and scales each image in the tree species multispectral image dataset, generating a new image each time it is scaled; for example, the scaling ratio range is set to [0.8, 1.2], and for the first image, a scaling ratio of 0.9 is randomly selected, and the image is scaled by a ratio of 0.9 to generate a new image; the new image is added to the data set as derivative data of the original image, expanding the original data set from 150 to 4160.

[0103] S2, using a U-Net model as the architecture, combining deep convolution and continuous expansion rate hollow convolution, applying a LeakyReLU activation function, and adding a multi-head cross attention mechanism on the jump path to construct a tree species classification model;

[0104] Specifically, the tree species classification model uses a U-Net model as the architecture, which includes an encoder, a decoder, a jump connection and a final convolution layer; the encoder is composed of a first convolution block, a second convolution block, a third convolution block, a fourth convolution block and a bottleneck convolution block, which is used to downsample the input image and extract features; the decoder is composed of a first upsampling block, a second upsampling block, a third upsampling block and a fourth upsampling block, which is used to upsample the feature map output by the encoder to a larger spatial size; the jump connection connects the corresponding blocks of the encoder and the decoder, splices the feature maps of the same size, effectively combines feature information of different scales, and improves the accuracy of segmentation; and a multi-head cross attention mechanism is added on the jump connection path; the multi-head cross attention mechanism captures different feature subspaces through parallel calculation of multiple attention heads; the final convolution layer uses a 1×1 convolution kernel to reduce the number of feature map channels to the required output channel number.

[0105] Specifically, first, the first convolutional block receives the input image, and transmits the output feature map to the next convolutional block; for example: the resolution of the input image is 1024x1024 pixels; the image is subjected to two 3x3 convolutions in the first convolutional block, and after each layer of convolution operation, a LeakyReLU activation function is used for nonlinear transformation; after completing the convolution operation in the first convolutional block, a maximum pooling layer is entered, a pooling operation is performed, and a feature map with a resolution of 512x512 pixels is outputted;

[0106] The feature map with a resolution of 512x512 pixels is transmitted to the second convolutional block, and a feature map with a resolution of 256x256 pixels is outputted; the feature map with a resolution of 256x256 pixels is transmitted to the third convolutional block, and a feature map with a resolution of 128x128 pixels is outputted; the feature map with a resolution of 128x128 pixels is transmitted to the fourth convolutional block, and a feature map with a resolution of 64x64 pixels is outputted;

[0107] The bottleneck convolutional block includes three layers of convolution operations, and outputs a feature map with a resolution of 64x64 pixels;

[0108] When the fourth convolutional block outputs a feature map with a resolution of 64x64 pixels, it is transmitted to the first upsampling block, and is spliced with the feature map with a resolution of 64x64 pixels outputted by the bottleneck convolutional block to obtain a feature splicing map; and based on the multi-head cross attention mechanism, the feature map outputted by the bottleneck convolutional block is taken as a query, and the feature map outputted by the fourth convolutional block is taken as a key and a value to obtain a multi-head cross attention output;

[0109] Specifically, the multi-head cross attention output is combined with the feature splicing map to obtain a decoder feature map, which helps to enhance the feature information; the decoder feature map is upsampled based on the first upsampling block; the use of the multi-head cross attention mechanism on the skip connection helps to integrate low-level local detail information from the encoder and high-level semantic information from the decoder, improves the fusion effect of the feature information, and enables the model to better understand and utilize information of different scales, and better capture and utilize the subtle feature differences of different tree species in the multispectral image.

[0110] The first upsampling block, the second upsampling block, the third upsampling block and the fourth upsampling block each include two layers of convolution operations; the two layers of convolution operations include the deep convolution and the dilated convolution;

[0111] Specifically, first, the first upsampling block performs feature splicing on the feature map output by the bottleneck convolution block and the feature map output by the fourth convolution block obtained through the jump connection to form the input of the first upsampling block, and performs upsampling; after the first upsampling block performs upsampling, the output image is transmitted to the second upsampling block; the second upsampling block performs feature splicing on the image output by the first upsampling block and the feature map output by the third convolution block obtained through the jump connection to form the input of the second upsampling block, and performs upsampling; after the second upsampling block performs upsampling, the output image is transmitted to the third upsampling block; the third upsampling block performs feature splicing on the image output by the second upsampling block and the feature map output by the second convolution block obtained through the jump connection to form the input of the third upsampling block, and performs upsampling; after the third upsampling block performs upsampling, the output image is transmitted to the fourth upsampling block; the fourth upsampling block performs feature splicing on the image output by the third upsampling block and the feature map output by the first convolution block obtained through the jump connection to form the input of the fourth upsampling block, and performs upsampling; the fourth upsampling block transmits the output image to the final convolution layer; based on the final convolution layer, a classification result image is output.

[0112] Specifically, the two-layer convolution operation includes a deep convolution and a hole convolution; the deep convolution is used for feature extraction; the feature extraction is performed by sliding a 3*3 convolution kernel at each position of an image, and for each sliding position, pixels in a 3*3 region around the sliding position are operated to capture local features of the image, and a formula is as follows:

[0113] ;

[0114] wherein y a represents a pixel in an output feature map of the deep convolution, and a pixel position is a; x c represents a pixel value in an input image corresponding to the convolution kernel; w c represents a weight value of the convolution kernel; b represents a bias term; k represents a size of the convolution kernel; a and c respectively represent pixel positions of an output image and an input image;

[0115] The hole convolution increases a receptive field of the convolution kernel through a dilation rate to capture larger range features; the hole convolution increases the receptive field of the convolution kernel, and in a sliding process of the 3*3 convolution kernel, adjacent calculation positions skip pixels according to a spacing of the dilation rate to cover a larger range of image regions, so that larger range features are captured, and a formula is as follows:

[0116] ;

[0117] wherein z arepresents a pixel in the output feature map of the empty convolution, and the pixel position is a; x rate(c) represents a pixel value in the input image according to the expansion rate rate of the empty convolution; rate(c) represents a sampling interval of the expansion rate; for example: rate = 2, in the sliding process, the adjacent calculation positions skip the pixels according to the interval 2, and a pixel z in the output feature map is calculated a ; specifically, the expansion rates of the empty convolutions in different convolution blocks are respectively: the expansion rate rate1 of the first convolution block, the expansion rate rate2 of the second convolution block, the expansion rate rate3 of the third convolution block, and the expansion rate rate4 of the fourth convolution block; for example: the expansion rate rate1 of the first convolution block is 1, the expansion rate rate2 of the second convolution block is 2, the expansion rate rate3 of the third convolution block is 3, and the expansion rate rate4 of the fourth convolution block is 4; different expansion rates can enable each convolution block to obtain features of different scales and different emphases; by setting the empty convolutions with different expansion rates in each convolution block, the model can learn more rich and representative features, from simple local features to complex environment-related features; the accumulation and combination of these features enhance the extraction and analysis ability of the tree species classification model for tree species features, thereby improving the accuracy and generalization ability of the tree species classification model in the tree species classification task, and helping to accurately distinguish different tree species.

[0118] Specifically, the LeakyReLU activation function sets the value of a to make the gradient in the negative half-axis region still exist, and the formula is as follows:

[0119] ;

[0120] wherein f(d) represents the LeakyReLU activation function, d represents the input value; a represents a small constant, which is used to control the slope of the activation function when d < 0, i.e. the negative slope; based on the Adam optimizer to optimize the tree species classification model, the optimal a value is selected as the negative slope.

[0121] S3, using the processed data set, training and using the Adam optimizer to optimize the tree species classification model;

[0122] Specifically, first, the processed data set is randomly divided into a training set, a validation set and a test set; for example, the processed 4160 image data set is randomly divided into a training set (2912), a validation set (624) and a test set (624) in the ratio of 70:15:15; in each training cycle, based on the training set, the tree species classification model is trained; in the process of training the tree species classification model, the gradient is generated through the back propagation algorithm; using the Adam optimizer, based on the gradient, the first and second moments of the gradient are used to automatically adjust the learning rate for each parameter; based on the automatic adjustment of the learning rate of each parameter, in each training cycle, the parameters of the tree species classification model are constantly updated to speed up the training process and improve the stability of convergence; wherein each parameter includes weight, bias and negative slope alpha value of LeakyReLU activation function; setting the batch size and learning rate ensures stable and effective training; after the training cycle ends, the performance of the tree species classification model is evaluated based on the validation set to help select hyperparameters; when the performance of the validation set no longer improves after several cycles, the early stopping mechanism is triggered to stop training to avoid overfitting; for example: when the performance of the validation set no longer improves after 10 consecutive cycles, the early stopping mechanism is triggered to stop training to avoid overfitting; based on the gradient optimization algorithm, the alpha value is used as the optimized hyperparameter, and the validation set is used as the target function for optimization; based on the test set, the final performance of the tree species classification model is evaluated, for example, the accuracy, recall rate, F value and other indicators are used to evaluate the model performance to help select hyperparameters and simulate the performance of the tree species classification model in actual data.

[0123] S4, based on the tree species classification model, coniferous forest tree species classification is performed.

[0124] Specifically, the tree species classification model outputs a classification result map with the same size as the input tree species multispectral image; the classification result map indicates the location of each tree species through color coding, for example: the spruce region is marked as blue, the pine region is marked as green, and the fir region is marked as red to indicate the location of each tree species; the proportion of different tree species in the classification result map is counted and marked for display, for example: the blue region (spruce) accounts for 30%, the green region (pine) accounts for 50%, and the red region (fir) accounts for 20%, and these proportions are marked for display to intuitively understand the distribution of different tree species in the test area.

[0125] The present application provides a technical solution, a tree species classification model of a multispectral image coniferous forest tree species classification method based on a convolutional neural network;

[0126] Specifically, the tree species classification model takes the U-Net model as the architecture, the U-Net model comprising an encoder, a decoder, a skip connection and a final convolutional layer; the encoder is composed of a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block and a bottleneck convolutional block; the decoder is composed of a first upsampling block, a second upsampling block, a third upsampling block and a fourth upsampling block; the skip connection connects the corresponding blocks of the encoder and the decoder; a multi-head cross attention mechanism is added on the skip connection path; the final convolutional layer is composed of a 1x1 convolutional kernel.

[0127] Specifically, the tree species classification model comprises the following steps: first, input the processed multispectral coniferous forest image into the first convolutional block; in the first convolutional block, perform two 3x3 convolutions, and after each convolution operation, perform nonlinear transformation by using a LeakyReLU activation function; after the convolution and activation function operations are completed, input to a max-pooling layer to perform a pooling operation, and transmit the output feature map to the second convolutional block; the operations of the second convolutional block, the third convolutional block and the fourth convolutional block are the same as those of the first convolutional block, the second convolutional block transmits the output feature map to the third convolutional block; the third convolutional block transmits the output feature map to the fourth convolutional block; and the bottleneck convolutional block comprises three convolutional operations, and outputs a feature map with the same size as the feature map output by the fourth convolutional block;

[0128] Next, when the fourth convolutional block transmits the output feature map to the first upsampling block, the feature map output by the bottleneck convolutional block is spliced to obtain a feature splicing map; and based on the multi-head cross attention mechanism, the feature map output by the bottleneck convolutional block is taken as a query, and the feature map output by the fourth convolutional block is taken as a key and a value to obtain a multi-head cross attention output;

[0129] The first upsampling block, the second upsampling block, the third upsampling block and the fourth upsampling block each comprise two layers of convolution operations; the two layers of convolution operations comprise the deep convolution and the dilated convolution; after the first upsampling block receives the multi-head cross attention output of the feature splicing graph of the fourth convolution block and the bottleneck convolution block, upsampling is performed; after the first upsampling block performs upsampling, the output image is transmitted to the second upsampling block; the second upsampling block performs feature splicing on the image output by the first upsampling block and the feature map output by the third convolution block obtained through the jump connection, forms the input of the second upsampling block, and performs upsampling; after the second upsampling block performs upsampling, the output image is transmitted to the third upsampling block; the third upsampling block performs feature splicing on the image output by the second upsampling block and the feature map output by the second convolution block obtained through the jump connection, forms the input of the third upsampling block, and performs upsampling; after the third upsampling block performs upsampling, the output image is transmitted to the fourth upsampling block; the fourth upsampling block performs feature splicing on the image output by the third upsampling block and the feature map output by the first convolution block obtained through the jump connection, forms the input of the fourth upsampling block, and performs upsampling; the fourth upsampling block transmits the output image to the final convolution layer; based on the final convolution layer, a classification result graph is output.

[0130] It will be obvious to a person skilled in the art that the application is not limited to the details of the above-described exemplary embodiments but can be implemented in other embodiments without departing from the scope of the application. The embodiments are to be considered in all respects as being illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the description above, and therefore all changes which come within the meaning and range of equivalency of the claims are to be embraced therein. Any reference signs in the claims should not be construed as limiting the scope of the claims.

Claims

1. A method for coniferous tree species classification based on convolutional neural network, characterized in that: The method comprises the steps of: S1, collecting tree species multispectral images, constructing a tree species multispectral image dataset, and performing weighted principal component analysis and data enhancement processing on the tree species multispectral image dataset; The original pixel value matrix X of the tree species multispectral image is (x ij ), wherein x ij represents the pixel value of the i-th pixel point on the j-th wave band; i=1, 2, …, n; i represents the sample number, that is, the pixel point, n represents the total number of samples, j=1, 2, …, p; j represents the number of wave bands, and p represents the total number of wave bands; The tree species multispectral images are standardized to eliminate differences between different wavebands due to dimensions, and the formula is as follows: ； wherein z ij is represented as a matrix of normalized pixel values; is represented as a mean value of the jth band data; s j is represented as a standard deviation of the jth band data; A weight vector w = (w1, w2, …, w p ) is set, and higher weight is given to the near-infrared band, effectively highlighting the tree species information; Based on the standardization processing and the weight vector, a covariance matrix C is obtained; the covariance matrix reflects the correlation between different wavebands and the dispersion degree of each waveband data; Based on the covariance matrix C, eigenvalues λ1, λ2, …, λ p and corresponding eigenvectors v1, v2, …, v p are obtained; wherein the eigenvalues represent the original data information contained in the corresponding principal components; The eigenvectors corresponding to the first m larger eigenvalues are selected in descending order according to the size of the eigenvalues, and the eigenvectors v1, v2, …, vm corresponding to the first m larger eigenvalues are selected. m ; wherein m is a positive integer and m≤p; based on the standardized pixel value matrix z ij and the selected feature vector matrix V = (v1, v2, …, v m ), a weighted principal component matrix is obtained; The weighted principal component matrix is the feature of the coniferous forest tree species extracted after the weighted principal component analysis; S2, using the U-Net model as the architecture, combining deep convolution and continuous expansion rate of the hole convolution, applying the LeakyReLU activation function, and adding a multi-head cross attention mechanism on the jump path to construct a tree species classification model; The tree species classification model uses the U-Net model as the architecture, and the U-Net model includes an encoder, a decoder, a jump connection, and a final convolution layer; The encoder is composed of a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, and a bottleneck convolution block, which is used to downsample the input image and extract features; The decoder is composed of a first upsampling block, a second upsampling block, a third upsampling block, and a fourth upsampling block, which is used to upsample the feature map output by the encoder to a larger spatial size; The jump connection connects the corresponding blocks of the encoder and the decoder, splices the feature maps of the same size, effectively combines the feature information of different scales, and improves the accuracy of segmentation; and a multi-head cross attention mechanism is added on the jump connection path; The multi-head cross attention mechanism captures different feature subspaces through parallel calculation of multiple attention heads; The final convolution layer reduces the number of feature map channels to the required output channel number using a 1x1 convolution kernel; S3, using the processed dataset to train and optimize the tree species classification model using the Adam optimizer; S4, based on the tree species classification model, classifying the coniferous forest tree species.

2. The method according to claim 1, wherein, The data enhancement processing includes: The data enhancement processing includes rotation, horizontal flipping, and scaling operations; The rotation randomly selects a rotation angle and rotates each image in the tree species multispectral image dataset one by one, and a new image is generated each time the image is rotated; The horizontal flipping randomly selects part of the images in the tree species multispectral image dataset and mirrors the part of the images along the horizontal direction to generate new images, The scaling sets a scaling range, randomly selects a scaling value within the scaling range, and scales each image in the tree species multispectral image dataset, and a new image is generated each time the image is scaled; The new image is added to the dataset as derivative data of the original image.

3. The multispectral image coniferous forest tree species classification method based on the convolutional neural network according to claim 1, wherein: The first convolutional block receives an input image and transmits an output feature map to the next convolutional block; the bottleneck convolutional block includes three layers of convolutional operations and outputs a fifth feature map with the same size as the feature map output by the fourth convolutional block; Four max-pooling layers are inserted between the first convolutional block, the second convolutional block, the third convolutional block, the fourth convolutional block and the bottleneck convolutional block to reduce the spatial dimension of the feature map; The max-pooling layer uses a pooling window to slide on the feature map output by the convolutional block, and selects the pixel with the maximum value in the window as the value of the corresponding position in the output feature map each time to reduce the spatial dimension of the feature map; The first convolutional block, the second convolutional block, the third convolutional block and the fourth convolutional block each include two layers of convolutional operations, and after each layer of convolutional operation, a LeakyReLU activation function is used for nonlinear transformation; The first upsampling block, the second upsampling block, the third upsampling block and the fourth upsampling block each include two layers of convolutional operations; The first upsampling block performs feature splicing on the feature map output by the bottleneck convolutional block and the feature map output by the fourth convolutional block obtained through the skip connection to form the input of the first upsampling block and perform upsampling; After the first upsampling block performs upsampling, the output image is transmitted to the next upsampling block, and the feature map of the corresponding block is obtained through the skip connection to perform feature splicing; The fourth upsampling block transmits the output image to the final convolutional layer; Based on the final convolutional layer, a classification result map is output.

4. The method according to claim 3, wherein, The combination of deep convolution and dilated convolution with a continuous expansion rate includes: The two layers of convolutional operations in the encoder and the decoder are a combination of deep convolution and dilated convolution with a continuous expansion rate; Based on the deep convolution, feature extraction is performed; The feature extraction is performed by sliding a 3x3 convolutional kernel at each position of the image, and for each sliding position, the pixels in the surrounding 3x3 region are operated to capture local image features, and the formula is as follows: ； wherein y a represents a pixel in the output feature map of the deep convolution, and the pixel position is a; x c represents a pixel value in the input image corresponding to the position of the convolution kernel; w c represents a weight value of the convolution kernel; b represents a bias term; k represents the size of the convolution kernel; and a and c represent the pixel positions of the output image and the input image, respectively. Based on the dilated convolution, the receptive field of the convolutional kernel is increased by the expansion rate to capture larger range features; The receptive field of the convolutional kernel is increased by the expansion rate by skipping pixels at an interval of the expansion rate during the sliding process of the 3x3 convolutional kernel to cover a larger range of image regions, achieving the effect of capturing larger range features, and the formula is as follows: ； wherein z a represents a pixel in the output feature map of the dilated convolution, and the pixel position is a; x rate(c) represents a pixel value in the input image according to the sampling interval of the dilated rate rate(c); rate(c) represents the sampling interval of the dilated rate The expansion rates of the dilated convolution in the first convolutional block, the second convolutional block, the third convolutional block and the fourth convolutional block are rate1, rate2, rate3 and rate4 respectively.

5. The method according to claim 1, wherein, Adding a multi-head cross-attention mechanism on the skip connection path includes: When the feature map output by the fourth convolutional block is transmitted to the upsampling block, it is spliced with the feature map output by the bottleneck convolutional block to obtain a feature splicing map; Based on the multi-head cross-attention mechanism, the feature map output by the bottleneck convolutional block is taken as the query, and the feature map output by the fourth convolutional block is taken as the key and the value to obtain a multi-head cross-attention output. The multi-head cross attention output is combined with the feature splicing map to obtain a decoder feature map, which helps to enhance feature information. Based on the first upsampling block, the decoder feature map is subjected to the upsampling.

6. The method according to claim 1, wherein, The LeakyReLU activation function performs nonlinear transformation, which includes: The LeakyReLU activation function sets the value of a to make the gradient in the negative half-axis region still exist, and the formula is as follows: ； Where f(d) represents the LeakyReLU activation function, d represents the input value, and a represents a small constant for controlling the slope of the activation function when d<0, i.e. negative slope. Based on the Adam optimizer, the tree species classification model is optimized, and the optimal a value is selected as the negative slope.

7. The method according to claim 6, wherein, The processed data set is used to train and optimize the tree species classification model using the Adam optimizer, which includes: The processed data set is randomly divided into a training set, a validation set and a test set; In each training cycle, based on the training set, the tree species classification model is trained; In the process of training the tree species classification model, the gradient is generated through the back propagation algorithm; The Adam optimizer automatically adjusts the learning rate for each parameter based on the gradient, using the first and second moments of the gradient; Based on the automatic adjustment of the learning rate of each parameter, the parameters of the tree species classification model are continuously updated in each training cycle to speed up the training process and improve the stability of convergence; Each parameter includes weight, bias and negative slope a value of LeakyReLU activation function; The batch size and learning rate are set to ensure stable and effective training; After the training cycle ends, the performance of the tree species classification model is evaluated based on the validation set to help select hyperparameters; when the performance of the validation set no longer improves after several cycles, the early stopping mechanism is triggered to stop training and avoid overfitting; Based on the test set, the final performance of the tree species classification model is evaluated to simulate the performance of the tree species classification model in actual data.

8. The method according to claim 1, wherein: The tree species classification model outputs a classification result map with the same size as the input tree species multispectral image; the classification result map indicates the location of each tree species through color coding; Based on the classification result map, the proportion of different tree species in the classification result map is counted and labeled for display.

Images