Hyperspectral image mining area tree species classification method combining cascade 3DCNN and SDTA coding

Abstract

The invention discloses a hyperspectral image mining area tree species classification method combining cascaded 3DCNN and SDTA coding. The method comprises the following steps: constructing a cascaded 3D convolutional neural network to extract local spectral features and spatial features; the attention encoder is transposed through segmentation depth, and the channel dimension feature weight is dynamically updated; after 3D convolution and a segmentation depth transpose attention encoder are combined, features between tree species with similar spectrums are extracted and separated in mining area tree species classification; and constructing a module fusing high-level features and low-level features based on residual connection, finally flattening the features and inputting the flattened features into a full connection layer, and outputting a final classification probability by using a softmax activation function. A mining area tree species classification deep network suitable for airborne hyperspectral data is constructed, and the high-dimensional feature extraction capability is improved through a cascaded 3D convolutional network architecture.

Description

Technical Field

[0001] The present invention relates to the technical field of mining area tree species classification, and in particular to a mining area tree species classification method using a hyperspectral image combining cascaded 3DCNN and SDTA encoding. Background Art

[0002] Mining causes problems such as surface subsidence, groundwater level decline, and vegetation degradation, which seriously threaten ecological security. China is accelerating the promotion of mine ecological restoration and developing green mines. Vegetation plays an important role in water conservation, soil and water conservation, regulating carbon balance, and alleviating regional microclimate. Therefore, vegetation restoration is regarded as a key link in mine ecological restoration, and afforestation is the most direct and important means of vegetation restoration. Fine classification of tree species is an important basis for revealing the temporal and spatial evolution of vegetation restoration in mining areas and rationally planning and designing mining area ecological restoration models. It helps to deeply understand the complexity and diversity of forest ecosystems, help effectively prevent and control forest diseases, guide mining area ecological management and restoration, optimize the utilization and protection of forest resources, and further coordinate the symbiotic relationship between man and nature. It has important scientific guidance and practical significance for mining area ecological restoration.

[0003] Different tree species have unique spectral characteristics. The reflectance information of hundreds of bands in hyperspectral images makes them have great potential in the field of tree species classification. Especially with the rapid development of unmanned aerial vehicles, airborne hyperspectral data can make up for the time-consuming and laborious defects of conventional manual field surveys. However, with the increase of spectral channel dimensions, problems such as increased data volume, reduced correlation between bands, and difficulty in feature extraction arise, resulting in reduced final classification accuracy and efficiency. Therefore, the key to using hyperspectral data for tree species classification is to explore efficient and accurate feature extraction solutions.

[0004] In recent years, tree species supervised classification technology based on hyperspectral remote sensing technology has been developed. For example, machine learning algorithms such as support vector machine random forest and stacking ensemble have been applied in tree species classification, proving that it is feasible to distinguish categories by using differences in tree species spectral information. However, machine learning requires artificial feature construction and selection. In tasks such as tree species classification where the relationship between data categories is complex, it is difficult to capture complex patterns between data. In addition, the spatial variation of tree species spectral features, namely spectral uncertainty, makes it difficult for the extracted features to adapt to all scenarios. Although the single knowledge provided by hyperspectral data can be compensated by spatial-spectral information fusion technology, these methods rely heavily on prior foundations and expert domain knowledge. In addition, deep learning methods that achieve end-to-end deep feature automatic extraction have been widely used in remote sensing image classification tasks. In particular, convolutional neural networks have received widespread attention due to their excellent feature extraction capabilities. Many scholars have applied CNN and its variants to tree species classification tasks.

[0005] Although the existing tree species classification methods have achieved good results, the following problems still exist.

[0006] (1) High-dimensional feature extraction requires complex network structures and the superposition of multiple layers of networks, which causes a surge in the number of network structure parameters and makes the network more difficult to train.

[0007] (2) The convolution kernel receptive field of traditional convolutional neural networks is limited, making it difficult to capture global feature information. Especially in the mining restoration area, the canopy width of coniferous tree species with a short restoration period is small and the characteristics are not obvious, and the exposed soil under the forest is prone to misclassification and confusion.

[0008] (3) The restoration period of the ecological restoration area in the mining area is different, the tree ages are different, and the phenomenon of same object with different spectra is obvious. It is necessary to further automatically extract high-level features for tree species classification tasks. Summary of the invention

[0009] The purpose of the present invention is to provide a method for classifying tree species in mining areas using hyperspectral images that combines cascaded 3DCNN and SDTA encoding to address the above-mentioned problems, construct a deep network for classifying tree species in mining areas that is suitable for airborne hyperspectral data, and improve the ability to extract high-dimensional features through a cascaded 3D convolutional network architecture.

[0010] The technical solution of the present invention is as follows: A method for classifying tree species in mining areas using hyperspectral images by combining cascaded 3DCNN and SDTA encoding includes the following steps: Construct a cascaded 3D convolutional neural network to extract local spectral and spatial features; Dynamically update the channel dimension feature weights by splitting the deep transposed attention encoder; By combining 3D convolution with segmentation deep transposed attention encoder, features between tree species with similar spectra can be extracted and separated in the classification of tree species in mining areas; A module that fuses high-level features and low-level features is built based on residual connections, and finally the features are flattened and input into the fully connected layer, and the softmax activation function is used to output the final classification probability.

[0011] Furthermore, the cascaded 3D convolutional neural network extracts local spectral features and spatial features, specifically comprising the following steps: First, the original hyperspectral image is subjected to principal component dimensionality reduction; The feature bands that have undergone principal component dimensionality reduction are sequentially passed through two consecutive 3D convolutional layers to obtain output feature 1; Then, output feature 1 is passed through three layers of 3D convolutional layers to obtain output feature 2; The output feature 1 and the output feature 2 are fused and connected, and then passed through three layers of 3D convolutional layers to obtain the output feature 3; The output features 1, 2 and 3 are connected, fused and input into the next network component.

[0012] Furthermore, the algorithm of the 3D convolution is: , in, It is Tier The feature map is at position The activation value of It is Tier The deviation of the feature map, Index All feature maps of the layer, To connect Layer feature maps and Layer and The convolution kernel of the feature map is at position The value of , , are the height, width and depth of the convolution kernel respectively.

[0013] Furthermore, the segmentation deep transposed attention encoder includes a feature encoding module and a self-attention module, in which: Input Features Divided into Subset Subset The number of levels is based on Adaptive; Except for the first subset, the calculation method of each subset is to fuse the output features of the previous subset and then pass Depth-with convolution Perform feature encoding ; Finally, the features of each subset are concatenated to obtain output features with multi-scale receptive fields.

[0014] Furthermore, the The expression is: .

[0015] Furthermore, in the self-attention module: The input features Modify it by reshape , after three lines, we get the query ,key ,value , before calculating the cross-covariance attention, by , application Norm-stable training; Only for Transpose, get , the calculation results are combined with softmax to obtain the attention score matrix, which is applied to Get the final attention map.

[0016] Furthermore, the self-attention module comprises: , , in, For input, is the output tensor.

[0017] Furthermore, the module for fusing high-level features and low-level features based on residual connection specifically includes the following steps: The features of the segmented deep transposed attention output are passed through a 3D convolutional layer for preliminary feature extraction and batch normalization. Then, they are passed through a 3D convolutional layer with a larger convolution kernel to capture deeper structural information and then batch normalized. Then, they are passed through another 3D convolutional layer and batch normalized again to obtain deep high-level features. The parallel convolution branches start from the previous feature extraction and go through 3D convolution and batch normalization in sequence, and finally add the output of the deep feature extraction branch to form a residual connection; The final output is converted into a one-dimensional vector through the flattening layer and input into the fully connected layer for classification; The output layer using a softmax activation function produces a classification decision.

[0018] Furthermore, the sizes of the 3D convolutional layers are different.

[0019] Furthermore, the first principal component dimensionality reduction of the original hyperspectral image is specifically to retain the first 30 principal components through spectral feature dimensionality reduction.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. A method for classifying tree species in mining areas using hyperspectral images by combining cascaded 3DCNN and SDTA encoding. A deep network for classifying tree species in mining areas suitable for airborne hyperspectral data is constructed. The high-dimensional feature extraction capability is improved through the cascaded 3D convolutional network architecture. The self-attention mechanism is introduced to improve the feature extraction capability of the model, better capture the global information of hyperspectral images, and combine with 3DCNN to achieve the fusion of local and global information. The residual connection is used to connect the entire network model, making the classification network more stable and improving the final tree species classification accuracy. 2. A method for classifying tree species in mining areas using hyperspectral images that combines cascaded 3DCNN and SDTA encoding. This classification network not only provides technical support for tree species classification in mining ecological restoration areas, but also has wide applicability to other tree species classification scenarios, providing decision support for forest resource inventory and smart forestry management. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the cascaded 3DCNN network structure of the present invention.

[0022] Figure 2 It is a schematic diagram of the structure of the SDTA encoder of the present invention.

[0023] Figure 3 Schematic diagram of the residual connection module of the present invention.

[0024] Figure 4 Comparison of tree species classification results of different methods of the present invention: (a): SDTA-3DCNN, (b): 2DCNN, (c): 3D-1DCNN, (d): 3DCNN, (e): RF, (f): SVM, (g): Stacking.

[0025] Figure 5 This is a comparison chart of classification accuracy of each category using different methods of the present invention.

[0026] Figure 6 It is the overall flow chart of the present invention. DETAILED DESCRIPTION

[0027] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0028] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0029] See also Figure 1-6 , a method for tree species classification in mining areas using hyperspectral images combining cascaded 3DCNN and SDTA encoding, such as Figure 6 As shown, the following steps are included: Construct a cascaded 3D convolutional neural network to extract local spectral features and spatial features; the structure is as follows Figure 1 shown.

[0030] The channel dimension feature weights are dynamically updated through the segmented deep transposed attention (SDTA) encoder; SDTA achieves efficient integration without adding additional parameters and multiplication-addition operations. SDTA mainly includes dynamic multi-scale feature fusion and global information encoding. The structure of the SDTA encoder is as follows: Figure 2 shown.

[0031] By combining 3D convolution with segmentation deep transposed attention encoder, features between tree species with similar spectra can be extracted and separated in the classification of tree species in mining areas; A module that fuses high-level features and low-level features is built based on residual connections, and finally the features are flattened and input into the fully connected layer, and the softmax activation function is used to output the final classification probability.

[0032] The cascaded 3D convolutional neural network extracts local spectral features and spatial features, including the following steps: First, the original hyperspectral image is subjected to principal component dimensionality reduction, and the first 30 principal components are retained through spectral feature dimensionality reduction; The feature bands that have undergone principal component dimensionality reduction are sequentially passed through two consecutive 3D convolutional layers to obtain output feature 1; Then, output feature 1 is passed through three layers of 3D convolutional layers to obtain output feature 2; The output feature 1 and the output feature 2 are fused and connected, and then passed through three layers of 3D convolutional layers to obtain the output feature 3; The output features 1, 2 and 3 are connected and fused and input to the next network component. The sizes of the above 3D convolutional layers are different.

[0033] The algorithm for 3D convolution is: , in, It is Tier The feature map is at position The activation value of It is Tier The deviation of the feature map, Index All feature maps of the layer, To connect Layer feature maps and Layer and The convolution kernel of the feature map is at position The value of , , are the height, width and depth of the convolution kernel respectively.

[0034] The segmentation deep transposed attention encoder includes a feature encoding module and a self-attention module. In the feature encoding module: Input Features Divided into Subset Subset The number of levels is based on Adaptive; Except for the first subset, the calculation method of each subset is to fuse the output features of the previous subset and then pass Depth-with convolution Perform feature encoding ; Finally, the features of each subset are concatenated to obtain output features with multi-scale receptive fields.

[0035] The expression is: .

[0036] In the self-attention module: Input features Modify it by reshape , after three lines, we get the query ,key ,value , before calculating the cross-covariance attention, by , application Norm-stable training; Only for Transpose, get , the calculation results are combined with softmax to obtain the attention score matrix, which is applied to Get the final attention map.

[0037] The self-attention module includes: , , in, For input, is the output tensor.

[0038] After combining 3D convolution with the channel attention mechanism, in the classification of tree species in mining areas, the features between tree species with similar spectra, such as similar coniferous species such as Yunnan pine and Chinese pine, can be effectively extracted and separated. This can improve the model's ability to understand complex spatial relationships and effectively reduce irrelevant or redundant feature information, thereby improving the accuracy and efficiency of classification or recognition. Of course, this combination also faces some difficulties, mainly reflected in the increase in model design and computational complexity. The integration of 3D convolution and channel attention mechanisms requires fine-tuned hyperparameters and complex model architectures, while increasing the computational cost and resource requirements during training and inference. As a lightweight attention mechanism module, SDTA effectively achieves model lightweighting and speeds up inference through adaptive channel cutting, with the fastest training speed reaching 8s / epoch.

[0039] The module based on residual connection to build a fusion high-level feature and low-level feature includes the following steps: The features of the segmented deep transposed attention output are passed through a 3D convolutional layer for preliminary feature extraction and batch normalization. Then, they are passed through a 3D convolutional layer with a larger convolution kernel to capture deeper structural information and then batch normalized. Then, they are passed through another 3D convolutional layer and batch normalized again to obtain deep high-level features. The parallel convolution branches start from the previous feature extraction and go through 3D convolution and batch normalization in sequence, and finally add the output of the deep feature extraction branch to form a residual connection; The final output is converted into a one-dimensional vector through a flattening layer and input into a fully connected layer for classification; a Dropout layer (with a dropout rate of 0.5) is set after the fully connected layer to prevent overfitting.

[0040] The output layer using a softmax activation function produces a classification decision.

[0041] Helps the model capture constant and low-level features and pass them directly to the deep part of the network, solving the gradient disappearance problem that may occur in the deep network and helping the network to train better.

[0042] Accuracy evaluation: In order to quantitatively evaluate the performance of classification results, F1-score, OA (Overall Accuracy) and Kappa are introduced as evaluation indicators. F1-score is used to evaluate the classification effectiveness of a single category, while OA and Kappa are used to evaluate the overall classification effectiveness.

[0043] , , , .

[0044] All experiments were implemented on a desktop computer equipped with 256.0GB memory and NVIDIA GeForce RTX3080. All algorithms were executed on Jupyter Notebook using Python language (v3.8.0). The deep learning framework is TensorFlow (V2.5.0). During the experiment, the optimal parameter selection for training the network was determined by manual parameter adjustment. The hyperparameters of the network are shown in Table 1.

[0045] Table 1 Network hyperparameter settings

[0046] Comparison of different classification methods: In order to verify the classification effect of the proposed method on hyperspectral data, seven classification methods including RF, SVM, Stacking, 3DCNN, 2DCNN, 3D-1DCNN and the proposed SDTA-3DCNN are used for comparative analysis. All these classification methods use the same data set and the data division method is also the same.

[0047] In order to verify the classification effectiveness of the proposed method, tree species in the tailings pond restoration area of ​​the mining area were selected and classified using a total of 6 methods including machine learning and deep learning. Figure 4The hyperspectral image (RGB band synthesis) and the classification results of the seven classification methods are shown. The quantitative evaluation results are shown in Table 2. Through visual interpretation, it was found that the three machine learning methods showed high consistency in classification results, and the classification results of the four deep learning methods were highly similar. However, there were large differences in the spatial distribution of the classification results of the machine learning classification method and the deep learning classification network, especially between alpine pine and Yunnan pine. The reason for this phenomenon may be that the spectral characteristics of alpine pine and Yunnan pine are relatively close among the four vegetations, and traditional machine learning algorithms are difficult to extract deep features. From the quantitative evaluation results, among the seven compared methods, the proposed SDTA3DCNN network has the highest overall classification accuracy (OA=99.03%, Kappa=0.99), the worst classification accuracy is SVM (OA=89.51%, Kappa=0.85), and the highest accuracy among the machine learning classification methods is Stacking (OA=94.65%, Kappa=0.93). Although the overall classification accuracy of the classification results of each deep learning classification network is not much different, from the quantitative evaluation of the classification results of each category, the SDTA3DCNN classification network performs better in each category ( Figure 5 ), even in the case of unbalanced samples, the classification accuracy is still high. For example, Tibetan cypress has only 2501 training samples, but its classification accuracy is the highest among all methods (Precision=97.49%, recall=0.93, F1-score=0.95).

[0048] Table 2 Comparison of classification accuracy of different methods

[0049] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A method for classifying tree species in mining areas using hyperspectral images by combining cascaded 3DCNN and SDTA encoding, characterized in that: The following steps are involved: Construct a cascaded 3D convolutional neural network to extract local spectral and spatial features; Dynamically update the channel dimension feature weights by splitting the deep transposed attention encoder; By combining 3D convolution with segmentation deep transposed attention encoder, features between tree species with similar spectra can be extracted and separated in the classification of tree species in mining areas; A module that fuses high-level features and low-level features is built based on residual connections, and finally the features are flattened and input into the fully connected layer, and the softmax activation function is used to output the final classification probability.

2. According to claim 1, a method for classifying tree species in mining areas by combining cascaded 3DCNN and SDTA encoding of hyperspectral images is characterized in that: The cascaded 3D convolutional neural network extracts local spectral features and spatial features, specifically comprising the following steps: First, the original hyperspectral image is subjected to principal component dimensionality reduction; The feature bands that have undergone principal component dimensionality reduction are sequentially passed through two consecutive 3D convolutional layers to obtain output feature 1; Then, output feature 1 is passed through three layers of 3D convolutional layers to obtain output feature 2; The output feature 1 and the output feature 2 are fused and connected, and then passed through three layers of 3D convolutional layers to obtain the output feature 3; The output features 1, 2 and 3 are connected, fused and input into the next network component.

3. The method for classifying tree species in mining areas by combining cascaded 3DCNN and SDTA encoding of hyperspectral images according to claim 1 or 2, characterized in that: The algorithm of the 3D convolution is: ， in, It is Tier The feature map is at position The activation value of It is Tier The deviation of the feature map, Index All feature maps of the layer, To connect Layer feature maps and Layer and The convolution kernel of the feature map is at position The value of , , are the height, width and depth of the convolution kernel respectively.

4. The method for classifying tree species in mining areas by combining cascaded 3DCNN and SDTA encoding of hyperspectral images according to claim 1 is characterized in that: The segmentation depth transposed attention encoder includes a feature encoding module and a self-attention module. In the feature encoding module: Input Features Divided into Subset Subset The number of levels is based on Adaptive; Except for the first subset, the calculation method of each subset is to fuse the output features of the previous subset and then pass Depth-with convolution Perform feature encoding ; Finally, the features of each subset are concatenated to obtain output features with multi-scale receptive fields.

5. The method for classifying tree species in mining areas by combining cascaded 3DCNN and SDTA encoding of hyperspectral images according to claim 4 is characterized in that: Said The expression is: 。 6. The method for classifying tree species in mining areas by combining cascaded 3DCNN and SDTA encoding of hyperspectral images according to claim 4 or 5, characterized in that: In the self-attention module: The input features Modify it by reshape , after three lines, we get the query ,key ,value , before calculating the cross-covariance attention, by , application Norm-stable training; Only for Transpose, get , the calculation results are combined with softmax to obtain the attention score matrix, which is applied to Get the final attention map.

7. The method for classifying tree species in mining areas by combining cascaded 3DCNN and SDTA encoding of hyperspectral images according to claim 6 is characterized in that: The self-attention module comprises: ， ， in, For input, is the output tensor.

8. The method for classifying tree species in mining areas by combining cascaded 3DCNN and SDTA encoding of hyperspectral images according to claim 1 is characterized in that: The module for building a fusion of high-level features and low-level features based on residual connection specifically includes the following steps: The features of the segmented deep transposed attention output are passed through a 3D convolutional layer for preliminary feature extraction and batch normalization. Then, they are passed through a 3D convolutional layer with a larger convolution kernel to capture deeper structural information and then batch normalized. Then, they are passed through another 3D convolutional layer and batch normalized again to obtain deep high-level features. The parallel convolution branches start from the previous feature extraction and go through 3D convolution and batch normalization in sequence, and finally add the output of the deep feature extraction branch to form a residual connection; The final output is converted into a one-dimensional vector through the flattening layer and input into the fully connected layer for classification; The output layer using a softmax activation function produces a classification decision.

9. The method for classifying tree species in mining areas by combining cascaded 3DCNN and SDTA encoding of hyperspectral images according to claim 2 is characterized in that: The sizes of the 3D convolutional layers are different.

10. The method for classifying tree species in mining areas by combining cascaded 3DCNN and SDTA encoding of hyperspectral images according to claim 2 is characterized in that: The first step of performing principal component dimensionality reduction on the original hyperspectral image is to retain the first 30 principal components through spectral feature dimensionality reduction.

Images