Global learning device and method for hyperspectral image classification

Abstract

The present invention provides a global learning device and method for hyperspectral image classification. The device includes: an encoder and a decoder; the encoder includes a spectral dimension adjustment layer, a first feature extraction layer and a second feature extraction layer in sequence according to the image processing order; the first feature extraction layer includes three MLBSA structure layers stacked together; the MLBSA structure layer includes three shuffled spectral attention SSA modules, two MLB layers and a downsampling layer; the input of the first SSA module is added and fused with the output of the downsampling layer after passing through the Zero‑padded convolution module, and the output is used as the output of the MLBSA structure layer; the decoder includes a first upsampling layer, a Concat layer and an output layer in sequence according to the image processing order; the output of the first MLBSA structure layer in the first feature extraction layer is used as the input of the first upsampling layer, the output of the second feature extraction layer and the output of the first upsampling layer are fused through the Concat layer, and the fusion result is processed by the output layer to complete the classification of the hyperspectral image.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image classification, and in particular to a global learning device and method for hyperspectral image classification. Background Art

[0002] Hyperspectral imaging technology can simultaneously detect the two-dimensional geometric spatial information and one-dimensional continuous spectral information of the target object, making the hyperspectral image have the characteristics of "image and spectrum in one". Geometric spatial information can reflect the size, shape and other external features of the target object, while spectral information can reflect the internal physical structure and chemical composition of the target object. Therefore, hyperspectral remote sensing is widely used in rock and mineral detection, marine plant detection, water resource application and land resource utilization.

[0003] How to build a more accurate and effective classification method is a key issue in the application of hyperspectral remote sensing technology. Traditional classification algorithms, such as support vector machine (SVM), three-dimensional wavelet transform, Gaussian mixture, etc., usually use band selection and feature extraction to reduce the dimension of the original image and project the image into a low-level feature space. These methods often change the band correlation of the original image, lose some spectral information, and cannot fully extract the abstract features in the hyperspectral image, thus affecting the classification accuracy.

[0004] In recent years, with the application and development of deep learning technology, algorithm models based on convolutional neural networks (CNNs) have been widely used in image classification (Lecun Y, Bottou L. Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE, 1998, 86 (11): P. 2278-2324.), speech recognition, target detection, image semantic segmentation and other fields. CNNs have shown strong feature extraction capabilities. More and more researchers use CNNs to replace traditional classification methods and apply them to hyperspectral image classification. The current CNN-based classification model is evolving towards a complex structure with deeper or wider layers. Although good results have been achieved to a certain extent, the depth means that the network model has more parameters, which not only increases the computational overhead and slows down the classification speed, but also places higher requirements on computer hardware. Summary of the invention

[0005] In order to improve the classification accuracy and classification speed of hyperspectral images, the present invention provides a global learning device and method for hyperspectral image classification.

[0006] On the one hand, the present invention provides a global learning device for hyperspectral image classification, comprising: an encoder and a decoder; the encoder comprises a spectral dimension adjustment layer, a first feature extraction layer and a second feature extraction layer in sequence according to the image processing order; the first feature extraction layer comprises three MLBSA structure layers stacked together; the MLBSA structure layer comprises three shuffled spectral attention SSA modules, two MLB layers and a downsampling layer; wherein the SSA modules and the MLB layers are cross-stacked with each other; the downsampling layer serves as the last sublayer of the MLBSA structure layer; the input of the first SSA module passes through a Zero-padded convolution module and then performs an addition fusion operation with the output of the downsampling layer, and the output of the MLBSA structure layer is used as the output; the MLB layer represents an improved linear bottleneck layer;

[0007] The decoder includes a first upsampling layer, a Concat layer and an output layer in the order of image processing; the output of the first MLBSA structure layer in the first feature extraction layer is used as the input of the first upsampling layer, the output of the second feature extraction layer and the output of the first upsampling layer are fused through the Concat layer, and the fusion result is processed by the output layer to complete the classification of the hyperspectral image.

[0008] Furthermore, the spectral dimension adjustment layer includes three sub-layers, which are SSA module, 1×1 convolution layer and MLB layer from shallow to deep.

[0009] Furthermore, the MLB layer includes a first convolution module, a second convolution module and a third convolution module stacked together in sequence; wherein the first convolution module and the second convolution module are composed of a convolution layer, a GN layer and a ReLU layer from shallow to deep layers; and the third convolution module includes a convolution layer and a GN layer in sequence.

[0010] Furthermore, the second feature extraction layer includes an SSA module, a first branch extraction layer for extracting global information, a second branch extraction layer for extracting local information, a feature fusion layer and a second upsampling layer; the output of the SSA module passes through the first branch extraction layer and the second branch extraction layer respectively; the outputs of the two branch extraction layers are fused through the feature fusion layer, and the output of the feature fusion layer after passing through the second upsampling layer is used as the output of the second feature extraction layer.

[0011] Furthermore, the first branch extraction layer includes two connected cross-attention CCA modules.

[0012] Furthermore, the second branch extraction layer adopts a dilated spatial pyramid pooling ASPP structure.

[0013] Furthermore, the output layer includes three sub-layers, which are two fourth convolution modules and a 1×1 convolution layer from shallow to deep layers; the fourth convolution module is composed of a convolution layer, a GN layer and a ReLU layer from shallow to deep layers.

[0014] On the other hand, the present invention provides a hyperspectral image classification method based on the above-mentioned device, comprising:

[0015] The general global random stratified UGSS sampling strategy is adopted to divide the data set into a training set, a validation set and a test set; the data set refers to the set consisting of all ground object sample data extracted from the hyperspectral image after ground object samples are extracted;

[0016] Using a training set and a validation set to train the device described in any one of claims 1 to 7 to obtain a trained classification model;

[0017] The classification model is used to classify the test set.

[0018] Furthermore, the SSA module processes the input data using formula (1), formula (2) and formula (3) in sequence:

[0019]

[0020] s c =F ex (z c ,W)=W 2 (ReLU((GN(W 1 (Shuffle(z c )))))) (2)

[0021]

[0022] Among them, z c It represents the value after encoding all the pixel values ​​of the cth band. H and W represent the height and width of the hyperspectral image respectively. u c (i,j) represents the pixel at the i-th row and j-th column in the c-th band. Shuffle represents the Shuffle function, which is used to perform the operation of shuffling the spectral dimension to increase interactivity. W 1 and W 2 represents two fully connected layers, s c represents the intermediate output of the cth band after being processed by the SSA module, Represents the final output of the cth band after processing by the SSA module.

[0023] Beneficial effects of the present invention:

[0024] (1) Based on the traditional linear bottleneck, an improved linear bottleneck MLB is designed according to the characteristics of the hyperspectral image. Based on this, the MLBSA structure layer is designed. By stacking the MLBSA structure for feature extraction, the feature information of the hyperspectral image can be fully extracted.

[0025] (2) A second feature extraction layer with a dual-branch structure is used at the end of the encoder to fully extract local and global spatial information; and in conjunction with the SS (shuffle mechanism) attention mechanism layer in the entire learning device, the spatial-spectral information of the hyperspectral image can be fully extracted;

[0026] (3) A quick connection is added after the first MLBSA structure layer to upsample it and combine it with the upsampled output in the second feature layer, making full use of low-level features and high-level features;

[0027] (4) Different from the classification method using data cube as input, the classification method of the present invention adopts a general global random stratified UGSS sampling strategy. Each time a complete image is input into the classification model, it can fully extract and integrate the global spatial-spectral information, and has a faster classification speed while ensuring higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of the structure of a global learning device for hyperspectral image classification provided by an embodiment of the present invention;

[0029] Figure 2 A structural diagram of the MLBSA structural layer provided by an embodiment of the present invention;

[0030] Figure 3 A structural diagram of an SSA module provided in an embodiment of the present invention;

[0031] Figure 4 A diagram showing the classification results of different models on an IP dataset provided by an embodiment of the present invention;

[0032] Figure 5 A diagram showing the classification results of different models on a PU dataset provided by an embodiment of the present invention;

[0033] Figure 6 A diagram showing the classification results of different models on the SA dataset provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] Example 1

[0036] Combination Figure 1 and Figure 2 As shown, the present invention provides a global learning device for hyperspectral image classification, comprising: an encoder and a decoder;

[0037] The encoder includes a spectral dimension adjustment layer, a first feature extraction layer and a second feature extraction layer in the order of image processing; the first feature extraction layer includes three MLBSA structure layers stacked together; the MLBSA structure layer includes three shuffle spectral attention SSA modules, two MLB layers and a downsampling layer; wherein the SSA modules and the MLB layers are cross-stacked with each other; the downsampling layer serves as the last sublayer of the MLBSA structure layer; the input of the first SSA module passes through a Zero-padded convolution module and then performs an addition fusion operation with the output of the downsampling layer, and the output of the MLBSA structure layer is used as the output of the MLBSA structure layer; the MLB layer represents a modified linear bottleneck layer; the function of the downsampling layer is to extract abstract features and save computing resources; MLBSA: Modified Linear Bottlenand spectralattention; SSA: Shuffle spectral Attention; MLB: Modified Linear Bottlen.

[0038] The decoder includes a first upsampling layer, a Concat layer and an output layer in the order of image processing; the output of the first MLBSA structure layer in the first feature extraction layer is used as the input of the first upsampling layer, the output of the second feature extraction layer and the output of the first upsampling layer are fused through the Concat layer, and the fusion result is processed by the output layer to complete the classification of the hyperspectral image.

[0039] In the embodiment of the present invention, feature extraction is performed by stacking the MLBSA structure, and a shortcut connection is added after the first MLBSA structure layer to upsample it and combine it with the output of the second feature extraction layer in the encoder, thereby making full use of low-level features and high-level features.

[0040] In addition, due to the existence of the downsampling layer, the dimensions of the input and output of the MLBSA structure layer are different, so a Zero-padded convolution module is used for the residual structure.

[0041] As an implementation method, Figure 1 As shown in Figure 1, the spectral dimension adjustment layer includes three sub-layers, which are SSA module, 1×1 convolution layer and MLB layer from shallow to deep.

[0042] Specifically, a spectral dimension adjustment layer is used to perform preliminary processing on the input hyperspectral image. Compared with the traditional convolution from high dimension to low dimension, the spectral dimension adjustment layer in the embodiment of the present invention uses a 1×1 convolution layer to perform an expansion operation, mapping the low-dimensional feature space to the high-dimensional space without reducing the tensor dimension.

[0043] As an implementation method, Figure 2 As shown, the MLB layer includes a first convolution module, a second convolution module, and a third convolution module stacked together in sequence; wherein the first convolution module and the second convolution module are composed of a convolution layer, a GN layer, and a ReLU layer from shallow to deep layers; and the third convolution module includes a convolution layer and a GN layer in sequence. GN: Group Normalization, group normalization.

[0044] In the structural design of the MLB layer, when mapping data from low-dimensional space to high-dimensional space, a linear activation function is not used like the traditional method, but a nonlinear activation function ReLU layer is used. This is because: The high dimensionality of the hyperspectral image itself; after mapping the low-dimensional tensor to the high-dimensional tensor, its own dimension is already high enough, and the use of nonlinear activation functions at this time will not lose too much information. At the same time, considering that in the process of feature extraction, using high-dimensional tensors to extract features can extract enough information, so that the extracted features are more conducive to classification, therefore, in the third convolution module with low-dimensional characteristics, if nonlinear activation functions are still used, more useful information will be lost, so no activation function is used in the third convolution module.

[0045] As an implementation method, Figure 1 As shown, the second feature extraction layer includes an SSA module, a first branch extraction layer for extracting global information, a second branch extraction layer for extracting local information, a feature fusion layer and a second upsampling layer; the output of the SSA module passes through the first branch extraction layer and the second branch extraction layer respectively; the outputs of the two branch extraction layers are fused through the feature fusion layer, and the output of the feature fusion layer after passing through the second upsampling layer is used as the output of the second feature extraction layer.

[0046] In an embodiment of the present invention, the first branch extraction layer includes two connected CCA modules; the second branch extraction layer adopts an ASPP structure. The CCA module is a simplification of the non-local module and can extract spatial information from the global perspective; ASPP extracts local spatial information from receptive fields of different sizes. CCA: Criss-cross attention; the module structure can refer to "Z. Huang, X. Wang, L. Huang, C. Huang, Y. Wei, and W. Liu," CCNet: Criss-Cross Attention for Semantic Segmentation", in Proceedings of the IEEE / CVF International Conference on Computer Vision, 2019, pp. 603-612.". ASPP: AtrousSpatial Pyramid Pooling, Atrous Spatial Pyramid Pooling; the structure can refer to "L.-C.Chen,G.Papandreou,I.Kokkinos,K. Murphy,ALJItopaYuille,and m.intelligence,"Deeplab: Semantic image segmentation with deep convolutional nets,atrousconvolution,and fully connected crfs,"IEEE Transactions on Pattern Analysisand Machine Intelligence,vol.40,no.4, pp.834-848,2017.".

[0047] By adopting a dual-branch extraction layer in the second feature extraction layer, local and global spatial information can be fully extracted; combined with the SSA module used in the global learning device, spatial-spectral information can be fully extracted.

[0048] As an implementation method, Figure 1 As shown, the output layer includes three sub-layers, which are two fourth convolution modules and a 1×1 convolution layer from shallow to deep layers; the fourth convolution module is composed of a convolution layer, a GN layer and a ReLU layer from shallow to deep layers.

[0049] Appropriate cross-channel interaction is very important for learning high-performance and efficient channel attention. The purpose of the two FC layers of the existing SEBlock is to capture nonlinear cross-channel interactions, but due to the dimensionality reduction itself, it will have a negative impact on the effect to a certain extent. Therefore, the embodiment of the present invention designs an SSA module, which adds a shuffle operation before global pooling, that is, the spectral dimension of HSI is shuffled before dimensionality reduction, and then the spectral dimension is normalized using the GN layer. In this way, the SSA module can enable sufficient information interaction between channels, and in conjunction with the first feature extraction layer, the combination of the two can continuously learn effective spectral features and spatial features, thereby improving the effect of the model.

[0050] Example 2

[0051] On the basis of the above embodiments, the embodiments of the present invention provide a global learning device for hyperspectral image classification, and provide parameter settings of each network layer or each module of the device.

[0052] In an embodiment of the present invention, in the MLB layer, the parameters of the convolution layer of the first convolution module are set as: convolution kernel is 1, Stride is 1, and Padding is 1; the parameters of the convolution layer of the second convolution module are set as: convolution kernel is 3, Stride is 1, and Padding is 1; the parameters of the convolution layer of the third convolution module are set as: convolution kernel is 3, Stride is 1, and Padding is 0.

[0053] In the MLBSA structure layer, the structure of the downsampling layer is: a convolution layer with a convolution kernel of 3, a stride of 2, and a padding of 1. Under this parameter setting, since the convolution layer with a stride of 2 and a padding of 1 will reduce the spatial size and increase the spectral dimension, in practical applications, a 2×2 average pooling module can be added immediately after the Zero-padded convolution module to further ensure that the input and output dimensions of the MLBSA structure layer are the same so that the two can be added and fused, such as Figure 2 By using Zero-padded combined with average pooling skip connection, the normal addition operation is guaranteed without adding additional parameters.

[0054] In the ASPP structure adopted by the second branch extraction layer, the expansion factors of the dilated convolution used in the dilated spatial pyramid pooling are set to rate=12, rate=24, and rate=36, respectively.

[0055] The feature fusion layer includes a Concat layer and a convolution module consisting of a 1×1 convolution layer, a GN layer and a ReLU layer in sequence; the outputs of the two branch extraction layers are fused through the Concat layer and then pass through the convolution module, and the output of the convolution module is the output of the feature fusion layer.

[0056] The first upsampling layer and the second upsampling layer are set to 2 times upsampling and 8 times upsampling respectively.

[0057] Before the output of the first upsampling layer is fused with the output of the second feature extraction layer, the output of the first upsampling layer also passes through a convolution module consisting of a 1×1 convolution layer, a GN layer and a ReLU layer, and the output of the convolution module and the output of the second feature extraction layer are fused through the Concat layer.

[0058] In the output layer, the convolution kernels of the convolution layers in the two fourth convolution modules are both 3.

[0059] Example 3

[0060] An embodiment of the present invention provides a hyperspectral image classification method, which uses the global learning device for hyperspectral image classification in the above embodiments. The classification method includes the following steps:

[0061] S301: using a universal global random stratified UGSS sampling strategy, the data set is divided into a training set, a validation set and a test set; the data set refers to a set consisting of all ground object sample data extracted from the hyperspectral image after ground object samples are extracted;

[0062] The UGSS sampling strategy uses the entire hyperspectral image as the input of the classification model. The specific idea is: set a fixed number of training samples. If the total number of samples of a certain feature is not enough to provide this fixed number of samples, proportional extraction is used. At the same time, during training, the extracted training samples are divided into training sets and validation sets again according to the set ratio. All extracted samples are divided into groups with a specified number of parameters according to the set parameters. When inputting, the data of an entire image is input, but when updating, only the extracted samples are updated, so as to achieve the effect of layered training, ensuring that the model can converge during training, and the way of extracting samples makes the UGSS sampling strategy applicable to most training tasks.

[0063] For example, for a data set, before training the network, it is necessary to extract samples of all objects to make training sets and validation sets, and the rest are used as test sets. If 200 samples are extracted, and there are only 150 samples of a certain object, then the present invention will extract according to the set ratio. For example, if the ratio is 0.8, then 120 samples are extracted as training sets and validation sets, and the rest are used as test sets. The extracted 120 samples are then divided into training sets and validation sets according to the set ratio. The extracted training set samples are evenly distributed to each group according to the set group (a hyperparameter). It should be noted that during training, although the present invention divides the training set, validation set and test set, all samples will still be input for training each time, but only the samples corresponding to the training set will participate in the gradient descent operation, and the remaining samples will not participate.

[0064] S302: training the global learning device using the training set and the validation set to obtain a trained classification model;

[0065] Specifically, based on the training set, the stochastic gradient descent algorithm is used to train the model parameters, and the model with the smallest loss rate on the validation set is saved as the best classification model.

[0066] S303: Classify the test set using the classification model.

[0067] As an implementation method, the SSA module sequentially processes input data using formula (1), formula (2) and formula (3); Figure 3 Shows the structure of the SSA module.

[0068]

[0069] s c =F ex (z c ,W)=W 2 (ReLU((GN(W 1 (Shuffle(z c )))))) (2)

[0070]

[0071] Among them, z c It represents the value after encoding all the pixel values ​​of the cth band. H and W represent the height and width of the hyperspectral image respectively. u c (i,j) represents the pixel at the i-th row and j-th column in the c-th band. Shuffle represents the Shuffle function, which is used to perform the operation of shuffling the spectral dimension to increase interactivity. W 1 and W 2 represents two fully connected layers, sc represents the intermediate output of the cth band after being processed by the SSA module, Represents the final output of the cth band after being processed by the SSA module.

[0072] On the basis of the above embodiment, before step S301, the method further includes: performing a zero-mean normalization operation on the labeled samples of the input hyperspectral image;

[0073] Specifically, the zero-mean normalization operation can be expressed using formula (4):

[0074]

[0075] in, Represents the pixel value of the i-th row and j-th column in the feature map of the n-th band, represents the mean value of all pixels in the nth band, σ n represents the standard deviation of the pixel value of the nth band, W, H, and N represent the width, height, and total number of bands of the hyperspectral image, respectively.

[0076] In order to verify the effects of the device and method of the present invention, the present invention also provides the following experimental data.

[0077] 1. Experimental environment

[0078] Hardware equipment: CPU is Intel(R) Xeon(R) E5-2682 v4@2.50GHz, GPU is NVIDIA GeForceRTX3060;

[0079] Software platform: Python version is 3.7, Cuda version is 11.0.194, and the Pytorch version 1.8.0 deep learning framework is used to build the model structure.

[0080] 2. Experimental Dataset

[0081] In order to measure the classification effect of the present invention, three benchmark hyperspectral datasets, Indian Pines (IP), Pavia University (PU) and Salinas (SA), are selected for experimental research. The detailed information of the three datasets is shown in Table 1.

[0082] Table 1 Detailed information of the three datasets

[0083]

[0084] 3. Experimental Setup

[0085] In order to make the size of the input image meet the downsampling requirements, the spectral dimension of the input image is kept at a multiple of 16 and zero padding is used. The number of groups for group normalization is set to 16, and the number of channels after each downsampling of the group is guaranteed to be a multiple of 16.

[0086] For all experiments, the global learning device is optimized using SGD with a poly learning rate strategy, with an initial learning rate of 0.001 and multiplied by The power value is set to 0.9, the momentum is set to 0.9, no data expansion strategy is used, the fixed number of samples extracted from each data set is 200, if the total number of samples of a certain feature is insufficient, the ratio parameter is set to 0.8, and the number of samples extracted is 0.75 for the training set and 0.25 for the validation set. For the selected samples, the number of groups is set to 10. In order to evaluate the performance of the method of the present invention, three commonly used indicators are used: overall accuracy (OA), average accuracy (AA) and Kappa coefficient (Kappa).

[0087] 4. Comparison of different classification models

[0088] Several typical classification models in recent years, including SVM-RBF (Reference 1), 1D-CNN (Reference 2), M3D-DCNN (Reference 3), SSRN (Reference 4), DBDA (Reference 5) and the classification model of the present invention (abbreviated as Proposed) are selected for detailed comparison. Table 2, Table 3 and Table 4 respectively show the final classification results of different models on IP, PU and SA data sets. Among them:

[0089] Document 1: Kuo BC, Ho HH, Li CH, et al.A Kernel-Based Feature SelectionMethod for SVM With RBF Kernel for Hyperspectral Image Classification[J].Selected Topics in Applied Earth Observations and Remote Sensing,IEEE Journalof, 2014,7(1):317-326.);

[0090] Document 2: Wei H, Yangyu H, Li W, et al. Deep Convolutional Neural Networks for Hyperspectral Image Classification[J]. Journal of Sensors, 2015, 2015: 1-12.

[0091] Document 3: M.He, B.Li and H.Chen, "Multi-scale 3D deep convolutional neural network for hyperspectral image classification," 2017IEEE International Conference on Image Processing (ICIP), Beijing, 2017, pp.3904-3908, doi: 10.1109 / ICIP.2017.8297014.

[0092] Document 4: Z.Zhong, J.Li, Z.Luo, MJIToGChapman, and R.Sensing, "Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework," vol.56, no.2, pp.847-858, 2017.

[0093] Document 5: R.Li, S.Zheng, C.Duan, Y.Yang, and XJRSWang, "Classification of hyperspectral image based on double-branch dual-attention mechanism network," vol.12, no.3, p.582, 2020.

[0094] Table 2 Comparison of classification accuracy of different models on IP dataset

[0095] Feature Type SVM-RBF 1D-CNN M3D-DCNN SSRN DBDA Proposed C1 58.77 92.22 96.67 100 88.1 100 C2 77.96 81.68 86.9 98.11 98.62 99.26 C3 71.68 81.44 90.59 98.02 98.34 97.79 C4 19.93 92.97 99.73 91.62 94.22 100 C5 89.84 94.59 98.34 99.08 99.09 100 C6 97.01 97.81 99.74 99.61 98.29 100 C7 75.88 98 100 96.67 83.83 100 C8 99.13 98.67 99.93 99.93 99.96 100 C9 51.78 100 100 100 93.33 100 C10 72.46 90.4 91.27 92.71 94.45 99.75 C11 90.22 72.36 78.29 99.33 99.45 99.34 C12 73.16 91.6 95.24 98.07 98.91 99.47 C13 72.62 100 100 96.67 93.81 100 C14 97.9 91.35 95.76 99.78 99.79 100 C15 57.92 85.16 98.44 97.03 98.59 100 C16 78.59 98.33 100 90.62 85.08 100 OA (%) 82.32±0.753 84.07±1.05 88.95±1.46 98.16±0.46 98.48±0.521 99.46±0.09 AA(%) 74.05±1.71 91.66±0.95 95.68±0.64 97.33±0.955 95.24±2.277 99.73±0.03 Kappa x 100 79.38±0.872 81.44±1.14 87.07±1.66 97.83±0.543 98.20±0.615 99.35±0.1

[0096] Table 3 Comparison of classification accuracy of different models on PU dataset

[0097] Feature Type SVM-RBF 1D-CNN M3D-DCN SSRN DBDA Proposed C1 96.83 85.27 94.59 99.9 99.56 99.98 C2 97.41 88.08 96.67 99.11 99.92 99.97 C3 76.04 82.91 93.52 92.14 98.98 100 C4 87.68 96.49 98.38 99.47 97.27 99.63 C5 97.56 99.74 100 100 99.9 100 C6 76.75 91.5 97.73 94.2 96.27 100 C7 66.36 91.6 97.69 98.17 99.32 100 C8 85.76 85.15 94.18 97.4 97.1 99.96 C9 99.87 99.88 99.83 99.78 97.55 100 OA (%) 90.34±0.678 88.78±2.25 96.41±0.84 97.9±1.83 98.90±0.607 99.95±0.00 AA(%) 87.14±0.694 91.18±0.86 96.95±0.43 97.80±1.3 98.46±0.630 99.95±0.00 Kappa x 100 87.22±0.868 85.26±2.78 95.2±1.11 97.20±2.4 98.53±0.809 99.94±0.00

[0098] Table 4 Comparison of classification accuracy of different models on SA dataset

[0099]

[0100]

[0101] It can be seen from the experimental results in Tables 2 to 4 that the OA, AA and Kappa values ​​of the classification model proposed by the present invention on the three data sets are higher than those of the other classification models. The accuracy of the SVM-RBF and 1D-CNN models using only spectral feature classification is significantly lower than that of the M3D-DCNN, SSRN, DBDA and the model proposed by the present invention using only spatial-spectral feature classification, which shows that high classification accuracy cannot be achieved using only spectral feature classification. In the IP data set, the band information of objects 2 and 11 is similar. For example, in M3D-DCNN, the accuracy of the two is not ideal, and for objects 1, 7, and 9 with a small number of samples, the accuracy of the seventh object of SSRN is only 96%, and the accuracy of the three objects in DBDA is 88.1%, 83.83% and 93.33%, respectively, which are not ideal, indicating that the two lack the ability to process small sample data. The method of the present invention has achieved an accuracy of more than 99% for both objects of similar bands and small sample data, indicating that the SSA module used in the present invention can select bands more accurately, and the use of the UGSS sampling strategy also solves the problem of small samples. In addition, among all classification methods, the method of the present invention has the smallest standard deviation, which shows that the method of the present invention has very good stability.

[0102] in addition, Figure 4-Figure 6 The following diagrams show the classification results of different classification models on IP, PU and SA datasets. (a) is a false color image and (b) is a label image. Figure 4-Figure 6 It can be clearly seen that only using spectral features for classification, such as SVM-RBF and 1D-CNN models, will produce many noise points, but methods based on spatial-spectral features overcome this shortcoming, such as M3D-DCNN, SSRN, and DBDA models, and achieve better results. By comparing with the ground truth map, the classification effect of the classification model proposed in this invention is smoother.

[0103] Table 5 Comparison of parameters, training time and test time of different classification models on three datasets

[0104]

[0105] It can be seen from the data in Table 5 that, under the IP data set, the training time of the classification model of the present invention is much shorter than that of all the comparison models, which is only 223.7 seconds; in the PU data set, the classification model of the present invention still has a great advantage in training time compared with most methods. Since SVM-RBF itself is relatively simple, and the DBDA code uses a strategy of terminating training in advance, the training time of the classification model of the present invention is longer than the two, but the accuracy of the classification model of the present invention has a great advantage over the two models and reaches saturation. In the SA data set, SVM-RBF and DBDA also have a comparative advantage, but the time difference between the classification model of the present invention and the two is very close. In all data sets, the classification model of the present invention guarantees the highest classification accuracy. For all three data sets, the test time of the classification model of the present invention is the shortest, with test times of only 0.06, 0.33, and 0.21 seconds in the three data sets, respectively, indicating that the classification model of the present invention can be better applied to related work after training.

[0106] In summary, the global learning device and classification method for hyperspectral image classification proposed in the present invention ensure higher accuracy and faster classification speed in classification work. In addition, the classification model of the present invention fully extracts the features in the hyperspectral image and still shows higher classification accuracy on objects with smaller sample data.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A global learning device for hyperspectral image classification, characterized in that include: Encoder and decoder; the encoder includes a spectral dimension adjustment layer, a first feature extraction layer and a second feature extraction layer in the order of image processing; the first feature extraction layer includes three MLBSA structure layers stacked together; the MLBSA structure layer includes three shuffled spectral attention SSA modules, two MLB layers and a downsampling layer; wherein the SSA module and the MLB layer are cross-stacked with each other; the downsampling layer is used as the last sublayer of the MLBSA structure layer; the input of the first SSA module is added and fused with the output of the downsampling layer after passing through the Zero-padded convolution module, and the output of the MLBSA structure layer is used as the output; the MLB layer represents an improved linear bottleneck layer; The decoder includes a first upsampling layer, a Concat layer and an output layer in sequence according to the image processing order; the output of the first MLBSA structure layer in the first feature extraction layer is used as the input of the first upsampling layer, the output of the second feature extraction layer and the output of the first upsampling layer are fused through the Concat layer, and the fusion result is processed by the output layer to complete the classification of the hyperspectral image; The spectral dimension adjustment layer includes three sub-layers, which are SSA module, 1×1 convolution layer and MLB layer from shallow to deep; The MLB layer includes a first convolution module, a second convolution module and a third convolution module stacked together in sequence; wherein the first convolution module and the second convolution module are composed of a convolution layer, a GN layer and a ReLU layer in sequence from the shallow layer to the deep layer; the third convolution module includes a convolution layer and a GN layer in sequence; The second feature extraction layer includes an SSA module, a first branch extraction layer for extracting global information, a second branch extraction layer for extracting local information, a feature fusion layer, and a second upsampling layer; the output of the SSA module passes through the first branch extraction layer and the second branch extraction layer respectively; the outputs of the two branch extraction layers pass through the feature fusion layer for feature fusion, and the output of the feature fusion layer passes through the second upsampling layer as the output of the second feature extraction layer; The SSA module includes two processing branches, one of which includes a residual layer, and the other includes a shuffle channel layer, a global pooling layer, a first convolution module and a convolution layer stacked in sequence; wherein the input of the SSA module is respectively input to the two processing branches, and the outputs of the two processing branches are added and fused as the output of the SSA module.

2. The global learning device for hyperspectral image classification according to claim 1, characterized in that: The first branch extraction layer includes two connected cross-attention CCA modules.

3. The global learning device for hyperspectral image classification according to claim 1, characterized in that: The second branch extraction layer adopts the atrous spatial pyramid pooling ASPP structure.

4. The global learning device for hyperspectral image classification according to claim 1, characterized in that: The output layer includes three sub-layers, which are two fourth convolution modules and a 1×1 convolution layer from shallow to deep layers; the fourth convolution module is composed of a convolution layer, a GN layer and a ReLU layer from shallow to deep layers.

5. A hyperspectral image classification method based on the device according to any one of claims 1 to 4, characterized in that: include: A general global random stratified UGSS sampling strategy is adopted to divide the data set into a training set, a validation set and a test set; the data set refers to a set consisting of all ground object sample data extracted from the hyperspectral image after the ground object samples are extracted; the specific idea of ​​the UGSS sampling strategy is: set a fixed number of training samples, if the total number of samples of a certain ground object is less than the fixed number of training samples, then extract samples according to the set ratio; divide the extracted samples into a training set and a validation set, and use the remaining samples after extraction as a test set; divide all the extracted training set samples into groups with a specified number of parameters according to the set parameters, and during training, input all the ground object samples in the entire image for training each time, but only the samples corresponding to the training set will participate in the gradient descent operation, and the remaining samples will not participate; Using a training set and a validation set to train the device described in any one of claims 1 to 4 to obtain a trained classification model; The classification model is used to classify the test set.

6. The hyperspectral image classification method according to claim 5, characterized in that: The SSA module processes the input data using formula (1), formula (2) and formula (3) in sequence: （1） （2） （3） in, z c Indicates c The value after encoding all the pixel values ​​of the band, H and W Represent the height and width of the hyperspectral image, u c ( i , j ) represents c In the band i Line j Column pixels, shuffle express shuffle function, which performs operations that shuffle the spectral dimensions to increase interactivity, W 1 and W 2 represents two fully connected layers. s c Indicates that after being processed by the SSA module c The intermediate output of the band, Indicates that after being processed by the SSA module c The final output of the band.

Images