A method and system for hyperspectral image classification based on quaternion deep network

Abstract

The application discloses a hyperspectral image classification method and system based on a quaternion deep network, mainly strengthens the learning ability of the network and improves the training efficiency of the network through quaternion algebra theory and deep learning theory, and solves the problems of few hyperspectral image data samples, spectral space variability and the like.The implementation scheme is as follows: firstly, considering the high dimensionality of data, dimensionality reduction is carried out through principal component analysis.Next, a deep attention module is constructed, important features are strengthened and noise interference is suppressed, so that the quality of data is improved.Then, a multi-branch network is constructed, the strengthened features are grouped, and a quaternion generator is designed to map the hyperspectral image data to a quaternion space for processing.Meanwhile, a multi-scale quaternion attention module is embedded to further strengthen the feature expression capability.Finally, the features of each branch are fused, and high-precision classification of the hyperspectral image is realized.

Description

Technical Field

[0001] This invention belongs to the field of hyperspectral remote sensing image processing technology, specifically relating to a hyperspectral image classification method based on quaternion deep networks. Background Technology

[0002] Hyperspectral imaging (HSI), based on advanced spectral imaging techniques, can simultaneously acquire spatial and spectral information of a target. Because HSI contains hundreds of narrow, continuous bands of data, it contains richer spectral information and has been widely used in many fields, such as mineral exploration and mapping, urban planning, land use surveys, and military reconnaissance. Among the many applications of HSI, classification is a fundamental and crucial task, aiming to assign a label to each sample to accurately identify different categories of substances.

[0003] Early research on HSI classification primarily relied on traditional machine learning methods such as Support Vector Machines, K-Nearest Neighbors, Random Forests, and Logistic Regression. These traditional methods utilize spectral information to achieve HSI classification. However, when spectral features are disturbed, relying solely on spectral information often fails to accurately identify target samples. To address this issue, fully utilizing the spatial information of HSI can effectively improve algorithm performance, leading to the proposal of several classification methods based on spectral and spatial features. While these methods have improved classification accuracy to some extent, most traditional machine learning methods focus on extracting shallow features, making it difficult to achieve superior classification results.

[0004] In recent years, with the rise of deep learning, deep learning-based methods have surpassed the performance of traditional methods, significantly improving the classification accuracy of HSI. Examples include Long Short-Term Memory networks, Convolutional Neural Networks, and Graph Convolutional Networks. While these deep learning-based networks have achieved good classification performance, factors such as the limited number of labeled HSI samples and high data dimensionality lead to problems during network learning, including a lack of training samples, a large number of parameters, and low training efficiency.

[0005] Quaternions, with their superior algebraic structure and operational properties, can effectively represent data and facilitate information exchange. While preliminary research results have been achieved in the field of hyperspectral imaging, the theoretical basis of quaternions is still immature and has not been well integrated with deep networks to fully leverage their advantages. Therefore, there is an urgent need to research and develop high-performance HSI classification methods based on quaternion theory to address the limitations of deep learning algorithms in HSI applications. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention aims to enhance the learning ability of networks, acquire key and information-rich features, improve network learning efficiency, and thus significantly improve HSI classification accuracy. The main contents are as follows: Dimensionality reduction is performed on HSI, and a depth-wise attention network is designed to process the data, improving data quality; a high-performance multi-branch network is constructed based on quaternions to map HSI data to the quaternion space for processing, effectively enhancing information interaction and reducing information loss and the number of parameters; a parallel strategy is adopted in the multi-branch structure to process data, improving network training efficiency, while embedding a multi-scale quaternion attention module to further enhance the network's learning ability while increasing feature richness.

[0007] The technical solution adopted in this invention is a hyperspectral image classification method based on quaternion deep networks, comprising the following steps:

[0008] S1: Perform data preprocessing on the hyperspectral image HSI;

[0009] S2: Construct a depth-wise attention module to obtain high-quality image feature data;

[0010] S3: Build a multi-branch network, group the image feature data, and design a quaternion generator to transform the data and obtain HSI data in quaternion form;

[0011] S4: Construct a multi-scale quaternion attention module, embed the multi-scale quaternion attention module into each branch of the multi-branch network, and use the multi-scale strategy to obtain information-rich multi-scale features. At the same time, the features are enhanced through the quaternion attention layer.

[0012] S5: The enhanced features are fused, and finally a classification layer is used to classify the hyperspectral image.

[0013] Furthermore, the data preprocessing in step S1 includes data dimensionality reduction and extraction of data cube blocks.

[0014] Furthermore, the specific implementation of step S1 is as follows:

[0015] First, principal component analysis (PCA) was used to analyze the HSI data. Dimensionality reduction is performed to obtain the dimensionality-reduced HSI data. H, W, C, and C′ represent the height, width, number of bands, and number of bands after dimensionality reduction of the HSI, respectively.

[0016] Then, extract cube blocks centered on each pixel of X′. As subsequent input, S represents the spatial size of the cube block.

[0017] Furthermore, the specific processing procedure of the depth-wise attention module (DWAM) in step S2 is as follows:

[0018] In the channel attention module CA, the input cube block F in F′ is obtained by convolution along the spectral dimension. in Cube S represents the spatial size of the cube, and C′ represents the number of bands in the HSI hyperspectral image after dimensionality reduction; subsequently, F′ in Average pooling (AvgPool) and max pooling (MaxPool) operations are performed respectively. The resulting vectors are then passed through a shared multilayer perceptron (MLP), and the features output by the MLP are summed. Finally, a nonlinear mapping is performed to obtain the attention weights F. spe Finally, F spe With F in The weighted feature map F′1 is obtained by performing element-wise multiplication. The specific calculation formula is as follows:

[0019] F in ′=Conv 1×1 (F in )

[0020] F spe =σ1(MLP(AvgPool(F) in ′))+MLP(MaxPool(F in ′)))

[0021] F′1=F spe ☉F in

[0022] Where Conv 1×1 (·) represents a two-dimensional convolution operation with a kernel size of 1×1; σ1 represents the Sigmoid activation function; ⊙ represents element-wise multiplication;

[0023] Next, F′1 is processed by depthwise convolution along the spectral direction using the spatial attention module SA to obtain F′2. Here, F′2 is processed by three different convolution kernels in parallel to obtain three feature maps, which are then convolved with F′1. ′ The weighted feature map is obtained by adding the two features together, and then a 1×1 convolution is used to perform channel mixing on the weighted feature map to generate the spatial spectral attention feature F. spa The entire process of the Spatial Attention (SA) module is represented as follows:

[0024]

[0025] Among them, DwConv j (·), j∈{1,2,3} represents the depthwise attention convolution operation through the j-th convolution kernel; then, Fspa and F in Element-wise multiplication is performed, and the features are further integrated through a 1×1 convolution operation to obtain the final feature map. The calculation formula is as follows:

[0026] F′3=Conv 1×1 (F spa ☉F in ).

[0027] Furthermore, the specific implementation process of step S3 is as follows:

[0028] The image feature data F′3 is divided into band groups to construct a multi-branch network framework. The grouping includes dividing F′3 into N groups, denoted as... The first g bands are selected as the features G1 of the first group, where g = C′ / N, g+1 to 2g are the features G2 of the second group, and so on.

[0029] The first set of features G1 is used as the input to the first branch; the second set of features G2 is added element-wise to G1 to obtain G′2, which is used as the input to the second branch; and so on, G... i With G′ i-1 G′ is obtained by adding elements one by one. i , as the input to the i-th branch;

[0030] In each branch, the HSI data is mapped to the quaternion space and processed using a quaternion generator. Let the i-th branch be G′. i The characteristic of the kth band of (i = 1, 2, ..., N) is: right Perform a two-dimensional convolution, and sum the outputs of each band to obtain the real part of the quaternion. As shown in the following formula:

[0031]

[0032] Where σ² is the Gaussian error linear unit GELU activation function; Conv(·) is a two-dimensional convolution operation; and simultaneously, along the spectral dimension, G′ is... i Perform a one-dimensional convolution operation to obtain a three-dimensional tensor And treat them as the three imaginary parts of the quaternion; obtain F I The formula is as follows:

[0033] F I =σ2(Conv(G′) i ))

[0034] Where Conv(·) represents a one-dimensional convolution operation, converting F... R and F IBy combining them, we obtain HSI data in quaternion form.

[0035] Furthermore, in step S4, the specific implementation of the multi-scale quaternion attention module MSQA is as follows:

[0036] In the i-th branch, HSI data in quaternion form Centered on the central pixel, select m pixels with scales λ1, λ2, ..., λm. m (λ i A cube block with dimensions ≤ S, the l-th dimension is denoted as

[0037] cube The input is fed into the quaternion convolution module (QCM) to obtain the feature map. QCM includes quaternion convolution QConv, max pooling MaxPool, quaternion batch normalization QBN, and quaternion activation operations.

[0038] Subsequently, Perform average pooling (AvgPool), two consecutive quaternion fully connected layers (QFC), and non-linear mapping operations to output weights. And and Element-wise multiplication yields Then obtain through QCM

[0039] Finally, the features at each scale The output is obtained through a shared multilayer perceptron (MLP).

[0040] Furthermore, the feature extraction process of the quaternion convolution module MSQA is as follows:

[0041]

[0042]

[0043] Where σ1 represents the Sigmoid activation function; σ3 is the ReLU activation function.

[0044] Furthermore, the Multilayer Perceptron (MLP) consists of a quaternion fully connected layer (QFC), a fully connected layer (QF), a dropout operation, and an activation operation.

[0045] Furthermore, the specific operations for feature fusion and classification in step S5 are as follows:

[0046] In each branch, features at different scales The output M is obtained by concatenation.i The formula is shown below:

[0047]

[0048] The semicolon ";" indicates a matrix concatenation operation.

[0049] Then, the features of all branches are merged, as shown below:

[0050] F = [M1; M1; ...; M N ]

[0051] Where N represents the number of branches in a multi-branch network;

[0052] Finally, softmax is used to implement classification.

[0053] The present invention also provides a hyperspectral image classification system based on quaternion deep networks, comprising:

[0054] The processor and memory are used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a hyperspectral image classification method based on a quaternion deep network as described in the above technical solution.

[0055] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows:

[0056] 1) This invention constructs a multi-branch network that can extract rich features while processing data in parallel, improving training efficiency. At the same time, the input of each branch is passed to the next branch, making full use of important information and reducing loss during information transmission.

[0057] 2) By constructing the DWAN module, the quality of the data is improved, which facilitates the subsequent learning of features and thus improves the classification accuracy.

[0058] 3) Mapping HSI data to quaternion space and utilizing the algebraic structure and operational properties of quaternions effectively reduces the number of parameters and enhances the interaction between information.

[0059] 4) The embedded MSQA further enhances important features, and the multi-scale strategy adopted enables the extracted features to more accurately characterize homogeneous regions, allowing the model to learn richer spatial information. Attached Figure Description

[0060] Figure 1 This is a diagram of the Multi-Branch Quaternion Attention Network (MQAN) framework in an embodiment of the present invention;

[0061] Figure 2 This is a flowchart of the Depth-by-Depth Attention Module (DWAM) in an embodiment of the present invention;

[0062] Figure 3 This is a design diagram of the quaternion generator (QG) in an embodiment of the present invention;

[0063] Figure 4 This is a diagram of the multi-scale quaternion attention (MSQA) module in an embodiment of the present invention;

[0064] Figure 5 This illustrates the impact of the number of principal components on the algorithm in this embodiment of the invention.

[0065] Figure 6 This illustrates the impact of scale size and number on the algorithm in this embodiment of the invention;

[0066] Figure 7 This is a flowchart of an embodiment of the present invention. Detailed Implementation

[0067] To demonstrate the technical solution and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0068] The Multi-Branch Quaternion Attention Network (MQAN) mainly consists of PCA, Depthwise Attention Module (DWAM), Quaternion Generator (QG), and Multi-Scale Quaternion Attention (MSQA) module, such as... Figure 1 As shown, considering the high dimensionality of the data, the original HSI is first reduced in dimensionality. Simultaneously, to improve data quality, DWAM is used to augment the data. Then, QG is used to transform the data to quaternion space. Next, a multi-branch network is constructed to process the data in parallel. Furthermore, MSQA is embedded in each branch to obtain rich information and better characterize the diversity of homogeneous regions. Finally, the multi-scale features extracted from each branch are fused and input into a Softmax classifier for classification.

[0069] like Figure 7 As shown in the figure, the present invention provides a hyperspectral image HSI classification method based on quaternion deep networks, which specifically includes the following steps:

[0070] S1: Preprocessing operations such as dimensionality reduction and data cube extraction are performed on the original hyperspectral image HSI data. The specific methods are as follows:

[0071] S1.1: Using Principal Component Analysis (PCA) on HSI data Perform dimensionality reduction to obtain the dimensionality-reduced data. H, W, C, and C′ represent the height, width, number of bands, and number of bands after dimensionality reduction of the HSI, respectively.

[0072] S1.2: Extract cube blocks centered on each pixel of X′. As input to the network, S represents the spatial size of the cube block.

[0073] S2: Construct a depth-wise attention module (DWAM) to effectively enhance important information and suppress noise interference, thereby improving the quality of the acquired HSI data, such as... Figure 2 As shown. The specific approach taken in S2 is as follows:

[0074] S2.1: In the Channel Attention (CA) module, F in F′ is obtained by convolution along the spectral dimension. in Subsequently, regarding F′ in Average pooling (AvgPool) and max pooling (MaxPool) operations are performed respectively. The resulting vectors are then passed through a shared multilayer perceptron (MLP), and the features output by the MLP are summed. Finally, a nonlinear mapping is performed to obtain the attention weights F. spe Finally, F spe With F in The weighted feature map F′1 is obtained by performing element-wise multiplication. The calculation formula for this process is as follows:

[0075] F in ′=Conv 1×1 (F in )

[0076] F spe =σ1(MLP(AvgPool(F) in ′))+MLP(MaxPool(F in ′)))

[0077] F′1=F spe ⊙F in

[0078] Where Conv 1×1 (·) represents a two-dimensional convolution operation with a kernel size of 1×1; σ1 represents the Sigmoid activation function; ⊙ represents element-wise multiplication.

[0079] S2.2: Next, F′1 is convolved along the spectral direction using the spatial attention (SA) module to obtain F′2. Here, F′2 is convolved in parallel with three different kernels to obtain three feature maps, which are then added to F′2 to obtain a weighted feature map. A 1×1 convolution is then used to channel-mix the weighted feature map to generate the spatial-spectral attention feature F. spa The entire SA module process is as follows:

[0080]

[0081] Among them, DwConvj (·),j∈{1,2,3} represents the depthwise attention convolution operation through the j-th convolution kernel.

[0082] S2.3: After that, F spa With F in Element-wise multiplication is performed, and the features are further integrated through a 1×1 convolution operation to obtain the final feature map. The calculation formula is as follows:

[0083] F′3=Conv 1×1 (F spa ⊙F in ).

[0084] S3: Construct a multi-branch network, group F′3, and design a quaternion generator (QG) for data transformation, such as... Figure 3 As shown. S3 is implemented in the following steps:

[0085] S3.1: Divide F′3 into N groups, denoted as The first g (g = C′ / N) bands are selected as the features G1 of the first group, g+1 to 2g are the features G2 of the second group, and so on.

[0086] S3.2: The first set of features G1 is used as the input to the first branch; the second set of features G2 is added element-wise to G1 to obtain G′2, which is used as the input to the second branch; similarly, G... i With G′ i-1 G′ is obtained by adding elements one by one. i , which serves as the input for the i-th branch.

[0087] S3.3: In each branch, the HSI data is mapped to the quaternion space using QG for processing. Let the i-th branch be G′. i The characteristic of the kth band of (i = 1, 2, ..., N) is: right Perform a two-dimensional convolution, and sum the outputs of each band to obtain the real part of the quaternion. As shown in the following formula:

[0088]

[0089] Where σ2 is the Gaussian error linear unit (GELU) activation function; Conv(·) is a two-dimensional convolution operation.

[0090] S3.4: Simultaneously, along the spectral dimension, G′ i Perform a one-dimensional convolution operation to obtain a three-dimensional tensor And treat these as the three imaginary parts of the quaternion. Obtain F I The formula is as follows:

[0091] F I =σ2(Conv(G′) i ))

[0092] Where Conv(·) represents a one-dimensional convolution operation.

[0093] S3.5: F R and F I By combining these elements, we obtain the HSI data Q in quaternion form. i .

[0094] S4: Design a multi-scale quaternion attention (MSQA) module, such as Figure 4 As shown. The specific implementation is as follows:

[0095] S4.1: In the i-th branch, with Centered on the central pixel, select m pixels with scales λ1, λ2, ..., λm. m (λ i A cube block with dimensions ≤ S, the l-th dimension is denoted as

[0096] S4.2: Move the cube block The input is fed into a quaternion convolutional module (QCM) to obtain the feature map. QCM includes quaternion convolution (QConv), MaxPool, quaternion batch normalization (QBN), and quaternion activation operations.

[0097] S4.3: Subsequently, Perform AvgPool operations, two consecutive quaternion fully connected layers (QFC), and nonlinear mapping operations to output weights. And and Element-wise multiplication yields Then obtain through QCM The feature extraction process of the MSQA module is shown below:

[0098]

[0099]

[0100] Where σ1 represents the Sigmoid activation function; σ3 is the ReLU activation function.

[0101] S4.4: Finally, combine the features at each scale. The output is obtained through a shared MLP. Here, MLP consists of a quaternion fully connected layer (QFC), a fully connected layer (QF), a dropout operation, and an activation operation.

[0102] S5: Fuse features and achieve hyperspectral image classification, as detailed below:

[0103] S5.1: In each branch, features at different scales The output M is obtained by concatenation. i The formula is shown below:

[0104]

[0105] The semicolon ";" indicates a matrix concatenation operation.

[0106] S5.2: Then, merge the features of all branches as follows:

[0107]

[0108] S5.3: Finally, the model is trained using the cross-entropy loss function, and classification is achieved using softmax.

[0109] To verify the effectiveness of the present invention, the following experiment was conducted:

[0110] In the embodiments of this invention, two publicly available datasets, PU and SV, are used for experiments to evaluate the effectiveness of the model. The PU dataset is an HSI dataset with a size of 610×340 and 103 channels, while the SV dataset is an HSI dataset with a size of 512×217 and 204 channels. Both datasets are divided into training, validation, and test sets. In PU, the training and validation sets account for 20% and 10% of the total samples, respectively, with the remaining samples used as the test set. In SV, the training and validation sets account for 10% and 10% of the total samples, respectively, with the remaining 80% used as the test set.

[0111] To comprehensively evaluate the model's performance, we use overall accuracy (OA), average accuracy (AA), and Kappa coefficient as evaluation metrics. OA reflects the proportion of samples correctly classified by the model; AA evaluates the model's performance on each class; and Kappa is used to evaluate the extent to which the model's performance exceeds the random level.

[0112] This invention uses PyCharm programming software and the Adam optimizer to train the model within the PyTorch deep learning framework. Here, the initial learning rate is set to 0.001, the batch size to 64, the training epochs to 100, and the average of ten experimental results is calculated to reduce errors caused by random sampling.

[0113] Furthermore, we analyzed three parameters—the number of principal components, the scale size, and their quantity—to determine the optimal parameters. First, Figure 5 This demonstrates the impact of the number of principal components on algorithm performance. From... Figure 5 It can be seen that as the number of principal components increases, the feature information becomes richer, and the operational accuracy (OA) improves accordingly. However, once the number of principal components reaches a certain level, the upward trend tends to level off because subsequent principal components contain less information. At the same time, a larger number of principal components also significantly increases the computational cost. Therefore, the number of principal components was set to 30 in both datasets.

[0114] The effects of scale size and number on algorithm performance are as follows: Figure 6 (a) and Figure 6 As shown in (b) in the diagram. From Figure 6 (a) It can be seen that the classification accuracy of both datasets first increases and then decreases as the scale increases. This is because when the scale is too small, it cannot effectively cover homogeneous regions; while when the scale is too large, it will include outlier pixels, thus interfering with feature learning, and the computational cost will also increase significantly. Therefore, in the PU dataset and SV dataset, the size of the regions is determined within the ranges of {7,13,15,19} and {7,11,13,17}, respectively. Furthermore, due to the diversity of shapes and sizes of the homogeneous regions in HSI, we select regions of different sizes as pixel learning features to achieve accurate characterization of homogeneous regions. Figure 6 (b) The impact of the number of scales on OA was investigated. Experimental results showed that OA was relatively high when 3 and 4 scales were selected, but the difference in accuracy between the two was not significant. Moreover, the more scales there were, the more computational cost increased significantly. Therefore, three scales were selected for experiments on both the PU and SV datasets, with sizes {7,13,19} and {7,11,13}, respectively.

[0115] To analyze the performance of each module, this invention conducted ablation experiments to study the impact of multi-branch architecture (MB), multi-scale strategy (MS), depthwise attention module (DWAM), and quaternion attention layer (QAL) on algorithm performance. Tables 1 and 2 show the experimental results on the PU and SV datasets. Here, the baseline network refers to a network that does not include BG, MS, DWAM, and QAL.

[0116] Table 1 shows the impact of each module on network performance in the PU dataset.

[0117]

[0118] Table 2 shows the impact of each module on network performance on the SV dataset.

[0119]

[0120] As shown in Table 1, after adopting MB, the network's OA, AA, and Kappa improved by 3.97%, 7.77%, and 5.72% respectively compared to the baseline. This indicates that MB can effectively extract rich features and improve classification performance. Furthermore, the introduced attention network also increased classification accuracy to some extent. However, the OA of the DWAM network was 7.75% higher than that of the QAL network. When all modules were present simultaneously, the network achieved the best classification performance, with OA, AA, and Kappa reaching 99.71%, 99.44%, and 99.62% respectively. On the other hand, MQAN yielded the smallest standard deviation, further highlighting the algorithm's stability.

[0121] As shown in Table 2, the classification performance of the network improved by 59.81% after introducing the multi-branch, multi-scale network framework. This is because the MB structure can fully utilize spectral features to optimize classification performance, while the MS strategy can acquire rich information and effectively depict the diversity of homogeneous regions, thus improving classification results. Furthermore, the OA of using DWAM is 4.55% higher than that of QAL, demonstrating that high-quality image data can significantly improve classification accuracy. Moreover, MQAN has the smallest standard deviation, indicating better robustness of the algorithm.

[0122] On the other hand, embodiments of the present invention also provide a hyperspectral image classification system based on quaternion deep networks, comprising:

[0123] The processor and memory are used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a hyperspectral image classification method based on a quaternion deep network as described in the above technical solution.

[0124] It should be understood that the above description of the embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art can make substitutions or modifications under the guidance of this invention without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A hyperspectral image classification method based on quaternion deep network, characterized in that, The method comprises the following steps: S1: data preprocessing is performed on the hyperspectral image HSI; S2: a depth-wise attention module is constructed to obtain high-quality image feature data; The depth-wise attention module comprises a channel attention module and a spatial attention module, the preprocessed hyperspectral image is taken as an original input, and the original input is sequentially subjected to processing of the channel attention module and the spatial attention module to obtain a spatial-spectral attention feature, then the spatial-spectral attention feature is subjected to element-by-element multiplication with the original input and is subjected to convolution operation to obtain a final feature map, i.e., image feature data; S3: a multi-branch network is built, the image feature data is grouped, and a quaternion generator is designed to perform data transformation to obtain HSI data in the form of quaternions; S4: a multi-scale quaternion attention module is constructed, the multi-scale quaternion attention module is embedded into each branch of the multi-branch network, and a multi-scale strategy is used to obtain information-rich multi-scale features, and the features are strengthened through a quaternion attention layer; In step S4, the multi-scale quaternion attention module MSQA is specifically implemented as follows: In the first branch, the central pixel of the HSI data in quaternion form is taken as the center, and a cubic block with a size of is selected, and the HSI data of the cubic block is represented as , where represents the size of the input image, and the first scale is denoted as . ( ). cuboid block input into a quaternion convolution module QCM to obtain a feature map wherein the QCM comprises a quaternion convolution QConv, a max pooling MaxPool, a quaternion batch normalization QBN, and a quaternion activation operation; Subsequently, Perform average pooling (AvgPool), two consecutive quaternion fully connected layers (QFC), and non-linear mapping operations to output weights. and will and Element-wise multiplication yields Then obtain through QCM ; Finally, the features of each scale are concatenated ( ) to get the output ; S5: the strengthened features are subjected to feature fusion processing, and finally a classification layer is used to realize hyperspectral image classification.

2. The hyperspectral image classification method based on quaternion deep network according to claim 1, characterized in that: The data preprocessing in step S1 comprises data dimension reduction and extraction of data cube blocks.

3. The method of claim 1, wherein the method is based on a quaternion deep network. The specific implementation of step S1 is as follows: Firstly, the HSI data is reduced dimension by principal component analysis (PCA) , and the reduced HSI data is obtained , , , , respectively represent the height, width, band number and reduced band number of HSI. Then, a cubic block is extracted around each pixel of the image As a subsequent input, represents the spatial size of the cubic block.​ 4. The method of claim 1, wherein the method is based on a quaternion deep network. The specific processing procedure of the depth-wise attention module DWAM in step S2 is as follows: In the channel attention module CA, the input cube block is convolved along the spectral dimension to obtain , the cube block , denotes the spatial size of the cube block, i.e. the size of the input image, denotes the number of bands of the hyperspectral image HSI after dimension reduction; subsequently, the average pooling AvgPool and the maximum pooling MaxPool operations are respectively performed on , the vectors obtained after the pooling are passed through a shared multi-layer perception MLP, and the features output by the MLP are added, followed by a nonlinear mapping to obtain the attention weight , finally, the is multiplied element by element with to obtain the weighted feature map , and the specific calculation formula is as follows: ; ; ; wherein is a two-dimensional convolution operation with a kernel size of ; denotes a Sigmoid activation function; is an element-wise multiplication operation; then, The spatial attention module (SA) is used to perform depthwise convolutions along the spectral direction to obtain the desired result. Here, Three feature maps are obtained by performing depthwise convolutions in parallel using three different convolutional kernels, and then they are combined with... Add them to obtain weighted feature maps, and then use them. Convolution performs channel blending on the weighted feature maps to generate spatial spectral attention features. The entire process of the Spatial Attention (SA) module is represented as follows: ; wherein represents a depth-wise attention convolution operation through the first convolution kernel; thereafter, element-wise multiplication is performed on and , and the features are further integrated through a convolution operation to obtain the final feature map , and the calculation formula is as follows: 。 5. The method of claim 1, wherein the method is based on a quaternion deep network. The specific implementation procedure of step S3 is as follows: Image feature data Waveband grouping is performed to construct a multi-branch network framework, wherein the grouping comprises: Dividing into groups, denoted as ), , represents the size of the input image; the first wavebands are selected as the features of the first group, wherein , to are the features of the second group, and so on. First set of features As input to the first branch; the second set of features and Element-by-element addition yields , as the input to the second branch; in turn, and Element-by-element addition yields As the first Input for each branch; In each branch, the HSI data is mapped to quaternion space for processing by a quaternion generator, let the feature of the i-th waveband of the j-th branch be The two-dimensional convolution is performed on , and the output of each waveband is added as the real part of the quaternion , as shown in the following formula: ​​​ ; wherein is a Gaussian Error Linear Unit, GELU, activation function; is a two-dimensional convolution operation; meanwhile, along the spectral dimension is one-dimensional convolution operation to obtain a three-dimensional tensor and take it as three imaginary parts of the quaternion; obtain The formula is as follows: ; wherein denotes a one-dimensional convolution operation, and and are combined, i.e. the HSI data in quaternion form is obtained .

6. The method of claim 1, wherein the method is based on a quaternion deep network. The feature extraction procedure of the quaternion convolution module MSQA is as follows: ; ; ; ; wherein denotes a Sigmoid activation function; is a RELU activation function; is an element-wise multiplication operation.

7. The method of claim 1, wherein the method is based on a quaternion deep network. The multi-layer perceptron MLP is composed of a quaternion full connection layer QFC, a full connection layer QF, a Dropout operation and an activation operation.

8. The method of claim 1, wherein the method is based on a quaternion deep network. The specific operation of feature fusion and classification in step S5 is as follows: In each branch, features of different scales ( ) are concatenated to obtain the output , which is formulated as follows: ; Wherein ";" represents a matrix splicing operation; Then, the features of all branches are fused, as follows: ; Wherein, N represents the number of branches in the multi-branch network; Finally, softmax is used to realize classification.

9. A hyperspectral image classification system based on quaternion deep network, characterized in that, It comprises: A processor and a memory, the memory is used to store program instructions, and the processor is used to call the storage instructions in the memory to execute a hyperspectral image classification method based on a quaternion deep network as claimed in any one of claims 1-8.

Images