Hyperspectral image classification method and system based on quaternion deep network

Abstract

The invention discloses a hyper-spectral image classification method and system based on a quaternion deep network, which mainly reinforces the learning ability of the network through a quaternion algebraic theory and a deep learning theory, improves the training efficiency of the network, and solves the problems of few hyper-spectral image data samples, spectral space variability and the like. According to the implementation scheme, firstly, the high dimension of data is considered, and dimension reduction is carried out through principal component analysis; then, a depth-by-depth attention module is constructed, important features are enhanced, noise interference is suppressed, and thus the quality of data is improved. And then, constructing a multi-branch network, grouping the enhanced features, and designing a quaternion generator to map the hyperspectral image data to a quaternion space for processing. Meanwhile, a multi-scale quaternion attention module is embedded to further enhance the feature expression ability. And finally, fusing the features of each branch and realizing high-precision classification of the hyperspectral image.

Description

Technical Field

[0001] The present invention belongs to the technical field of hyperspectral remote sensing image processing, and in particular relates to a hyperspectral image classification method based on a quaternion deep network. Background Art

[0002] Hyperspectral imagery (HSI) can simultaneously obtain the spatial and spectral information of a target based on advanced spectral imaging technology. Since HSI contains hundreds of narrow and continuous band data, it contains richer spectral information and has been widely used in many fields, such as mineral exploration and mapping, urban planning, land use surveys, and military reconnaissance. Among the many applications of HSI, classification is a basic and critical task, and its goal is to assign a label to each sample to accurately identify different categories of substances.

[0003] In early studies, HSI classification mainly relied on traditional machine learning methods, such as support vector machines, K-nearest neighbors, random forests, and logistic regression. These traditional methods achieve HSI classification by utilizing spectral information. However, when the spectral features are disturbed, using only spectral information often cannot accurately identify the target sample. In order to solve the above problems, making full use of the spatial information of HSI can effectively improve the performance of the algorithm, and some classification methods based on spectral and spatial features have been proposed. Although these methods have improved the classification accuracy to a certain extent, most traditional machine learning methods focus on the extraction of shallow features and it is difficult to achieve better classification results.

[0004] In recent years, with the rise of deep learning, deep learning-based methods have surpassed the performance of traditional methods and significantly improved the classification accuracy of HSI. For example, long short-term memory networks, convolutional neural networks, graph convolutional networks, etc. Although the above deep learning-based networks have achieved good classification performance, factors such as the small number of HSI labeled samples and high data dimensions have led to problems such as lack of training samples, large number of parameters, and low training efficiency during network learning.

[0005] Quaternions can effectively represent data information and realize information interaction due to their superior algebraic structure and operational properties. At present, quaternions have achieved preliminary research results in the field of hyperspectral images, but their technical theory is not yet mature and has not been well combined with deep networks to give full play to their advantages. Therefore, it is urgent to combine quaternion theory to study and develop high-performance HSI classification methods to solve the development limitations of deep learning algorithms in the field of HSI applications. Summary of the invention

[0006] In view of the shortcomings of the prior art, the purpose of the present invention is to enhance the learning ability of the network, obtain key and information-rich features, improve the learning efficiency of the network, and thus achieve a significant improvement in the accuracy of HSI classification. The main contents are as follows: HSI is dimensionalized and a depth-by-depth attention network is designed to process the data to improve data quality; a high-performance multi-branch network is constructed based on quaternions to map HSI data to quaternion space for processing, effectively enhancing information interaction and reducing information loss and the number of parameters; a parallel strategy is used to process data in a multi-branch structure to improve the training efficiency of the network, and a multi-scale quaternion attention module is embedded to further enhance the network learning ability while enhancing feature richness.

[0007] The technical solution adopted by the present invention is a hyperspectral image classification method based on quaternion deep network, comprising the following steps:

[0008] S1: Data preprocessing of hyperspectral image HSI;

[0009] S2: Build a depth-by-depth attention module to obtain high-quality image feature data;

[0010] S3: Build a multi-branch network, group the image feature data and design a quaternion generator to transform the data to obtain HSI data in quaternion form;

[0011] S4: Construct a multi-scale quaternion attention module, embed the multi-scale quaternion attention module into each branch of the multi-branch network, and use the multi-scale strategy to obtain informative multi-scale features, while strengthening the features through the quaternion attention layer;

[0012] S5: Perform feature fusion processing on the enhanced features, and finally use the classification layer to realize hyperspectral image classification.

[0013] Furthermore, the data preprocessing in step S1 includes data dimension reduction and extraction of data cube blocks.

[0014] Furthermore, the specific implementation of step S1 is as follows:

[0015] First, principal component analysis (PCA) is used to analyze the HSI data. Perform dimensionality reduction to obtain HSI data after dimensionality reduction H, W, C, and C′ represent the height, width, number of bands, and number of bands after dimensionality reduction of HSI, respectively;

[0016] Then, take each pixel of X′ as the center and extract the cube block As the subsequent input, S represents the spatial size of the cube block.

[0017] Furthermore, the specific processing process of the depth-by-depth attention module DWAM in step S2 is as follows:

[0018] In the channel attention module CA, the input cube F in Convolution along the spectral dimension obtains F′ in , cube block S represents the spatial size of the cube block, C′ represents the number of bands of the hyperspectral image HSI after dimensionality reduction; then, F′ in Perform average pooling AvgPool and maximum pooling MaxPool operations respectively, pass the pooled vector through the shared multi-layer perceptron MLP, add the features output by the MLP, and then perform nonlinear mapping to obtain the attention weight F spe Finally, F spe With F in Perform element-by-element multiplication to obtain the weighted feature map F′1. The specific calculation formula is as follows:

[0019] F in ′=Conv 1×1 (F in )

[0020] F spe =σ1(MLP(AvgPool(F in ′))+MLP(MaxPool(F in ′)))

[0021] F′1=F spe ☉F in

[0022] Among them, Conv 1×1 (·) is a two-dimensional convolution operation with a convolution kernel size of 1×1; σ1 represents the Sigmoid activation function; ⊙ is an element-by-element multiplication operation;

[0023] Next, F′1 is convolved depth-wise along the spectral direction through the spatial attention module SA to obtain F′2. Here, F′2 is convolved depth-wise in parallel through three different convolution kernels to obtain three feature maps, and they are combined with F ′ 2 add to obtain the weighted feature map, and then use 1×1 convolution to mix the weighted feature map channels to generate the spatial spectral attention feature F spa , the process of the entire spatial attention module SA is expressed as follows:

[0024]

[0025] Where DwConv j (·),j∈{1,2,3} represents the depth-wise attention convolution operation through the jth convolution kernel; after that, Fspa and F in Perform element-by-element multiplication and further integrate the features through 1×1 convolution operation to obtain the final feature map The calculation formula is as follows:

[0026] F′3=Conv 1×1 (F spa ☉F in ).

[0027] Furthermore, the specific implementation process of step S3 is as follows:

[0028] The image feature data F′3 is grouped by bands to construct a multi-branch network framework, where the grouping includes: dividing F′3 into N groups, denoted as Select the first g bands as the features G1 of the first group, where g = C′ / N, g+1 to 2g as the features G2 of the second group, and so on;

[0029] The first set of features G1 is used as the input of the first branch; the second set of features G2 is added element by element to G1 to get G′2, which is used as the input of the second branch; successively, G i and G′ i-1 Add element by element to get G′ i , as the input of the i-th branch;

[0030] In each branch, the HSI data is mapped to the quaternion space for processing through the quaternion generator. Let the i-th branch G′ i The characteristics of the kth band (i=1,2,…,N) are right Perform a two-dimensional convolution and add the output of each band as the real part of the quaternion As shown below:

[0031]

[0032] Where σ2 is the Gaussian error linear unit GELU activation function; Conv(·) is a two-dimensional convolution operation; at the same time, G′ is i Perform a one-dimensional convolution operation to obtain a three-dimensional tensor And use it as the three imaginary parts of the quaternion; get F I The formula is as follows:

[0033] F I =σ2(Conv(G′ i ))

[0034] Where Conv(·) represents a one-dimensional convolution operation. R and F ICombine them to get the HSI data in quaternion form

[0035] Furthermore, in step S4, the specific implementation of the multi-scale quaternion attention module MSQA is as follows:

[0036] In the i-th branch, the HSI data in the form of quaternions As the center pixel of the image, select m pixels with scales of λ1, λ2,…,λ m (λ i ≤S), the lth scale is denoted by

[0037] The cube block Input into the quaternion convolution module QCM to get the feature map Among them, QCM includes quaternion convolution QConv, maximum pooling MaxPool, quaternion batch normalization QBN and quaternion activation operations;

[0038] Then, Perform average pooling AvgPool and two consecutive quaternion fully connected layers QFC and nonlinear mapping operations, and output weights and will and Perform element-wise multiplication to get Then, the QCM was used to obtain

[0039] Finally, the features of each scale are Through a shared multi-layer perceptron MLP, the output is obtained

[0040] Furthermore, the feature extraction process of the quaternion convolution module MSQA is as follows:

[0041]

[0042]

[0043] Where σ1 represents the Sigmoid activation function; σ3 represents the RELU activation function.

[0044] Furthermore, the multi-layer perceptron MLP consists of a quaternion fully connected layer QFC, a fully connected layer QF, a Dropout operation and an activation operation.

[0045] Furthermore, the specific operations of feature fusion and classification in step S5 are as follows:

[0046] In each branch, features of different scales are The output M is obtained by splicingi , the formula is as follows:

[0047]

[0048] Where “;” indicates the concatenation operation of the matrix;

[0049] Then, the features of all branches are fused as follows:

[0050] F=[M1;M1;…;M N ]

[0051] Where N represents the number of branches in the multi-branch network;

[0052] Finally, softmax is used to achieve classification.

[0053] The present invention also provides a hyperspectral image classification system based on a quaternion deep network, comprising:

[0054] A processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a hyperspectral image classification method based on a quaternion deep network as described in the above technical solution.

[0055] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows:

[0056] 1) The present invention constructs a multi-branch network that can process data in parallel while extracting rich features, thereby improving training efficiency. At the same time, the input of each branch is passed to the next branch, making full use of important information and reducing the loss in the process of information transmission.

[0057] 2) By building the DWAN module, the quality of the data is improved, which facilitates the subsequent feature learning and thus improves the classification accuracy.

[0058] 3) Mapping HSI data to quaternion space, using the algebraic structure and operational properties of quaternions, effectively reduces the number of parameters and strengthens the interaction between information.

[0059] 4) The embedded MSQA further strengthens the important features, and the adopted multi-scale strategy enables the extracted features to more accurately characterize the homogeneous areas, enabling the model to learn richer spatial information. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a framework diagram of a multi-branch quaternion attention network (MQAN) in an embodiment of the present invention;

[0061] Figure 2 is a flow chart of a depth-by-depth attention module (DWAM) in an embodiment of the present invention;

[0062] Figure 3 It is a design diagram of a quaternion generator (QG) in an embodiment of the present invention;

[0063] Figure 4 Schematic diagram of a multi-scale quaternion attention (MSQA) module in an embodiment of the present invention;

[0064] Figure 5 The influence of the number of principal components on the algorithm in the embodiment of the present invention;

[0065] Figure 6 The influence of scale size and number on the algorithm in the embodiment of the present invention;

[0066] Figure 7 The figure is a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to demonstrate the technical solution and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings.

[0068] The multi-branch quaternion attention network (MQAN) is mainly composed of PCA, depth-wise attention module (DWAM), quaternion generator (QG), and multi-scale quaternion attention (MSQA) modules, such as Figure 1 As shown. Considering the high dimensionality of the data, the original HSI is first reduced in dimensionality. At the same time, in order to improve the quality of the data, DWAM is used to strengthen the data. Then, QG is used to transform the data into quaternion space. Next, a multi-branch network is constructed to process the data in parallel. In addition, MSQA is embedded in each branch to obtain rich information and better characterize the diversity of homogeneous areas. Finally, the multi-scale features extracted from each branch are fused (Fusion) and input into the Softmax classifier for classification.

[0069] like Figure 7 As shown, a hyperspectral image HSI classification method based on quaternion deep network provided by an embodiment of the present invention specifically includes the following steps:

[0070] S1: Perform preprocessing operations such as data dimension reduction and data cube block extraction on the original hyperspectral image HSI data. The specific methods are as follows:

[0071] S1.1: Using principal component analysis (PCA) to analyze HSI data Perform dimensionality reduction to obtain the reduced dimensionality data H, W, C, and C′ represent the height, width, number of bands, and number of bands after dimensionality reduction of HSI, respectively.

[0072] S1.2: Extract cube blocks with each pixel of X′ as the center As input to the network, S represents the spatial size of the cube.

[0073] S2: Construct a depth-wise attention module (DWAM) to effectively enhance important information and suppress noise interference, thereby improving the quality of the collected HSI data, such as Figure 2 As shown. The specific method in S2 is as follows:

[0074] S2.1: In the channel attention (CA) module, F in Convolution along the spectral dimension obtains F′ in Then, for F′ in Perform average pooling (AvgPool) and maximum pooling (MaxPool) operations respectively, pass the pooled vector through a shared multi-layer perceptron (MLP), add the features output by the MLP, and then perform nonlinear mapping to obtain the attention weight F spe Finally, F spe With F in Perform element-by-element multiplication to obtain the weighted feature map F′1. The calculation formula of this process is as follows:

[0075] F in ′=Conv 1×1 (F in )

[0076] F spe =σ1(MLP(AvgPool(F in ′))+MLP(MaxPool(F in ′)))

[0077] F′1=F spe ⊙F in

[0078] Among them, Conv 1×1 (·) is a two-dimensional convolution operation with a kernel size of 1×1; σ1 represents the Sigmoid activation function; ⊙ is an element-by-element multiplication operation.

[0079] S2.2: Then, F′1 is convolved depth-wise along the spectral direction through the spatial attention (SA) module to obtain F′2. Here, F′2 is convolved depth-wise in parallel through three different convolution kernels to obtain three feature maps, and they are added to F′2 to obtain the weighted feature map, and then the weighted feature map is mixed by channel using 1×1 convolution to generate the spatial spectral attention feature F spa The entire SA module process is as follows:

[0080]

[0081] Where DwConvj (·),j∈{1,2,3} represents the depth-wise attention convolution operation through the jth convolution kernel.

[0082] S2.3: After that, F spa With F in Perform element-by-element multiplication and further integrate the features through 1×1 convolution operation to obtain the final feature map The calculation formula is as follows:

[0083] F′3=Conv 1×1 (F spa ⊙F in ).

[0084] S3: Build a multi-branch network, group F′3 and design a quaternion generator (QG) for data conversion, such as Figure 3 S3 is implemented in the following steps:

[0085] S3.1: Divide F′3 into N groups, denoted as The first g (g=C′ / N) bands are selected as the features G1 of the first group, g+1 to 2g are the features G2 of the second group, and so on.

[0086] S3.2: The first set of features G1 is used as the input of the first branch; the second set of features G2 is added element by element to G1 to obtain G′2, which is used as the input of the second branch; similar to the above process, G i and G′ i-1 Add element by element to get G′ i , as the input of the i-th branch.

[0087] S3.3: In each branch, the HSI data is mapped to the quaternion space for processing through QG. Let the i-th branch G′ i The characteristics of the kth band (i=1,2,…,N) are right Perform a two-dimensional convolution and add the output of each band as the real part of the quaternion As shown below:

[0088]

[0089] Where σ2 is the Gaussian Error Linear Unit (GELU) activation function; Conv(·) is a two-dimensional convolution operation.

[0090] S3.4: At the same time, along the spectral dimension, G′ i Perform a one-dimensional convolution operation to obtain a three-dimensional tensor And use it as the three imaginary parts of the quaternion. Get F I The formula is as follows:

[0091] F I =σ2(Conv(G′ i ))

[0092] Where Conv(·) represents a one-dimensional convolution operation.

[0093] S3.5: F R and F I Combine them to get the HSI data Q in quaternion form i .

[0094] S4: Design a multi-scale quaternion attention (MSQA) module, such as Figure 4 The specific implementation is as follows:

[0095] S4.1: In the i-th branch, As the center pixel of the image, select m pixels with scales of λ1, λ2,…,λ m (λ i ≤S), the lth scale is denoted by

[0096] S4.2: Place the cube blocks Input into the quaternion convolution module (QCM) to obtain the feature map QCM includes quaternion convolution (QConv), MaxPool, quaternion batch normalization (QBN), and quaternion activation operations.

[0097] S4.3: Then, Perform AvgPool and two consecutive quaternion fully connected layers (QFC) and nonlinear mapping operations to output weights and will and Perform element-wise multiplication to get Then, the QCM was used to obtain The feature extraction process of the MSQA module is as follows:

[0098]

[0099]

[0100] Where σ1 represents the Sigmoid activation function; σ3 represents the RELU activation function.

[0101] S4.4: Finally, the features of each scale are Through a shared MLP, the output is Here, the MLP consists of a quaternion fully connected layer (QFC), a fully connected layer (QF), a dropout operation, and an activation operation.

[0102] S5: Fuse the features and implement hyperspectral image classification. The specific implementation is as follows:

[0103] S5.1: In each branch, features of different scales are combined The output M is obtained by splicing i , the formula is as follows:

[0104]

[0105] The “;” indicates the concatenation operation of the matrix.

[0106] S5.2: Then, the features of all branches are fused as follows:

[0107]

[0108] S5.3: Finally, the model is trained using the cross entropy loss function and softmax is used for classification.

[0109] In order to verify the effect of the present invention, the following experiment was carried out:

[0110] In the embodiments of the present invention, two public data sets, PU and SV, are used for experiments to evaluate the effectiveness of the model. The PU data set is an HSI with a size of 610×340 and 103 channels, and the SV data set is an HSI with a size of 512×217 and 204 channels. Both data sets are divided into training set, validation set and test set. In PU, the training set and validation set account for 20% and 10% of the total number of samples, respectively, and the remaining samples are used as test sets. In SV, the ratios of the training set and sample set are 10% and 10% of the number of samples, respectively, and the remaining 80% is used as a test set.

[0111] In order to comprehensively evaluate the performance of the model, we use overall accuracy (OA), average accuracy (AA) and Kappa coefficient (Kappa) as evaluation indicators. OA reflects the proportion of samples correctly classified by the model; AA evaluates the performance of the model on each category; Kappa is used to evaluate the degree to which the model performance exceeds the random level.

[0112] The present invention adopts PyCharm programming software and uses Adam optimizer to perform model training in PyTorch deep learning framework. Here, the initial learning rate is set to 0.001, the batch size is 64, the training cycle is 100 rounds, and the average value of ten experimental results is calculated to reduce the error caused by random sampling.

[0113] In addition, we analyzed the number of principal components, the size of the scale and its quantity to determine the optimal parameters. First, Figure 5 The effect of the number of principal components on the performance of the algorithm is shown. Figure 5 It can be seen that as the number of principal components increases, the feature information becomes richer and richer, and OA increases accordingly. However, when the number of principal components reaches a certain level, the upward trend tends to be flat due to the small amount of information contained in the subsequent principal components. At the same time, the more principal components there are, the more calculations will be significantly increased. Therefore, the number of principal components is set to 30 in both data sets.

[0114] The effects of scale size and number on algorithm performance are as follows: Figure 6 (a) and Figure 6 As shown in (b) in the figure. Figure 6 (a) It can be seen that as the scale increases, the classification accuracy of the two datasets shows a trend of first increasing and then decreasing. This is because when the scale is too small, the homogeneous area cannot be effectively covered; when the scale is too large, heterogeneous pixels will be included, which will interfere with feature learning and the amount of calculation will increase significantly. Therefore, in the PU dataset and the SV dataset, the size of the region is determined in the range of {7, 13, 15, 19} and {7, 11, 13, 17} respectively. Due to the diversity of the shapes and sizes of the HSI homogeneous regions, we select regions of different sizes as pixels to learn features and achieve accurate characterization of the homogeneous region. Figure 6 (b) The effect of the number of scales on OA is explored. The experimental results show that when 3 and 4 scales are selected, OA is relatively high, but the accuracy difference between the two is not obvious, and the more scales, the more computational effort will increase significantly. Therefore, in both PU and SV datasets, three scales are selected for experiments, with sizes of {7, 13, 19} and {7, 11, 13}, respectively.

[0115] In order to analyze the performance of each module, the present invention conducts ablation experiments to study the impact of multi-branch structure (MB), multi-scale strategy (MS), depth-by-depth attention module (DWAM), and quaternion attention layer (QAL) on the performance of the algorithm. Tables 1 and 2 show the experimental results on the PU and SV datasets, where the baseline network refers to the network that does not contain BG, MS, DWAM, and QAL.

[0116] Table 1 The impact of each module on network performance on the PU dataset

[0117]

[0118] Table 2 The impact of each module on network performance on the SV dataset

[0119]

[0120] As can be seen from Table 1, after adopting MB, the OA, AA and Kappa of the network are improved by 3.97%, 7.77% and 5.72% compared with the Baseline, which shows that MB can effectively extract rich features and improve classification performance. In addition, the introduced attention network also increases the classification accuracy to a certain extent. However, when DWAM is used, it is 7.75% higher than the OA using QAL. When all modules exist at the same time, the classification effect of the network is the best, and its OA, AA and Kappa reach 99.71%, 99.44% and 99.62% respectively. On the other hand, MQAN has the smallest standard deviation, which further highlights the stability of the algorithm.

[0121] As can be seen from Table 2, after the introduction of the multi-branch multi-scale network framework, the classification performance of the network is improved by 59.81%. The reason is that the MB structure can make full use of the spectral features to optimize the classification performance, while the MS strategy can obtain rich information and effectively depict the diversity of homogeneous areas, thereby improving the classification effect. In addition, the use of DWAM is 4.55% higher than that of OA using QAL, showing that high-quality image data can achieve a significant improvement in classification accuracy. Moreover, MQAN has the smallest standard deviation, indicating that the algorithm is more robust.

[0122] On the other hand, an embodiment of the present invention further provides a hyperspectral image classification system based on a quaternion deep network, comprising:

[0123] A processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a hyperspectral image classification method based on a quaternion deep network as described in the above technical solution.

[0124] It should be understood that the above description of the embodiments is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.

Claims

1. A hyperspectral image classification method based on quaternion deep network, characterized in that: The following steps are involved: S1: Data preprocessing of hyperspectral image HSI; S2: Build a depth-by-depth attention module to obtain high-quality image feature data; S3: Build a multi-branch network, group the image feature data and design a quaternion generator to transform the data to obtain HSI data in quaternion form; S4: Construct a multi-scale quaternion attention module, embed the multi-scale quaternion attention module into each branch of the multi-branch network, and use the multi-scale strategy to obtain informative multi-scale features, while strengthening the features through the quaternion attention layer; S5: Perform feature fusion processing on the enhanced features, and finally use the classification layer to realize hyperspectral image classification.

2. The hyperspectral image classification method based on quaternion deep network according to claim 1 is characterized in that: The data preprocessing in step S1 includes data dimension reduction and extracting data cube blocks.

3. The hyperspectral image classification method based on quaternion deep network according to claim 1 is characterized in that: The specific implementation of step S1 is as follows: First, principal component analysis (PCA) is used to analyze the HSI data. Perform dimensionality reduction to obtain HSI data after dimensionality reduction H, W, C, and C′ represent the height, width, number of bands, and number of bands after dimensionality reduction of HSI, respectively; Then, take each pixel of X′ as the center and extract the cube block As the subsequent input, S represents the spatial size of the cube block.

4. The hyperspectral image classification method based on quaternion deep network according to claim 1 is characterized by: The specific processing process of the depth-by-depth attention module DWAM in step S2 is as follows: In the channel attention module CA, the input cube F in Convolution along the spectral dimension obtains F′ in , cube block S represents the spatial size of the cube block, that is, the size of the input image, and C′ represents the number of bands of the hyperspectral image HSI after dimensionality reduction; then, F′ in Perform average pooling AvgPool and maximum pooling MaxPool operations respectively, pass the pooled vector through the shared multi-layer perceptron MLP, add the features output by the MLP, and then perform nonlinear mapping to obtain the attention weight F spe Finally, F spe With F in Perform element-by-element multiplication to obtain the weighted feature map F′1. The specific calculation formula is as follows: F in ′=Conv 1×1 (F in ) F spe =σ1(MLP(AvgPool(F in ′))+MLP(MaxPool(F in ′))) F′1=F spe ☉F in Among them, Conv 1×1 (·) is a two-dimensional convolution operation with a convolution kernel size of 1×1; σ1 represents the Sigmoid activation function; ⊙ is an element-by-element multiplication operation; Next, F′1 is convolved depth-wise along the spectral direction through the spatial attention module SA to obtain F′2. Here, F′2 is convolved depth-wise in parallel through three different convolution kernels to obtain three feature maps, and they are added to F′2 to obtain the weighted feature map, and then the weighted feature map is mixed through 1×1 convolution to generate the spatial spectral attention feature F spa , the process of the entire spatial attention module SA is expressed as follows: Where DwConv j (·), j∈{1, 2, 3} represents the depth-wise attention convolution operation through the jth convolution kernel; then, F spa and F in Perform element-by-element multiplication and further integrate the features through 1×1 convolution operation to obtain the final feature map The calculation formula is as follows: F′3=Conv 1×1 (F spa ☉F in )。 5. The hyperspectral image classification method based on quaternion deep network according to claim 1 is characterized in that: The specific implementation process of step S3 is as follows: The image feature data F′3 is grouped by bands to construct a multi-branch network framework, where the grouping includes: dividing F′3 into N groups, denoted as S represents the size of the input image; the first g bands are selected as the features G1 of the first group, where g = C′ / N, g+1 to 2g are the features G2 of the second group, and so on; The first set of features G1 is used as the input of the first branch; the second set of features G2 is added element by element to G1 to get G′2, which is used as the input of the second branch; successively, G i and G′ i-1 Add element by element to get G′ i , as the input of the i-th branch; In each branch, the HSI data is mapped to the quaternion space for processing through the quaternion generator. Let the i-th branch G′ i The characteristics of the kth band (i=1, 2, ..., N) are right Perform a two-dimensional convolution and add the output of each band as the real part of the quaternion As shown below: Where σ2 is the Gaussian error linear unit GELU activation function; Conv(·) is a two-dimensional convolution operation; at the same time, G′ is i Perform a one-dimensional convolution operation to obtain a three-dimensional tensor And use it as the three imaginary parts of the quaternion; get F I The formula is as follows: F I =σ2(Conv(G′ i )) Where Conv(·) represents a one-dimensional convolution operation. R and F I Combine them to get the HSI data in quaternion form 6. The hyperspectral image classification method based on quaternion deep network according to claim 1 is characterized by: In step S4, the specific implementation of the multi-scale quaternion attention module MSQA is as follows: In the i-th branch, the HSI data in the form of quaternions The center pixel is taken as the center, and m scales are selected with sizes λ1, λ2, …, λ m (λ i ≤S), S represents the size of the input image, and the lth scale is recorded as The cube block Input into the quaternion convolution module QCM to get the feature map Among them, QCM includes quaternion convolution QConv, maximum pooling MaxPool, quaternion batch normalization QBN and quaternion activation operations; Then, Perform average pooling AvgPool and two consecutive quaternion fully connected layers QFC and nonlinear mapping operations, and output weights and will and Perform element-wise multiplication to get Then, the QCM was used to obtain Finally, the features of each scale are Through a shared multi-layer perceptron MLP, the output is obtained 7. The hyperspectral image classification method based on quaternion deep network according to claim 6 is characterized by: The feature extraction process of the quaternion convolution module MSQA is as follows: Where σ1 represents the Sigmoid activation function; σ3 is the RELU activation function; ⊙ is the element-by-element multiplication operation.

8. The hyperspectral image classification method based on quaternion deep network according to claim 6 is characterized by: The multi-layer perceptron MLP consists of a quaternion fully connected layer QFC, a fully connected layer QF, a Dropout operation, and an activation operation.

9. The hyperspectral image classification method based on quaternion deep network according to claim 6 is characterized by: The specific operations of feature fusion and classification in step S5 are as follows: In each branch, features of different scales are The output M is obtained by splicing i , the formula is as follows: Where ";" indicates the concatenation operation of the matrix; Then, the features of all branches are fused as follows: F=[M1;M1;…;M N ] Where N represents the number of branches in the multi-branch network; Finally, softmax is used to achieve classification.

10. A hyperspectral image classification system based on quaternion deep network, characterized in that: include: A processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a hyperspectral image classification method based on a quaternion deep network as described in any one of claims 1 to 9.

Images