Full-distance double-path hyperspectral image classification method

Abstract

A full-distance double-path hyperspectral image classification method belongs to the field of remote sensing image processing and classification, and comprises the following steps: firstly, carrying out principal component analysis (PCA) dimension reduction on the spectral dimension of hyperspectral data, then extracting spatial-spectral features of different scales through multi-scale convolution, and enhancing the feature expression ability by utilizing a spatial-spectral joint attention mechanism. Thirdly, optimizing the features by adopting a forward propagation module, and recombining the multi-scale features through interval grouping and multiple expansion strategies (row priority, column priority, jumping type and chessboard interleaving); and further enhancing feature representation through a module S6 and spectral dimension fusion, finally integrating multi-source features through an adaptive fusion module, inputting the multi-source features into a classifier to obtain a classification result, and calculating classification accuracy. Compared with other methods, the classification accuracy is improved, and the performance is more stable.

Description

Technical Field

[0001] This invention belongs to the field of land cover classification and identification in remote sensing image processing and hyperspectral information extraction, and specifically relates to a full-range dual-path hyperspectral image classification method. Background Technology

[0002] In the fields of remote sensing image processing and hyperspectral information extraction, hyperspectral image data is a widely available dataset. Dimensionality reduction and classification analysis of hyperspectral data can reduce computational time complexity, remove noise interference from the dataset, and uncover potential information such as land cover ratios and landform features. However, due to the high dimensionality of the original spectral images and the high repetition of information in closely spaced bands, the classification process is computationally intensive and susceptible to noise. Existing classification methods often directly use the original spectral images or employ only simple dimensionality reduction methods, failing to fully utilize the spatial texture information and inter-band information of the original spectral images. Furthermore, the selected bands may not achieve ideal classification results. This poses a significant challenge to the classification problem of hyperspectral image data, thus requiring effective and novel technical methods to solve the classification problem of hyperspectral image data. Summary of the Invention

[0003] To address the above problems, this invention provides a full-range dual-path hyperspectral image classification method, the method comprising the following steps:

[0004] Principal component analysis is performed on the spectral dimension of a hyperspectral dataset X to compress its spectral dimension to c dimensions, resulting in compressed hyperspectral image data X. PCA According to the obtained X PCA The multi-scale segmentation module yields three hyperspectral image data X after convolution at different scales. L X M X S ;

[0005] Based on the obtained hyperspectral image data at three different scales, X L X M X S Spatial-spectral joint attention calculations were performed separately to obtain the data sequence X at three scales after spatial-spectral joint attention weighting. LT X MT X ST The three data points are then concatenated to obtain the concatenated data sequence X. TCat ;

[0006] Based on the obtained spliced ​​data sequence X TCat The forward propagation module processes the data to obtain the processed data sequence X. T ;

[0007] Based on the obtained hyperspectral image data at three different scales, X L X M X S Perform interval grouping to obtain grouping result X. L1 X M1 X S1 X L2 X M2 X S2 X L3 X M3 X S3 X L4 X M4 X S4 ; X L1 X M1 X S1 Divide the data into groups, expand them in row-major order, and concatenate them into a new data sequence X1. L2 X M2 X S2 Divide the data into groups, expand them column-wise, and concatenate them into a new data sequence X2. L3 X M3 X S3 Divide the data into groups, perform jump expansion, and concatenate them into a new data sequence X3. L4 X M4 X S4 Divide the data into groups, unfold them in a chessboard pattern, and then combine them into a new data sequence X4.

[0008] The obtained spliced ​​sequences X1, X2, X3, and X4 are processed by module S6, and the processing results are spliced ​​along the spectral dimensions to obtain the spliced ​​sequence X. M ;

[0009] Based on the obtained data sequence X T X M The adaptive fusion module processes the data to obtain the final output sequence X. Fusion The data is then fed into a classifier for classification to obtain the classification result.

[0010] Based on the obtained classification results, the classification accuracy on the spectral image dataset is calculated.

[0011] Furthermore, the hyperspectral image data X L X M X S The formula is as follows:

[0012] X PCA =PCA(X)

[0013] X L =Linear L (Conv3DL (X PCA ))

[0014] X M =Linear M (Conv3D M (X PCA ))

[0015] X S =Linear S (Conv3D S (X PCA ))

[0016] Where X represents the hyperspectral image dataset, PCA(·) represents principal component analysis, and Linear... L Linear M Linear S These represent linear layer processing at three scales, Conv3D L Conv3D L Conv3D L These represent 3D convolution processing at three different scales.

[0017] Furthermore, the data sequence X after spatial-spectral joint attention weighting at the three scales... LT X MT X ST The formula is as follows:

[0018] X LT =SSAttention(X) L )

[0019] X MT =SSAttention(X) M )

[0020] X ST =SSAttention(X) S )

[0021] Where SSAttention(·) is the joint attention function for the spatial spectrum, and the calculation of SSAttention(·) is as follows:

[0022]

[0023] Where ((·) represents the softmax activation function, d is the number of multi-head attention heads, and Q, K, V, S, and C are the matrices required for calculating the joint attention function, as shown below:

[0024] Q = K = V = Linear(X) L )

[0025] S=AdaptiveAvgPool2D(Conv2D(GeLU(BN(X L )))

[0026] C = GeLU(BN(Conv2D(X) L )))

[0027] Where Linear(·) represents a linear layer, BN(·) represents a batch normalization layer, GeLU(·) represents the GeLU activation function, Conv2D(·) represents 2D convolution, and AdaptiveAvgPool2D(·) represents 2D adaptive average pooling. M and X S The calculation method for the spatial-spectral joint attention is the same; based on the obtained data sequence X after weighting by the spatial-spectral joint attention at three scales... LT X MT X ST The three data sequences are concatenated to obtain the concatenated data sequence X. TCat As shown below:

[0028] X TCat =cat(X) LT X MT X ST )

[0029] Here, cat(·) represents the data sequence concatenation operation.

[0030] Furthermore, the data sequence X processed by the forward propagation module T The formula is as follows:

[0031] X T =Mlp(X TCat )

[0032] Where Mlp(·) is the forward propagation module, as shown below:

[0033] Mlp(X TCat )=LN(Linear(Dropout(GeLU(Linear(X TCat )))))

[0034] Dropout(·) is the Dropout regularization layer.

[0035] Furthermore, the grouping result X L1 X M1 X S1 X L2 X M2 X S2 X L3X M3 X S3 X L4 X M4 X S4 The formula is as follows:

[0036] (X L1 X L2 X L3 X L4} = Interval(X L )

[0037] (X M1 X M2 X M3 X M4} = Interval(X M )

[0038] {X S1 X S2 X S3 X S4} = Interval(X S )

[0039] Where Interval(·) represents the interval grouping operation, which sequentially extracts elements from the data sequence to be grouped and places them into the corresponding groups to form the grouping results for each of the three scales; then X L1 X M1 X S1 The data is divided into groups, expanded in row-major order, and then concatenated to form a new data sequence X1, as shown below:

[0040] X1 = cat(Flatten) Row (X L1 X M1 X S1 ))

[0041] Among them, Flatten Row (·) indicates row-first expansion; X will be processed first. L2 X M2 X S2 Divide the data into a group, expand it column-wise, and concatenate them into a new data sequence X2, as shown below:

[0042] X2 = cat(Flatten) Column (X L2 X M2 X S2 ))

[0043] Among them, Flatten Column (·) indicates column-first expansion; X will be processed accordingly. L3 XM3 X S3 The data is divided into groups, expanded by skipping, and then concatenated to form a new data sequence X3, as shown below:

[0044] X3 = cat(Flatten) Jump (X L3 X M3 X S3 ))

[0045] Among them, Flatten Jump (·) indicates jump unpacking; X is then processed. L4 X M4 X S4 The data is divided into groups, unfolded in a chessboard pattern, and then concatenated to form a new data sequence X4, as shown below:

[0046] X4 = cat(Flatten) Chess (X L4 X M4 X S4 ))

[0047] Among them, Flatten Chess (·) indicates that the chessboard is arranged in an alternating pattern.

[0048] Furthermore, the splicing sequence X M The formula is as follows:

[0049] X M =cat(S6(X1, X2, X3, X4))

[0050] Where S6(·) represents the processing of module S6, as shown below:

[0051]

[0052]

[0053] y t =Ch t

[0054] y = [y1, y2, ..., y L ]

[0055] in, C represents the trainable parameters of the S6 module, and x represents the trainable parameters of the S6 module. t h represents the input of module S6 at time t. t and h t-1 Let y represent the hidden states at time t and t-1, respectively, and y be the output sequence, y1, y2, ..., y3. L Let x1, x2, ..., x LThe corresponding output.

[0056] Furthermore, the most output sequence X Fusion The formula is as follows:

[0057] X Fusion =Fusion(X) T X M )

[0058] Wherein, Fusion(·) represents the adaptive fusion module, as shown below:

[0059] Fusion(·)=gX T +(1-g)X M

[0060] Where g is the weight matrix, and g is calculated as follows:

[0061] g = (GeLU(Linear(Sigmoid(X)) T ))))

[0062] Where Sigmoid(·) is the Sigmoid activation function; the resulting X Fusion The data is fed into a classifier for classification to obtain the classification result.

[0063] Furthermore, based on the obtained classification results, the classification accuracy on the spectral image dataset is calculated.

[0064] This invention provides a full-range dual-path hyperspectral image classification method, which has the following advantages:

[0065] (1) The method adopts a multi-scale partitioning and spatial-spectral joint attention mechanism, which can effectively capture the spatial-spectral features of hyperspectral images at different scales and fully explore the multi-scale spatial-spectral joint information.

[0066] (2) The method enhances the diversity of local features and improves the model’s ability to represent complex structures in hyperspectral images by using interval grouping and diversified unfolding strategies (row priority, column priority, skip, chessboard interlacing).

[0067] (3) The method combines an adaptive fusion module to dynamically integrate features of different scales and groups, optimize feature representation, and thus improve the accuracy of hyperspectral image classification. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a flowchart of a full-range dual-path hyperspectral image classification method provided by the present invention;

[0070] Figure 2 This is a schematic diagram of a hyperspectral image; Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0072] Exemplary methods

[0073] like Figure 1 This invention provides a full-range dual-path hyperspectral image classification method, the steps of which are as follows:

[0074] Step S110: Perform principal component analysis on the spectral dimension of a hyperspectral dataset X to compress its spectral dimension to c dimensions, obtaining the compressed hyperspectral image data X. PCA According to the obtained X PCA The multi-scale segmentation module yields three hyperspectral image data X after convolution at different scales. L X M X S Its formula is as follows:

[0075] X PCA =PCA(X)

[0076] X L =Linear L (Conv3D L (X PCA ))

[0077] X M =Linear M (Conv3D M (X PCA ))

[0078] X S=Linear S (Conv3D S (X PCA ))

[0079] Where X represents the hyperspectral image dataset, PCA(·) represents principal component analysis processing, and Linear... L Linear M Linear S These represent linear layer processing at three scales, Conv3D L Conv3D L Conv3D L These represent 3D convolution processing at three different scales.

[0080] Step S120: Based on the obtained hyperspectral image data X at three different scales L X M X S Spatial-spectral joint attention calculations were performed separately to obtain the data sequence X at three scales after spatial-spectral joint attention weighting. LT X MT X ST Its formula is as follows:

[0081] X LT =SSAttention(X) L )

[0082] X MT =SSAttention(X) M )

[0083] X ST =SSAttention(X) S )

[0084] Wherein, SSAttention(·) is the joint attention function for the spatial spectrum, and its formula is as follows:

[0085]

[0086] Where ξ(·) represents the softmax activation function, d is the number of multi-head attention heads, and Q, K, V, S, and C are the matrices required for calculating the joint attention function, as shown below:

[0087] Q = K = V = Linear(X) L )

[0088] S=AdaptiveAvgPool2D(Conv2D(GeLU(BN(X L )))

[0089] C = GeLU(BN(Conv2D(X) L )))

[0090] Where Linear(·) represents a linear layer, BN(·) represents a batch normalization layer, GeLU(·) represents the GeLU activation function, Conv2D(·) represents 2D convolution, and AdaptiveAvgPool2D(·) represents 2D adaptive average pooling. M and X S The calculation method for the spatial-spectral joint attention is the same; based on the obtained data sequence X after weighting by the spatial-spectral joint attention at three scales... LT X MT X ST The three data sequences are concatenated to obtain the concatenated data sequence X. TCat Its formula is as follows:

[0091] X TCat =cat(X) LT X MT X ST )

[0092] Here, cat(·) represents the data sequence concatenation operation.

[0093] Step S130: Based on the obtained spliced ​​data sequence X TCat The forward propagation module processes the data to obtain the processed data sequence X. T Its formula is as follows:

[0094] X T =Mlp(X TCat )

[0095] Where Mlp(·) is the forward propagation module, and its expression formula is as follows:

[0096] Mlp(X TCat )=LN(Linear(Dropout(GeLU(Linear(X TCat )))))

[0097] Dropout(·) is the Dropout regularization layer.

[0098] Step S140: Based on the obtained hyperspectral image data X at three different scales L X M X S Perform interval grouping to obtain grouping result X. L1 X M1 X S1 X L2 X M2X S2 X L3 X M3 X S3 X L4 X M4 X S4 Its formula is as follows:

[0099] {X L1 X L2 X L3 X L4} = Interval(X L )

[0100] {X M1 X M2 X M3 X M4} = Interval(X M )

[0101] {X S1 X S2 X S3 X S4} = Interval(X S )

[0102] Where Interval(·) represents the interval grouping operation, which sequentially extracts elements from the data sequence to be grouped and places them into the corresponding groups to form the grouping results for each of the three scales; then X L1 X M1 X S1 The data is divided into groups, expanded in row-major order, and then concatenated to form a new data sequence X1, which is expressed by the following formula:

[0103] X1 = cat(Flatten) Row (X L1 X M1 X S1 ))

[0104] Among them, Flatten Row (·) indicates row-first expansion; X will be processed first. L2 X M2 X S2 The data is divided into groups, expanded column-wise, and then concatenated to form a new data sequence X2, which is expressed by the following formula:

[0105] X2 = cat(Flatten) Column (X L2 X M2 X S2 ))

[0106] Among them, FlattenColumn (·) indicates column-first expansion; X will be processed accordingly. L3 X M3 X S3 The data is divided into groups, expanded by skipping, and then concatenated to form a new data sequence X3, which is expressed by the following formula:

[0107] X3 = cat(Flatten) Jump (X L3 X M3 X S3 ))

[0108] Among them, Flatten Jump (·) indicates jump unpacking; X is then processed. L4 X M4 X S4 The data is divided into groups, unfolded in a chessboard pattern, and then pieced together to form a new data sequence X4, which can be expressed as follows:

[0109] X4 = cat(Flatten) Chess (X L4 X M4 X S4 ))

[0110] Among them, Flatten Chess (·) indicates that the chessboard is arranged in an alternating pattern.

[0111] Step S150: Process the obtained spliced ​​sequences X1, X2, X3, and X4 using module S6, and then splice the processing results along the spectral dimensions to obtain the spliced ​​sequence X. M Its formula is as follows:

[0112] X M =cat(S6(X1, X2, X3, X4))

[0113] Where S6(·) represents the processing of module S6, and its expression formula is as follows:

[0114]

[0115]

[0116] y t =Ch t

[0117] y = [y1, y2, ..., y L ]

[0118] in, C represents the trainable parameters of the S6 module, and x represents the trainable parameters of the S6 module. t h represents the input of module S6 at time t.t and h t-1 Let y represent the hidden states at time t and t-1, respectively, and y be the output sequence, y1, y2, ..., y3. L Let x1, x2, ..., x L The corresponding output.

[0119] Step S160: Based on the obtained data sequence X T X M The adaptive fusion module processes the data to obtain the final output sequence X. Fusion Its formula is as follows:

[0120] X Fusion =Fusion(X) T X M )

[0121] Wherein, Fusion(·) represents the adaptive fusion module, and its expression formula is as follows:

[0122] Fusion(·)=gX T +(1-g)X M

[0123] Where g is the weight matrix, and g is calculated as follows:

[0124] g = (GeLU(Linear(Sigmoid(X)) T ))))

[0125] Where Sigmoid(·) is the Sigmoid activation function; the resulting X Fusion The data is fed into a classifier for classification to obtain the classification result.

[0126] Step S170: Calculate the classification accuracy on the spectral image dataset based on the obtained classification results.

[0127] This implementation first performs principal component analysis (PCA) to reduce the spectral dimension of the original hyperspectral data. Then, it generates feature data at three different scales through multi-scale convolution, and concatenates these features after applying joint spatial and spectral attention weighting. Next, it performs forward propagation on the concatenated sequence. Simultaneously, the multi-scale features are expanded and processed using four grouping strategies, and then concatenated along the spectral dimension after processing by the S6 module. Finally, the forward propagation results and grouping results are integrated through an adaptive fusion module to output the final feature sequence for classification, and the classification accuracy is calculated.

[0128] To further illustrate, if a hyperspectral image dataset is classified according to this implementation method, a classification result with an accuracy higher than most methods will be obtained.

[0129] Detailed Implementation Results

[0130] This implementation uses the publicly available Indian Pines dataset, specifically 200 valid bands. Details of the dataset are as follows:

[0131] The Indian Pines dataset covers farmland and some forests, buildings, and other features in northwestern Indiana. This dataset contains the following two files:

[0132] The hyperspectral image data file is a three-dimensional data matrix with dimensions of 145*145*220. 145*145 represents the spatial size of the entire scene, which contains 21,025 pixels. The spectral vector corresponding to each pixel contains reflectance values ​​of 220 bands, of which 200 are valid. Each pixel corresponds to a ground resolution of approximately 20 meters by 20 meters.

[0133] The ground truth label file is a 145*145 two-dimensional matrix, where each label pixel corresponds one-to-one with the corresponding pixel position in the hyperspectral data; each element in the matrix is ​​an integer between 0 and 16, representing the land cover category of an element.

[0134] To verify the superiority of this implementation method, it will be compared with several existing spectral image land cover classification methods, including IGroupSS-Mamba, 3DSS-Mamba, SF, and CASST methods. The accuracy of these methods in different categories and the values ​​of three indicators, OA, AA, and Kappa, will be compared on the Indian Pines public dataset. The specific data comparison is shown in Table 1.

[0135] Table 1. Classification accuracy (%) of the Indian Pines dataset.

[0136]

[0137]

[0138] The data comparison in the table above clearly shows that Ours achieved the best performance, significantly improving the accuracy of spectral image land cover classification. The quantitative results fully demonstrate the superiority of Ours, as it can fully explore the joint spatial-spectral features at different scales through multi-scale partitioning and a spatial-spectral joint attention mechanism. It also effectively enhances the diversity of local feature representation by utilizing interval grouping and diversified unfolding strategies. Simultaneously, the adaptive feature fusion module dynamically optimizes multi-source feature representation, enabling the model to more fully explore the spatial-spectral correlation characteristics of hyperspectral data and obtain more discriminative feature representations, thereby significantly improving the classification accuracy of hyperspectral images. Extensive experiments demonstrate that this method outperforms existing methods. Regarding the parameter settings for this implementation, in all experiments, patch_size was set to 15, PCA dimensionality was reduced to 32 dimensions, and the training set ratio was 10%. The model architecture was set to 3 scales, 4 groups, and 4 attention heads for attention calculation. All experiments were trained for 100 epochs to ensure optimal learned parameters.

[0139] This implementation proposes a full-range dual-path hyperspectral image classification method for land cover classification analysis of common hyperspectral images. First, PCA dimensionality reduction is performed on the data. Then, three sets of feature maps are extracted through multi-scale convolution, and each is weighted by spatial-spectral attention before being concatenated. Simultaneously, four grouping strategies (row-first, column-first, skip-group, and checkerboard-style) are used to reconstruct the feature maps, and the spectral features are fused using the S6 module. Finally, the dual-branch features are fused, and the classification result is output through a classifier, and the accuracy is calculated. Experimental results on the public dataset Indian Pines using three metrics demonstrate that this implementation achieves higher classification accuracy and superior performance compared to other methods.

[0140] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A method for classifying dual-path hyperspectral images across all distances, characterized in that, The method includes the following steps: Principal component analysis is performed on the spectral dimension of a hyperspectral dataset X to compress its spectral dimension to c dimensions, resulting in compressed hyperspectral image data X. PCA According to the obtained X PCA The multi-scale segmentation module yields three hyperspectral image data X after convolution at different scales. L XM, XS; Based on the obtained hyperspectral image data at three different scales, X L X M X S Spatial-spectral joint attention calculations were performed separately to obtain the data sequence X at three scales after spatial-spectral joint attention weighting. LT X MT X ST The three data points are then concatenated to obtain the concatenated data sequence X. TCat ; Based on the obtained spliced ​​data sequence X TCat The forward propagation module processes the data to obtain the processed data sequence X. T ; Based on the obtained hyperspectral image data at three different scales, X L X M X S Perform interval grouping to obtain grouping result X. L1 X M1 X S1 X L2 X M2 X S2 X L3 X M3 X S3 X L4 X M4 X S4 ; X L1 X M1 X S1 Divide the data into groups, expand them in row-major order, and concatenate them into a new data sequence X1. L2 X M2 X S2 Divide the data into groups, expand them column-wise, and concatenate them into a new data sequence X2. L3 X M3 X S3 Divide the data into groups, perform jump expansion, and concatenate them into a new data sequence X3. L4 X M4 X S4 Divide the data into groups, unfold them in a chessboard pattern, and then stitch them together to form a new data sequence X4; The obtained spliced ​​sequences X1, X2, X3, and X4 are processed by module S6, and the processing results are spliced ​​along the spectral dimensions to obtain the spliced ​​sequence X. M ; Based on the obtained data sequence X T X M The adaptive fusion module processes the data to obtain the final output sequence X. Fusion The data is then fed into a classifier to obtain the classification result. Based on the obtained classification results, the classification accuracy on the spectral image dataset is calculated.

2. The full-range dual-path hyperspectral image classification method according to claim 1, characterized in that, Principal component analysis is performed on the spectral dimension of a hyperspectral dataset X to compress its spectral dimension to c dimensions, resulting in compressed hyperspectral image data X. PCA According to the obtained X PCA The multi-scale segmentation module yields three hyperspectral image data X after convolution at different scales. L X M X S The definition is as follows: X PCA =PCA(X) X L =Linear L (Conv3D L (X PCA )) X M =Linear M (Conv3D M (X PCA )) X S =Linear S (Conv3D S (X PCA )) Where X represents the hyperspectral image dataset, PCA(·) represents principal component analysis, and Linear... L Linear M Linear S These represent linear layer processing at three scales, Conv3D L Conv3D L Conv3D L These represent 3D convolution processing at three different scales.

3. The full-range dual-path hyperspectral image classification method according to claim 1, characterized in that, Based on the obtained hyperspectral image data at three different scales, X L X M X S Spatial-spectral joint attention calculations were performed separately to obtain the data sequence X at three scales after spatial-spectral joint attention weighting. LT X MT X ST The definition is as follows: X LT =SSAttention(X L ) X MT =SSAttention(X M ) X ST =SSAttention(X S ) Where SSAttention(·) is the joint attention function for the spatial spectrum, and the calculation of SSAttention(·) is as follows: Where ξ(·) represents the softmax activation function, d is the number of multi-head attention heads, and Q, K, V, S, and C are the matrices required for calculating the spatial-spectral joint attention, as shown below: Q=K=V=Linear(X L ) S=AdaptiveAvgPool2D(Conv2D(GeLU(BN(X L ))) C=GeLU(BN(Conv2D(X L ))) Where Linear(·) represents a linear layer, BN(·) represents a batch normalization layer, GeLU(·) represents the GeLU activation function, Conv2D(·) represents 2D convolution, and AdaptiveAvgPool2D(·) represents 2D adaptive average pooling. M and X S The calculation method for the spatial-spectral joint attention is the same; based on the obtained data sequence X after weighting by the spatial-spectral joint attention at three scales... LT X MT X ST The three data sequences are concatenated to obtain the concatenated data sequence X. TCat As shown below: X TCat =cat(X LT ,X MT ,X ST ) Here, cat(·) represents the data sequence concatenation operation.

4. The full-range dual-path hyperspectral image classification method according to claim 1, characterized in that, Based on the obtained spliced ​​data sequence X TCat The forward propagation module processes the data to obtain the processed data sequence X. T The definition is as follows: X T =Mlp(X TCat ) Where Mlp(·) is the forward propagation module, as shown below: Mlp(X TCat )=LN(Linear(Dropout(GeLU(Linear(X TCat ))))) Dropout(·) is the Dropout regularization layer.

5. The full-range dual-path hyperspectral image classification method according to claim 1, characterized in that, Based on the obtained hyperspectral image data at three different scales, X L X M X S Perform interval grouping to obtain grouping result X. L1 X M1 X S1 X L2 X M2 X S2 X L3 X M3 X S3 X L4 X M4 X S4 The definition is as follows: {X L1 ,X L2 ,X L3 ,X L4 }=Interval(X L ) {X M1 ,X M2 ,X M3 ,X M4 }=Interval(X M ) {X S1 ,X S2 ,X S3 ,X S4 }=Interval(X S ) Where Interval(·) represents the interval grouping operation, which sequentially extracts elements from the data sequence to be grouped and places them into the corresponding groups to form the grouping results for each of the three scales; then X L1 X M1 X S1 The data is divided into groups, expanded in row-major order, and then concatenated to form a new data sequence X1, as shown below: X1=cat(Flatten Row (X L1 ,X M1 ,X S1 )) Among them, Flatten Row (·) indicates row-first expansion; X will be processed first. L2 X M2 X S2 Divide the data into a group, expand it column-wise, and concatenate them into a new data sequence X2, as shown below: X2=cat(Flatten Column (X L2 ,X M2 ,X S2 )) Among them, Flatten Column (·) indicates column-first expansion; X will be processed first. L3 X M3 X S3 The data is divided into groups, expanded by skipping, and then concatenated into a new data sequence X3, as shown below: X3=cat(Flatten Jump (X L3 ,X M3 ,X S3 )) Among them, Flatten Jump (·) indicates jump unpacking; X is then processed. L4 X M4 X S4 The data is divided into groups, unfolded in a chessboard pattern, and then concatenated to form a new data sequence X4, as shown below: X4=cat(Flatten Chess (X L4 ,X M4 ,X S4 )) Among them, Flatten Chess (·) indicates that the chessboard is arranged in an alternating pattern.

6. The full-range dual-path hyperspectral image classification method according to claim 1, characterized in that, The obtained spliced ​​sequences X1, X2, X3, and X4 are processed by module S6, and the processing results are spliced ​​along the spectral dimensions to obtain the spliced ​​sequence X. M The definition is as follows: X M =cat(S6(X1、X2、X3、X4)) Where S6(·) represents the processing of module S6, as shown below: the t =Ch t y=[y1,y2,...,y L ] in, C represents the trainable parameters of the S6 module, and x represents the trainable parameters of the S6 module. t h represents the input of module S6 at time t. t and h t-1 Let y represent the hidden states at time t and t-1, respectively, and y be the output sequence, y1, y2, ..., y3. L Let x1, x2, ..., x L The corresponding output.

7. The full-range dual-path hyperspectral image classification method according to claim 1, characterized in that, Based on the obtained data sequence X T X M The adaptive fusion module processes the data to obtain the final output sequence X. Fusion The definition is as follows: X Fusion =Fusion(X T ,X M ) Wherein, Fusion(·) represents the adaptive fusion module, as shown below: Fusion(·)=gX T +(1-g)X M Where g is the weight matrix, and g is calculated as follows: g=(GeLU(Linear(Sigmoid(X T )))) Where Sigmoid(·) is the Sigmoid activation function; the resulting X Fusion The data is fed into a classifier for classification to obtain the classification result.

8. Based on the obtained classification results, calculate the classification accuracy on the spectral image dataset.

Images