A self-supervised clustering method for hyperspectral images based on local-global dual-branch networks

Abstract

This invention belongs to the field of unsupervised hyperspectral image technology and discloses a self-supervised clustering method for hyperspectral images based on a local-global dual-branch network, including: (1) constructing 3D pixel blocks; (2) obtaining shallow depth features through a ResNet module; (3) injecting the shallow features into a dual-path network module; (4) calculating the similarity between local features and cluster centers to obtain the local semantic probability distribution, and then obtaining the target distribution; (5) feeding global features into a feedforward neural network to obtain the probability of each pixel and obtain the global semantic probability distribution; (6) constructing a network loss function through a dual self-supervised mechanism to guide the update of the entire network model. This invention effectively improves the recognition accuracy of hyperspectral objects by introducing a dual-path network architecture to mine local detail features and global semantic features of hyperspectral images.

Description

Technical Field

[0001] This invention belongs to the field of unsupervised hyperspectral image technology, specifically relating to a self-supervised clustering method for hyperspectral images based on a local-global dual-branch network. Background Technology

[0002] Hyperspectral images are three-dimensional cubic data obtained by capturing and imaging the ground using a hyperspectral imager. Unlike the three-channel visible light images in everyday life, hyperspectral images can have dozens or even hundreds of spectral channels, providing much richer spectral information. This spectral information can be used to distinguish different land cover categories, which is helpful for practical applications such as vegetation cover analysis, ecological environment monitoring, and geological exploration. Hyperspectral image classification identifies and analyzes pixels in hyperspectral images based on spectral information to determine the category of land cover in the actual scene corresponding to each pixel. Depending on whether the samples contain label information, hyperspectral image classification includes supervised classification and unsupervised classification. However, obtaining sample labels often requires field surveys, a time-consuming, labor-intensive, and costly process. This is especially true in areas affected by natural disasters or difficult to survey, making field surveys difficult or even impossible. Typically, obtaining a large number of labeled training samples is very difficult; therefore, unsupervised classification or clustering methods have wider application value.

[0003] In recent years, deep learning methods, with their powerful feature extraction capabilities, have been widely applied to unsupervised classification tasks of hyperspectral images. Deep clustering algorithms typically utilize self-supervised learning to construct supervised information to update the network model. For example, regarding subspace-based deep clustering, Junjian Zhang et al. proposed the Self-Supervised Convolutional Subspace Clustering Network (S2ConvSCN CVPR2019), which includes a feature extraction module, a self-representation module, a spectral clustering module, and a classification module. The results of spectral clustering are used to supervise the update of the entire network, forming an end-to-end trainable joint optimization framework. J Lv, Z Kang et al. proposed the Pseudo-Supervised Deep Subspace Clustering (PSSS IEEE2021), which learns the similarity between data through a self-representation layer, uses pairwise similarity to weight the reconstruction loss to capture local structural information, and utilizes the representation learned by the autoencoder and the graph generated by the self-representation to construct pseudo-graphs and pseudo-labels to guide network training. Furthermore, graph clustering is also frequently used in deep clustering. For example, Deyu Bo et al. proposed the Structural Deep Clustering Network (SDCN WWW2020), which stacks several layers of graph neural networks during feature extraction to learn higher-order structural information and also incorporates a dual self-supervised module. While these deep network models mine local details in images based on convolutional neural networks, they lack an understanding and fusion of global image information, failing to fully capture contextual information and thus unable to model the interdependencies between global features. With the introduction of the visual Transformer, it has attracted increasing attention from researchers due to its ability to mine long-range dependencies in images. The Transformer is a deep neural network based on a self-attention mechanism, capable of handling long-range dependencies and parallel computation. Based on these characteristics, the Transformer has demonstrated excellent performance in fields such as image classification. For example, ZeLiu et al. proposed the Swin Transformer (SwinT ICCV2021) model, which, by introducing a moving window, can compute attention within non-overlapping local windows and allows cross-window computation. This method overcomes the problem of lack of connection between windows generated by conventional window partitioning strategies in visual Transformers, thus bringing higher efficiency and lower complexity.Wenhai Wang et al. proposed the Pyramid Vision Transformer (PVTICCV2021), which introduces the pyramid structure of convolutional neural networks into the visual Transformer, enabling it to generate multi-scale features. It also utilizes non-overlapping image patches to reduce sequence length and progressively shrinks the pyramid resolution to reduce feature map computation. Hua-Bao Ling et al. proposed the VisionTransformer for Contrastive Clustering (VTCC arXiv 2022), which introduces the visual Transformer into contrastive learning to achieve deep clustering. It performs two random augmentations on each image, inputting these augmented samples into a visual Transformer encoder with two shared weights to extract feature representations from the two augmented views. An instance projector and a clustering projector are then introduced to process these feature representations for instance-level contrastive learning and global clustering structure learning.

[0004] The Transformer models mentioned above use a single-structure model to learn features. However, hyperspectral images are complex data that combine spatial and spectral elements, and a single structure is insufficient to fully extract the intrinsic features of hyperspectral images. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention utilizes CNN to extract local detail features and leverages the advantages of multi-layer global Transformers to extract long-dependent global features. It proposes a novel hyperspectral self-supervised clustering method based on a local-global dual-path architecture, which effectively improves the recognition accuracy of hyperspectral ground features.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0007] This invention is a self-supervised clustering method for hyperspectral images based on a local-global dual-branch network. The local-global dual-branch network specifically includes a dual-path network module, a ResNet module, and a dual self-supervised module. The dual-path network module consists of a CNN module and a multi-layer global Transformer module. Specifically, the self-supervised clustering method for hyperspectral images includes the following steps:

[0008] Step 1: Preprocess the hyperspectral image to construct a 3D pixel block centered on each pixel.

[0009] Step 2: Input the 3D pixel block obtained in Step 1 into the ResNet module to obtain the shallow depth features of the hyperspectral image;

[0010] Step 3: Inject the shallow deep features obtained in Step 2 into the dual-path network module, namely the CNN module to extract local detail features and the multi-layer global Transformer module to extract global semantic features;

[0011] Step 4: Based on the local detail features obtained by the CNN module in Step 3, the similarity between the feature vector of the local detail features and the cluster center vector of each category using the kernel function is calculated to obtain the predicted distribution of each sample belonging to the category cluster, i.e., the local semantic probability distribution. Then, the local detail feature representation is optimized by learning high confidence assignment to improve the cohesion of the category cluster and calculate the target distribution.

[0012] Step 5: Feed the global semantic features obtained from the multi-layer global Transformer module in Step 3 into the feedforward neural network and use the Softmax function to obtain the global semantic probability distribution.

[0013] Step 6: Based on the predicted distribution and target distribution obtained in Step 4 and the global semantic probability distribution obtained in Step 5, construct a network loss function through a dual self-supervised module to guide the update of the entire local-global dual-branch network.

[0014] A further improvement of the present invention is that the ResNet module in the local-global dual-branch network includes a three-layer residual structure, each residual structure including two convolutional layers, each convolutional layer followed by a batch normalization layer and a ReLU activation function, and the 3D pixel block H pca ∈R p×p×c′ The shallow depth features X are obtained through the ResNet module. s ∈R p ×p×c′ .

[0015] A further improvement of the present invention is that the CNN module in the dual-path network module comprises four convolutional layers, each of which is followed by a ReLU activation function, for the shallow depth features X obtained in step 2. s ∈R p×p×c′ Local detail features X are obtained through a CNN module. l ∈R p×p×z , where z is the number of output channels in the last convolutional layer of the CNN module.

[0016] A further improvement of the present invention is that the multi-layer global Transformer module in the dual-path network module is composed of a stack of multi-layer global Transformer models.

[0017] A further improvement of the present invention is that the dual self-supervised module in the local-global dual-branch network uses KL divergence as a loss function to guide the update of the entire local-global dual-branch network.

[0018] A further improvement of the present invention is that the specific process of the multi-layer global Transformer module extracting global semantic features is as follows:

[0019] Step 3.1: The shallow features X obtained in Step 2... s ∈R p×p×c′ Transformed into X through a 1×1 convolutional layer s ∈R p ×p×dim Where dim represents the dimensions of Q, K, and V in the self-attention mechanism, and is flattened into a two-dimensional vector X∈R (p.p)×dim And add learnable positional codes;

[0020] Step 3.2: First, pass X through a LayerNorm layer and multiply it by three learnable weight matrices W. q W k W v We obtain Q, K, and V, which are then divided into h parts along the spectral dimension, as shown in the following formula:

[0021] X L1 =LayerNorm1(X),

[0022] Q = {Q1, Q2, Q3, ..., Q} h} = Linear(X L1 ) = X L1 W q ,

[0023] K = {K1, K2, K3, ..., K} h} = Linear(X L1 ) = X L1 W k ,

[0024] V = {V1, V2, V3, ..., V} h} = Linear(X L1 ) = X L1 W v ,

[0025] Where h represents the number of heads in the multi-head self-attention;

[0026] Step 3.3, transfer each Q i With each K i The attention score is obtained by performing a dot product operation, and then divided by a factor to prevent the gradient vanishing problem. Then, normalization is performed using Softmax to obtain the attention weights, which are related to V. i Multiplication, specifically represented as:

[0027]

[0028] Among them, Q i K i V i Represents the i-th header: Query, Key, Value, X i Let d represent the result obtained from the i-th self-attention. k K represents i The dimension;

[0029] Step 3.4: After obtaining the attention results for each head in Step 3.3, the results of h heads are concatenated and then fused through a linear layer, as follows:

[0030] X attn (Q,K,V)=Concat(X1,X2,…,X h W o

[0031] Among them, W o It is the projection matrix of the linear layer;

[0032] X obtained in steps 3.5 and 3.4 attn After residual join, the specific representation is as follows:

[0033] X attn ′=X+X attn ;

[0034] X obtained in steps 3.6 and 3.5 attn′ After passing through a LayerNorm layer, MLP, and residual connections, the specific representation is as follows:

[0035] X L2 =LayerNorm2(X attn′ ),

[0036] X hid =Linear(ReLU(Linear(X) L2 ))),

[0037] X hid′ =X attn +X hid ,

[0038] Among them, X hid This is the output of the MLP, X hid′It is the final output result after passing through a global Transformer module.

[0039] The beneficial effects of this invention are as follows: This invention utilizes CNN branches and multi-layer global Transformer branches to extract local detail features and global semantic features of hyperspectral images, respectively. Through a dual self-supervised mechanism, it uses KL divergence as a loss function to guide the training of the entire network model, thereby achieving unsupervised classification of hyperspectral images. Specifically:

[0040] (1) The present invention extracts shallow depth features of hyperspectral images through the ResNet module. The residual connection structure in ResNet can correlate features at different levels, thereby improving the performance of the model.

[0041] (2) This invention improves the recognition accuracy of hyperspectral ground features by introducing a dual-path network architecture to mine local detail features and global semantic features of hyperspectral images.

[0042] (3) This invention uses a dual self-supervised module and KL divergence as the loss function to effectively supervise and guide the training and updating of the entire network model, forming an end-to-end learnable joint optimization network framework, which effectively improves the accuracy of hyperspectral land cover classification. Attached Figure Description

[0043] Figure 1 This is a flowchart of the hyperspectral image self-supervised clustering method of the present invention.

[0044] Figure 2 This is an architecture diagram of the local-global dual-branch network of the present invention.

[0045] Figure 3 These are visualizations of the K-means method, the AE+K-means method, and the method of this invention on the SalinasA dataset. Among them, 3(a) is the actual ground cover distribution map of the SalinasA dataset, 3(b) is the clustering effect map of the SalinasA dataset using the method of this invention, 3(c) is the clustering effect map of the SalinasA dataset using the K-means method, and 3(d) is the clustering effect map of the SalinasA dataset using the AE+K-means method.

[0046] Figure 4 These are visualizations of the classification methods using the K-means method, the AE+K-means method, and the CNTR method of this invention on the Indian Pines dataset. 4(a) is a map showing the actual ground cover distribution in the Indian Pines dataset. Figure 4(b) is the clustering effect diagram of the Indian Pines dataset using the method of this invention, 4(c) is the clustering effect diagram of the Indian Pines dataset using the Kmeans method, and 4(d) is the clustering effect diagram of the Indian Pines dataset using the AE+Kmeans method. Detailed Implementation

[0047] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of the invention, these practical details are not essential.

[0048] This invention is a self-supervised clustering method for hyperspectral images based on a local-global dual-branch network, denoted as CNTR.

[0049] like Figure 2 As shown, the local-global dual-branch network of the present invention specifically includes a dual-path network module, a ResNet module, and a dual self-supervised module. The dual-path network module consists of a CNN module and a multi-layer global Transformer module. The multi-layer global Transformer module in the dual-path network module is composed of stacked multi-layer global Transformer models. The dual self-supervised module in the local-global dual-branch network uses KL divergence as a loss function to guide the update of the entire local-global dual-branch network. The ResNet module in the local-global dual-branch network includes a three-layer residual structure. Each residual structure includes two convolutional layers, each followed by a batch normalization layer and a ReLU activation function. (3D pixel block H) pca ∈R p×p×c′ The shallow depth features X are obtained through the ResNet module. s ∈R p ×p×c′ .

[0050] The CNN module in the dual-path network module consists of four convolutional layers, each followed by a ReLU activation function, which is applied to the shallow depth features X obtained in step 2. s ∈R p×p×c′ Local detail features X are obtained through a CNN module. l ∈R p×p×z , where z is the number of output channels in the last convolutional layer of the CNN module.

[0051] like Figure 1 As shown, the hyperspectral image self-supervised clustering method of the present invention specifically includes the following steps:

[0052] Step 1: Preprocess the hyperspectral image to construct a 3D pixel block centered on each pixel.

[0053] Specifically: Step 1.1, for the original hyperspectral image H∈R h×w×c First, principal component analysis is used to reduce the number of bands in the hyperspectral image H to the number of bands c′, denoted as H. pca ∈R h×w×c′ Where h, w, and c are the length, width, and number of bands of the hyperspectral image, respectively;

[0054] Step 1.2, for H pca ∈R h×w×c′ The edge filling operation is performed, filling pixels with a value of 0, with a filling dimension of p / 2. Then, the filled image is divided into p×p×c segments centered on the pixels. ′ A pixel block of size, where p represents the height and width of the pixel block;

[0055] Step 2: Transfer the 3D pixel block H obtained in Step 1 pca ∈R p×p×c′ The image is fed into a ResNet module to obtain shallow depth features X of the hyperspectral image. s ∈R p×p×c′ .

[0056] Step 3: The shallow depth features X obtained in Step 2 are... s ∈R p×p×c′ Local detail features X are obtained through a CNN module. l ∈R p×p×z , where z is the number of output channels in the last convolutional layer of the CNN module;

[0057] Step 4: The shallow features X obtained in Step 2... s ∈R p×p×c′ Transformed into X through a 1×1 convolutional layer s ∈R p×p×dim Where dim represents the dimensions of Q, K, and V in the self-attention mechanism, and is flattened into a two-dimensional vector X∈R (p.p)×dim And add learnable positional codes;

[0058] Step 5: Convert the two-dimensional vector X∈R obtained in Step 4. (p.p)×dim The data is fed into a multi-layer global Transformer module to extract global semantic features X. g ∈R (p.p)×dim The specific process is as follows:

[0059] Step 5.1: Pass X through a LayerNorm layer and multiply it by three learnable weight matrices W. q W k Wv We obtain Q, K, and V, which are then divided into h parts along the spectral dimension, as shown in the following formula:

[0060] X L1 =LayerNorm1(X),

[0061] Q = {Q1, Q2, Q3, ..., Q} h} = Linear(X L1 ) = X L1 W q ,

[0062] K = {K1, K2, K3, ... K} h} = Linear(X L1 ) = X L1 W k ,

[0063] V = {V1, V2, V3, ..., V} h} = Linear(X L1 ) = X L1 W v ,

[0064] Where h represents the number of heads in the multi-head self-attention;

[0065] Step 5.2, transfer each Q i With each K i The attention score is obtained by performing a dot product operation, and then divided by a factor to prevent the gradient vanishing problem. Then, normalization is performed using Softmax to obtain the attention weights, which are related to V. i Multiplication, specifically represented as:

[0066]

[0067] Among them, Q i K i V i Represents the i-th header: Query, Key, Value, X i Let d represent the result obtained from the i-th self-attention. k K represents i The dimension;

[0068] Step 5.3: After obtaining the attention results for each head in Step 5.2, the results of h heads are concatenated and then fused through a linear layer, as follows:

[0069] X attn (Q,K,V)=Concat(X1,X2,...,X h W o

[0070] Among them, W o It is the projection matrix of the linear layer;

[0071] X obtained in steps 5.4 and 5.3 attn After residual join, the specific representation is as follows:

[0072] X attn′ =X+X attn ;

[0073] X obtained in steps 5.5 and 5.4 attn′ After passing through a LayerNorm layer, MLP, and residual connections, the specific representation is as follows:

[0074] X L2 =LayerNorm2(X attn′ ),

[0075] X hid =Linear(ReLU(Linear(X) L2 ))),

[0076] X hid′ =X attn +X hid ,

[0077] Among them, X hid This is the output of the MLP, X hid′ It is the final output result after passing through a global Transformer module.

[0078] Step 6: Based on the local detail features X obtained by the CNN module in Step 3 l ∈R p×p×z The similarity between the feature vectors of the local detailed features and the cluster center vectors of each category using the t-distribution as the kernel function is calculated to obtain the predicted distribution of each sample belonging to a category cluster, i.e., the local semantic probability distribution. The formula for calculating the local semantic probability distribution is as follows:

[0079]

[0080] Among them, h i Let μ represent the i-th depth feature. j Let represent the central eigenvector of the j-th cluster, v be the degrees of freedom of the t-distribution (set to 1), and q be the eigenvector of the j-th cluster. ij The probability of assigning sample i to cluster j, i.e., soft assignment, is Q = [q ij This is considered as the predicted distribution of all samples.

[0081] The obtained predicted distribution Q is sharpened to obtain a high-confidence assignment, and the target distribution P is constructed using the following formula:

[0082]

[0083] Among them, f j =∑ i q ij This indicates the frequency of soft clustering.

[0084] Step 7: Combine the global semantic features X obtained from the multi-layer global Transformer module. g ∈R (p·p)×dim The data is fed into a feedforward neural network and the global semantic probability distribution Z is obtained using the Softmax function.

[0085] Step 8: Based on the predicted and target distributions obtained in Step 4 and the global semantic probability distribution obtained in Step 5, a network loss function is constructed using a dual self-supervised module to guide the update of the entire local-global dual-branch network. Specifically, using the dual self-supervised mechanism with KL divergence as the loss function, the difference between the local semantic probability distribution Q and the target probability distribution P, and the difference between the global semantic probability distribution Z and the target probability distribution P obtained in Step 9), are reduced to unify the local-global dual-branch network and guide its training and update. The formula for the KL divergence loss function is as follows:

[0086]

[0087] L=αL clus +βL pred ,

[0088] Where L represents the total loss function of the model, and α>0 and β>0 are tradeoff coefficients, which are hyperparameters.

[0089] The specific experimental data are shown in Tables 1 and 2.

[0090] Table 1

[0091]

[0092] Table 2

[0093]

[0094]

[0095] Table 1 presents a quantitative evaluation of different clustering algorithms for the SalinasA dataset (ACC, ARI, AMI, NMI, homogeneity, completeness, V_measure, FMI (%)).

[0096] Table 2 presents a quantitative evaluation of different clustering algorithms for the Indian Pines dataset (ACC, ARI, AMI, NMI, homogeneity, completeness, V_measure, FMI (%)).

[0097] As can be seen from Tables 1 and 2, the CNTR method proposed in this invention, compared with Kmeans and AE-Kmeans methods, makes full use of CNN and multi-layer global Transformer structure to extract local details and global depth features, and achieves better clustering accuracy.

[0098] Figure 3 and Figure 4 Visualized classification maps of the K-means method, the AE+K-means method, and the proposed CNTR method on the SalinasA and Indian Pines datasets are presented. The maps show that, compared to the other two methods, the proposed CNTR method obtains more uniform regions and has the fewest misclassified pixels. The classification results demonstrate that the CNTR method significantly enhances the ability to identify similar land features.

[0099] This invention improves the accuracy of hyperspectral ground feature identification by introducing a dual-path network architecture to mine local detail features and global semantic features of hyperspectral images.

[0100] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A self-supervised clustering method for hyperspectral images based on local-global dual-branch networks, characterized in that: The local-global dual-branch network specifically includes a dual-path network module, a ResNet module, and a dual self-supervised module. The dual-path network module consists of a CNN module and a multi-layer global Transformer module. Specifically, the hyperspectral image self-supervised clustering method includes the following steps: Step 1: Preprocess the hyperspectral image to construct a 3D pixel block centered on each pixel. Step 2: Input the 3D pixel block obtained in Step 1 into the ResNet module to obtain the shallow depth features of the hyperspectral image; Step 3: Inject the shallow depth features obtained in Step 2 into the dual-path network module, namely the CNN module to extract local detail features and the multi-layer global Transformer module to extract global semantic features; Step 4: Based on the local detail features obtained by the CNN module in Step 3, the similarity between the feature vector of the local detail features and the cluster center vector of each category using the kernel function is calculated to obtain the predicted distribution of each sample belonging to the category cluster, i.e., the local semantic probability distribution. Then, the local detail feature representation is optimized by learning high confidence assignment to improve the cohesion of the category cluster and calculate the target distribution. Step 5: Feed the global semantic features obtained from the multi-layer global Transformer module in Step 3 into the feedforward neural network and use the Softmax function to obtain the global semantic probability distribution. Step 6: Based on the predicted distribution and target distribution obtained in Step 4 and the global semantic probability distribution obtained in Step 5, construct a network loss function through a dual self-supervised module to guide the update of the entire local-global dual-branch network.

2. The hyperspectral image self-supervised clustering method based on local-global dual-branch networks according to claim 1, characterized in that: The ResNet module in the local-global dual-branch network includes a three-layer residual structure. Each residual structure includes two convolutional layers, followed by a batch normalization layer and a ReLU activation function. The 3D pixel block H... pca ∈R p×p×c′ The shallow depth features X are obtained through the ResNet module. s ∈R p×p×c′ , where p represents the height and width of the pixel block.

3. The hyperspectral image self-supervised clustering method based on local-global dual-branch networks according to claim 2, characterized in that: The CNN module in the dual-path network module consists of four convolutional layers, each followed by a ReLU activation function, which is applied to the shallow depth features X obtained in step 2. s ∈R p×p×c′ Local detail features X are obtained through a CNN module. l ∈R p×p×z , where z is the number of output channels in the last convolutional layer of the CNN module.

4. The hyperspectral image self-supervised clustering method based on local-global dual-branch networks according to claim 3, characterized in that: The specific process by which the multi-layer global Transformer module extracts global semantic features is as follows: Step 3.1: The shallow features X obtained in Step 2... s ∈R p×p×c′ Transformed into X through a 1×1 convolutional layer s ∈R p×p×dim Where dim represents the dimensions of Q, K, and V in the self-attention mechanism, and is flattened into a two-dimensional vector X∈R (p.p)×dim And add learnable positional codes; Step 3.2: First, pass X through a LayerNorm layer and multiply it by three learnable weight matrices W. q W k W v We obtain Q, K, and V, which are then divided into h parts along the spectral dimension, as shown in the following formula: X L1 =LayerNorm1(X), Q={Q1,Q2,Q3,...,Q h }=Linear(X L1 )=X L1 ,W q , K={K1,K2,K3,...,K h }=Linear(X L1 )=X L1 W K , V={V1,V2,V3,...,V h }=Linear(X L1 )=X L1 W v , Where h represents the number of heads in the multi-head self-attention; Step 3.3, transfer each Q i With each K i The attention score is obtained by performing a dot product operation, and then divided by a factor to prevent the gradient vanishing problem. Then, normalization is performed using Softmax to obtain the attention weights, which are related to V. i Multiplication, specifically represented as: Among them, Q i K i V i Represents the i-th header: Query, Key, Value, X i Let d represent the result obtained from the i-th self-attention. k K represents i The dimension; Step 3.4: After obtaining the attention results for each head in Step 3.3, the results of h heads are concatenated and then fused through a linear layer, as follows: X attn (Q,K,V)=Concat(X1,X2,...,X h )W o Among them, W o It is the projection matrix of the linear layer; X obtained in steps 3.5 and 3.4 attn After residual join, the specific representation is as follows: X attn′ =X+X attn ; X obtained in steps 3.6 and 3.5 attn′ After passing through a LayerNorm layer, MLP, and residual connections, the specific representation is as follows: X L2 =LayerNorm2(X attn′ ), X hid =Linear(ReLU(Linear(X L2 ))), X hid′ =X attn +X hid , Among them, X hid This is the output of the MLP, X hid′ It is the final output result after passing through a global Transformer module.

5. The hyperspectral image self-supervised clustering method based on local-global dual-branch networks according to claim 1, characterized in that: The multi-layer global Transformer module in the dual-path network module is composed of a stack of multi-layer global Transformer models.

6. The hyperspectral image self-supervised clustering method based on local-global dual-branch networks according to claim 1, characterized in that: The dual self-supervised module in the local-global dual-branch network uses KL divergence as a loss function to guide the update of the entire local-global dual-branch network.

Images