Hyperspectral image self-supervised clustering method based on local-global double-branch network

Abstract

The invention belongs to the technical field of hyperspectral image unsupervised, and discloses a hyperspectral image self-supervised clustering method based on a local-global double-branch network, which comprises the following steps: (1) constructing a 3D pixel block; (2) acquiring shallow depth features through a ResNet module; (3) injecting shallow layer features into a dual-path network module; (4) calculating the similarity between the local features and the clustering center to obtain local semantic probability distribution, and then obtaining target distribution; (5) the global features are sent to a feedforward neural network to obtain the probability of each pixel, and global semantic probability distribution is obtained; and (6) constructing a network loss function through a dual self-supervision mechanism to guide the updating of the whole network model. According to the method, the local detail features and the global semantic features of the hyperspectral image are mined by introducing the dual-path network architecture, so that the recognition precision of the hyperspectral ground features is effectively improved.

Description

Technical Field

[0001] The invention belongs to the technical field of unsupervised hyperspectral image, and specifically relates to a hyperspectral image self-supervised clustering method based on a local-global double-branch network. Background Art

[0002] Hyperspectral images are three-dimensional cube data obtained by taking pictures and imaging the ground with a hyperspectral imager. Unlike three-channel visible light images in life, the number of spectral channels of hyperspectral images can reach dozens or even hundreds, which can provide richer spectral information. These spectral information can be used to distinguish different types of land objects, which is helpful for practical applications such as vegetation cover analysis, ecological environment detection, and geological exploration. Hyperspectral image classification identifies and analyzes the pixels in the hyperspectral image according to the spectral information, and determines the type of land objects in the actual scene corresponding to each pixel. Depending on whether the sample contains label information, hyperspectral image classification includes supervised classification and unsupervised classification. However, obtaining sample labels often requires field surveys, which is time-consuming, labor-intensive, and costly. Especially in areas where natural disasters occur or are difficult to survey, field surveys become difficult or even impossible. Generally speaking, it is very difficult to obtain a large number of labeled training samples, so unsupervised classification or clustering methods have a wider range of application value.

[0003] In recent years, deep learning methods have been widely used in unsupervised classification tasks of hyperspectral images due to their powerful feature extraction capabilities. Deep clustering algorithms usually use self-supervised learning to build supervised information to update network models. For example, based on subspace deep clustering, Junjian Zhang et al. proposed a self-supervised convolutional subspace clustering network (S2ConvSCN CVPR2019), which includes a feature extraction module, a self-expression module, a spectral clustering module, and a classification module. The results of spectral clustering are used to supervise the update of the entire network, forming an end-to-end trainable joint optimization framework. J Lv, Z Kang et al. proposed a pseudo-supervised deep subspace clustering network (PSSS IEEE2021), which learns the similarity between data through a self-expression layer, uses pairwise similarity to weight the reconstruction loss to capture local structural information, and uses the representation learned by the autoencoder and the graph generated by the self-expression to construct pseudo-graphs and pseudo-labels to guide network training. In addition, graph clustering is also commonly used in deep clustering. For example, Deyu Bo et al. proposed the Structural Deep Clustering Network (SDCN WWW2020), which stacked several layers of graph neural networks in the feature extraction process to learn high-order structural information, and also added a dual self-supervision module. The above deep network models are based on convolutional neural networks to mine local detail information of images, but lack the understanding and fusion of global information of images, and cannot fully capture the contextual information in the image, so it is impossible to model the interdependence between global features. With the introduction of visual Transformer, Transformer has attracted more and more attention from researchers because of its ability to mine long-range dependencies of images. Transformer is a deep neural network based on the self-attention mechanism that can handle long-distance dependencies and can be calculated in parallel. Based on these characteristics, Transformer has shown excellent performance in fields such as image classification. For example, ZeLiu et al. proposed the Swin Transformer (SwinT ICCV2021) model, which, by introducing a moving window, can calculate attention within non-overlapping local windows and allow cross-window calculations. This method overcomes the problem of lack of connection between windows generated by the conventional window division strategy in the visual Transformer, thereby bringing higher efficiency and lower complexity.Wenhai Wang et al. proposed the Pyramid Vision Transformer (PVTICCV2021), which introduced the pyramid structure of the convolutional neural network into the visual transformer, enabling it to generate multi-scale features, and using non-overlapping image blocks to reduce the length of the sequence, and the progressive shrinking of the pyramid resolution can reduce the calculation of the feature map. Hua-Bao Ling et al. proposed the Vision Transformer for Contrastive Clustering (VTCC arXiv 2022), which introduced the visual transformer into contrastive learning to achieve deep clustering. Each image is randomly enhanced twice, and the two enhanced samples are input into the visual transformer encoder with two weight sharing, and the feature representation of the two enhanced views is extracted. The instance projector and clustering projector are introduced to process the feature representation for instance-level contrastive learning and global clustering structure learning.

[0004] The above Transformer models use a single structure model to learn features. However, hyperspectral images are complex data that combines space and spectrum, and a single structure is difficult to fully explore the intrinsic features of hyperspectral images. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention utilizes CNN to extract local detail features and uses the advantages of multi-layer global Transformer to extract long-dependent global features, and proposes a new hyperspectral self-supervised clustering method based on local-global dual-path architecture. This method effectively improves the recognition accuracy of hyperspectral objects.

[0006] In order to achieve the above object, the present invention is achieved through the following technical solutions:

[0007] The present invention is a hyperspectral image self-supervised clustering method based on a local-global dual-branch network, wherein the local-global dual-branch network specifically includes a dual-path network module, a ResNet module and a dual self-supervisory module, wherein the dual-path network module is composed of a CNN module and a multi-layer global Transformer module. Specifically, the hyperspectral image self-supervised clustering method specifically includes the following steps:

[0008] Step 1: preprocess the hyperspectral image and construct a 3D pixel block centered at each pixel point;

[0009] Step 2: Send the 3D pixel block obtained in step 1 to the ResNet module to obtain the shallow depth features of the hyperspectral image;

[0010] Step 3: Inject the shallow depth features obtained in step 2 into the dual-path network module, namely, the CNN module to extract local detail features and the multi-layer global Transformer module to extract global semantic features;

[0011] Step 4: Based on the local detail features obtained by the CNN module in step 3, the kernel function is used to calculate the similarity between the feature vector of the local detail features and the kmeans algorithm clustering center vector of each category, and the predicted distribution of each sample belonging to the category cluster, that is, the local semantic probability distribution, and then the local detail feature representation is optimized by learning high confidence distribution, the cohesion of the category cluster is improved, and the target distribution is calculated;

[0012] Step 5: Send the global semantic features obtained by the multi-layer global Transformer module in step 3 into the feedforward neural network and use the Softmax function to obtain the global semantic probability distribution;

[0013] Step 6: Based on the predicted distribution and target distribution obtained in step 4 and the global semantic probability distribution obtained in step 5, a network loss function is constructed through a dual self-supervision module to guide the update of the entire local-global dual-branch network.

[0014] A further improvement of the present invention is that the ResNet module in the local-global dual-branch network includes a three-layer residual structure, each of which includes two convolutional layers, each of which is followed by a batch normalization layer and a ReLU activation function, and the 3D pixel block H pca ∈R p×p×c′ The shallow depth feature X is obtained through the ResNet module s ∈R p ×p×c′ .

[0015] A further improvement of the present invention is that the CNN module in the dual-path network module comprises four convolutional layers, each of which is followed by a ReLU activation function, and the shallow depth feature X obtained in step 2 is s ∈R p×p×c′ Obtain local detail features X through the CNN module l ∈R p×p×z , where z is the number of output channels in the last convolutional layer in the CNN module.

[0016] A further improvement of the present invention is that the multi-layer global Transformer module in the dual-path network module is composed of a multi-layer global Transformer model stack.

[0017] A further improvement of the present invention is that the dual self-supervision module in the local-global dual-branch network uses KL divergence as a loss function to guide the update of the entire local-global dual-branch network.

[0018] A further improvement of the present invention is that the specific process of extracting global semantic features by the multi-layer global Transformer module is as follows:

[0019] Step 3.1: The shallow feature X obtained in step 2 s ∈R p×p×c′ Transformed into X through a 1×1 convolutional layer s ∈R p ×p×dim , where dim represents the dimensions of Q, K, V in the self-attention mechanism and is flattened into a two-dimensional vector X∈R (p.p)×dim , and add learnable positional encodings;

[0020] Step 3.2: Pass X through the LayerNorm layer and multiply it by three learnable weight matrices W respectively. q , W k , W v , we get Q, K, and V, which are divided into h parts along the spectral dimension respectively. The formula is as follows:

[0021] X L1 =LayerNorm1(X),

[0022] Q={Q1,Q2,Q3,...,Q h}=Linear(X L1 )=X L1 W q ,

[0023] K={K1,K2,K3,...,K h}=Linear(X L1 )=X L1 W k ,

[0024] V={V1,V2,V3,...,V h}=Linear(X L1 )=X L1 W v ,

[0025] Among them, h represents the number of heads in multi-head self-attention;

[0026] Step 3.3: Each Q i With each K i Perform a dot product operation to get the attention score, and divide it by Then use Softmax to normalize and get the attention weight, which is consistent with V i Multiplication, specifically expressed as:

[0027]

[0028] Among them, Q i , K i , V i Indicates the i-th head Query, Key, Value, X i represents the result of the i-th self-attention, d k K i Dimensions;

[0029] Step 3.4: After obtaining the attention results of each head in step 3.3, concatenate the results of h heads and then fuse them through a linear layer, as follows:

[0030] X attn (Q,K,V)=Concat(X1,X2,…,X h )W o

[0031] Among them, W o is the projection matrix of the linear layer;

[0032] Step 3.5, X obtained in step 3.4 attn After the residual connection, the specific expression is as follows:

[0033] X attn ′=X+X attn ;

[0034] Step 3.6, X obtained in step 3.5 attn′ After another LayerNorm layer, MLP and residual connection, the specific representation is as follows:

[0035] X L2 =LayerNorm2(X attn′ ),

[0036] X hid =Linear(ReLU(Linear(X L2 ))),

[0037] X hid′ =X attn +X hid ,

[0038] Among them, X hid is the output of MLP, X hid′It is the final output result after a layer of global Transformer module.

[0039] The beneficial effects of the present invention are as follows: the present invention utilizes CNN branches and multi-layer global Transformer branches to extract local detail features and global semantic features of hyperspectral images respectively, and uses KL divergence as a loss function through a dual self-supervision mechanism to guide the training of the entire network model, thereby achieving unsupervised classification of hyperspectral images. Specifically:

[0040] (1) The present invention extracts shallow depth features of hyperspectral images through the ResNet module. The residual connection structure in ResNet can correlate features at different levels to improve the representation and performance of the model.

[0041] (2) The present invention introduces a dual-path network architecture to mine the local detail features and global semantic features of hyperspectral images, thereby effectively improving the recognition accuracy of hyperspectral objects.

[0042] (3) The present invention employs a dual self-supervision module and uses KL divergence as the loss function to effectively supervise and guide the training and updating of the entire network model, forming an end-to-end learnable joint optimization network framework, which effectively improves the accuracy of hyperspectral object classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the hyperspectral image self-supervised clustering method of the present invention.

[0044] Figure 2 It is an architecture diagram of the local-global dual-branch network of the present invention.

[0045] Figure 3 3(a) is a visualization classification diagram of the Kmeans method, the AE+Kmeans method and the method of the present invention on the SalinasA dataset, wherein 3(b) is a clustering effect diagram of the SalinasA dataset using the method of the present invention, 3(c) is a clustering effect diagram of the SalinasA dataset using the Kmeans method, and 3(d) is a clustering effect diagram of the SalinasA dataset using the AE+Kmeans method.

[0046] Figure 4 is a visualized classification diagram of the Kmeans method, the AE+Kmeans method and the CNTR method of the present invention on the Indian Pines dataset, wherein 4(a) is a real feature distribution map of the Indian Pines dataset, Figure 4(b) is a clustering effect diagram of the Indian Pines dataset using the method of the present invention, 4(c) is a clustering effect diagram of the Indian Pines dataset using the Kmeans method, and 4(d) is a clustering effect diagram of the Indian Pines dataset using the AE+Kmeans method. DETAILED DESCRIPTION

[0047] The following will disclose the embodiments of the present invention with drawings. For the purpose of clear description, many practical details will be described together in the following description. However, it should be understood that these practical details should not be used to limit the present invention. That is to say, in some embodiments of the present invention, these practical details are not necessary.

[0048] The present invention is a hyperspectral image self-supervised clustering method based on a local-global dual-branch network, denoted as CNTR.

[0049] like Figure 2 As shown, the local-global dual-branch network of the present invention specifically includes a dual-path network module, a ResNet module and a dual self-supervision module. The dual-path network module is composed of a CNN module and a multi-layer global Transformer module. The multi-layer global Transformer module in the dual-path network module is composed of a multi-layer global Transformer model stack; the dual self-supervision module in the local-global dual-branch network uses KL divergence as a loss function to guide the update of the entire local-global dual-branch network. The ResNet module in the local-global dual-branch network includes a three-layer residual structure, each of which includes two convolutional layers, each of which is followed by a batch normalization layer and a ReLU activation function, and the 3D pixel block H pca ∈R p×p×c′ The shallow depth feature X is obtained through the ResNet module s ∈R p ×p×c′ .

[0050] The CNN module in the dual-path network module consists of four convolutional layers, each of which is followed by a ReLU activation function to activate the shallow depth feature X obtained in step 2. s ∈R p×p×c′ Obtain local detail features X through the CNN module l ∈R p×p×z , where z is the number of output channels in the last convolutional layer in the CNN module.

[0051] like Figure 1 As shown, the hyperspectral image self-supervised clustering method of the present invention specifically includes the following steps:

[0052] Step 1: Preprocess the hyperspectral image and construct a 3D pixel block centered at each pixel.

[0053] Specifically: Step 1.1, for the original hyperspectral image H∈R h×w×c First, principal component analysis is used to reduce the bands of the hyperspectral image H to the number of bands of c′, denoted as H pca ∈R h×w×c′ , where h, w and c are the length, width and number of bands of the hyperspectral image respectively;

[0054] Step 1.2: H pca ∈R h×w×c′ The edge filling operation is performed, the pixel value is filled with 0 pixels, the filling dimension is p / 2, and then the filled image is divided into p×p×c with the pixel point as the center ′ A pixel block of size p, where p represents the height and width of the pixel block;

[0055] Step 2: The 3D pixel block H obtained in step 1 pca ∈R p×p×c′ Send it to the ResNet module to obtain the shallow depth feature X of the hyperspectral image s ∈R p×p×c′ .

[0056] Step 3: The shallow depth feature X obtained in step 2 s ∈R p×p×c′ Obtain local detail features X through the CNN module l ∈R p×p×z , where z is the number of output channels in the last convolutional layer in the CNN module;

[0057] Step 4: The shallow feature X obtained in step 2 s ∈R p×p×c′ Transformed into X through a 1×1 convolutional layer s ∈R p×p×dim , where dim represents the dimensions of Q, K, V in the self-attention mechanism and is flattened into a two-dimensional vector X∈R (p.p)×dim , and add learnable positional encodings;

[0058] Step 5: Substitute the two-dimensional vector X∈R obtained in step 4 (p.p)×dim Send it to the multi-layer global Transformer module to extract the global semantic feature X g ∈R (p.p)×dim The specific process is:

[0059] Step 5.1: Pass X through the LayerNorm layer and multiply it by three learnable weight matrices W respectively. q , W k , Wv , we get Q, K, and V, which are divided into h parts along the spectral dimension respectively. The formula is as follows:

[0060] X L1 =LayerNorm1(X),

[0061] Q={Q1,Q2,Q3,…,Q h}=Linear(X L1 )=X L1 W q ,

[0062] K={K1,K2,K3,…K h}=Linear(X L1 )=X L1 W k ,

[0063] V={V1,V2,V3,...,V h}=Linear(X L1 )=X L1 W v ,

[0064] Among them, h represents the number of heads in multi-head self-attention;

[0065] Step 5.2: Each Q i With each K i Perform a dot product operation to get the attention score, and divide it by Then use Softmax to normalize and get the attention weight, which is consistent with V i Multiplication, specifically expressed as:

[0066]

[0067] Among them, Q i , K i , V i Indicates the i-th head Query, Key, Value, X i represents the result of the i-th self-attention, d k K i Dimensions;

[0068] Step 5.3: After obtaining the attention results of each head in step 5.2, concatenate the results of h heads and then fuse them through a linear layer, as follows:

[0069] X attn (Q,K,V)=Concat(X1,X2,...,X h )W o

[0070] Among them, W o is the projection matrix of the linear layer;

[0071] Step 5.4, X obtained in step 5.3 attn After the residual connection, the specific expression is as follows:

[0072] X attn′ =X+X attn ;

[0073] Step 5.5, X obtained in step 5.4 attn′ After another LayerNorm layer, MLP and residual connection, the specific representation is as follows:

[0074] X L2 =LayerNorm2(X attn′ ),

[0075] X hid =Linear(ReLU(Linear(X L2 ))),

[0076] X hid′ =X attn +X hid ,

[0077] Among them, X hid is the output of MLP, X hid′ It is the final output result after a layer of global Transformer module.

[0078] Step 6: Based on the local detail feature X obtained by the CNN module in step 3 l ∈R p×p×z , using t distribution as the kernel function to calculate the similarity between the feature vector of the local detail feature and the kmeans algorithm clustering center vector of each category, and obtain the predicted distribution of each sample belonging to the category cluster, that is, the local semantic probability distribution. The calculation formula of the local semantic probability distribution is as follows:

[0079]

[0080] Among them, h i represents the i-th deep feature, μ j represents the central eigenvector of the jth cluster, v is the degree of freedom of the t distribution, set to 1, and q ij is the probability of assigning sample i to cluster j, i.e., soft assignment, Q = [q ij ] is considered as the predicted distribution of all samples.

[0081] The obtained prediction distribution Q is sharpened to obtain a high confidence distribution and construct the target distribution P. The calculation formula is as follows:

[0082]

[0083] Among them, f j =∑ i q ij Represents the soft clustering frequency.

[0084] Step 7: The global semantic features X obtained by the multi-layer global Transformer module g ∈R (p·p)×dim Send it into the feedforward neural network and use the Softmax function to obtain the global semantic probability distribution Z;

[0085] Step 8: Based on the predicted distribution and target distribution obtained in step 4 and the global semantic probability distribution obtained in step 5, a network loss function is constructed through a dual self-supervision module to guide the update of the entire local-global dual-branch network. Specifically, the dual self-supervision mechanism uses KL divergence as a loss function to unify the local-global dual-branch network and guide the training update of the local-global dual-branch network by reducing the difference between the local semantic probability distribution Q and the target probability distribution P and the difference between the global semantic probability distribution Z and the target probability distribution P obtained in step 9). The formula of the KL divergence loss function is as follows:

[0086]

[0087] L=αL clus +βL pred ,

[0088] Among them, L represents the total loss function of the model, α>0 and β>0 are trade-off coefficients and are hyperparameters.

[0089] The specific experimental data are shown in Table 1 and Table 2.

[0090] Table 1

[0091]

[0092] Table 2

[0093]

[0094]

[0095] Table 1 is the quantitative evaluation of different clustering algorithms for the SalinasA dataset (ACC, ARI, AMI, NMI, homogeneity, completeness, V_measure, FMI (%))

[0096] Table 2 is the quantitative evaluation of different clustering algorithms for the Indian Pines dataset (ACC, ARI, AMI, NMI, homogeneity, completeness, V_measure, FMI (%))

[0097] It can be seen from Table 1 and Table 2 that the method CNTR proposed in the present invention, compared with the Kmeans and AE-Kmeans methods, makes full use of CNN and multi-layer global Transformer structure to extract local details and global depth features, and achieves better clustering accuracy.

[0098] Figure 3 and Figure 4 The visual classification diagrams of the Kmeans method, AE+Kmeans method and the CNTR method of the present invention on the SalinasA dataset and the Indian Pines dataset are shown. It can be seen from the figure that compared with the other two methods, the proposed CNTR obtains a more uniform area with the least misclassified pixels. The classification results show that the CNTR method can significantly enhance the recognition ability of similar objects.

[0099] The present invention introduces a dual-path network architecture to mine local detail features and global semantic features of hyperspectral images, thereby effectively improving the recognition accuracy of hyperspectral objects.

[0100] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A hyperspectral image self-supervised clustering method based on a local-global dual-branch network, characterized by: The local-global dual-branch network specifically includes a dual-path network module, a ResNet module and a dual self-supervision module. The dual-path network module is composed of a CNN module and a multi-layer global Transformer module. Specifically, the hyperspectral image self-supervised clustering method specifically includes the following steps: Step 1: preprocess the hyperspectral image and construct a 3D pixel block centered at each pixel point; Step 2: Send the 3D pixel block obtained in step 1 to the ResNet module to obtain the shallow depth features of the hyperspectral image; Step 3: Inject the shallow depth features obtained in step 2 into the dual-path network module, namely, the CNN module to extract local detail features and the multi-layer global Transformer module to extract global semantic features; Step 4: Based on the local detail features obtained by the CNN module in step 3, the kernel function is used to calculate the similarity between the feature vector of the local detail features and the kmeans algorithm clustering center vector of each category, and the predicted distribution of each sample belonging to the category cluster, that is, the local semantic probability distribution, and then the local detail feature representation is optimized by learning high confidence distribution, the cohesion of the category cluster is improved, and the target distribution is calculated; Step 5: Send the global semantic features obtained by the multi-layer global Transformer module in step 3 into the feedforward neural network and use the Softmax function to obtain the global semantic probability distribution; Step 6: Based on the predicted distribution and target distribution obtained in step 4 and the global semantic probability distribution obtained in step 5, a network loss function is constructed through a dual self-supervision module to guide the update of the entire local-global dual-branch network.

2. The hyperspectral image self-supervised clustering method based on a local-global dual-branch network according to claim 1, characterized in that: The ResNet module in the local-global dual-branch network includes a three-layer residual structure, each of which includes two convolutional layers, each of which is followed by a batch normalization layer and a ReLU activation function. The 3D pixel block H pca ∈R p×p×c′ The shallow depth feature X is obtained through the ResNet module s ∈R p×p×c′ , where p represents the height and width of the pixel block.

3. The hyperspectral image self-supervised clustering method based on a local-global dual-branch network according to claim 2, characterized in that: The CNN module in the dual-path network module consists of four convolutional layers, each of which is followed by a ReLU activation function to activate the shallow depth feature X obtained in step 2. s ∈R p×p×c′ Obtain local detail features X through the CNN module l ∈R p×p×z , where z is the number of output channels in the last convolutional layer in the CNN module.

4. The method for self-supervised clustering of hyperspectral images based on a local-global dual-branch network according to claim 3, characterized in that: The specific process of extracting global semantic features by the multi-layer global Transformer module is as follows: Step 3.1: The shallow feature X obtained in step 2 s ∈R p×p×c′ Transformed into X through a 1×1 convolutional layer s ∈R p×p×dim , where dim represents the dimensions of Q, K, V in the self-attention mechanism and is flattened into a two-dimensional vector X∈R (p.p)×dim , and add learnable positional encodings; Step 3.2: Pass X through the LayerNorm layer and multiply it by three learnable weight matrices W respectively. q , W k , W v , we get Q, K, and V, which are divided into h parts along the spectral dimension respectively. The formula is as follows: X L1 =LayerNorm1(X), Q={Q1,Q2,Q3,...,Q h }=Linear(X L1 )=X L1 ,W q , K={K1,K2,K3,...,K h }=Linear(X L1 )=X L1 W K , V={V1,V2,V3,...,V h }=Linear(X L1 )=X L1 W v , Among them, h represents the number of heads in multi-head self-attention; Step 3.3: Each Q i With each K i Perform a dot product operation to get the attention score, and divide it by Then use Softmax to normalize and get the attention weight, which is consistent with V i Multiplication, specifically expressed as: Among them, Q i , K i , V i Indicates the i-th head Query, Key, Value, X i represents the result of the i-th self-attention, d k K i Dimensions; Step 3.4: After obtaining the attention results of each head in step 3.3, concatenate the results of h heads and then fuse them through a linear layer, as follows: X attn (Q,K,V)=Concat(X1,X2,...,X h )W o Among them, W o is the projection matrix of the linear layer; Step 3.5, X obtained in step 3.4 attn After the residual connection, the specific expression is as follows: X attn′ =X+X attn ; Step 3.6, X obtained in step 3.5 attn′ After another LayerNorm layer, MLP and residual connection, the specific representation is as follows: X L2 =LayerNorm2(X attn′ ), X hid =Linear(ReLU(Linear(X L2 ))), X hid′ =X attn +X hid , Among them, X hid is the output of MLP, X hid′ It is the final output result after a layer of global Transformer module.

5. The hyperspectral image self-supervised clustering method based on a local-global dual-branch network according to claim 1, characterized in that: The multi-layer global Transformer module in the dual-path network module is composed of a multi-layer global Transformer model stack.

6. The method for self-supervised clustering of hyperspectral images based on a local-global dual-branch network according to claim 1, characterized in that: The dual self-supervision module in the local-global dual-branch network uses KL divergence as a loss function to guide the update of the entire local-global dual-branch network.

Images