Intelligent structured hyperspectral image clustering method and device

Abstract

The application discloses an intelligent structured hyperspectral image clustering method and device, and the method comprises the following steps: constructing a double local graph structure and a double graph encoder to mine high-order structure correlation between pixels, and extracting robust local spectral-spatial features; integrating local feature learning and global clustering structure mining into a unified learning framework, embedding a shared self-expression layer after the double graph encoder to capture consistent global clustering structure of the pixels; adopting a double graph decoder to reconstruct the input graph structure matrix, constructing a feature optimization mechanism guided by the global structure to optimize the local feature distribution by using the global clustering structure; optimizing the learning of the features and the global clustering structure in an end-to-end manner, combining reconstruction loss, self-expression loss and distribution loss to obtain an overall objective function for training a network model, and then obtaining a hyperspectral image clustering result, and classifying and identifying different ground objects based on the clustering result. The device comprises a processor and a memory.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and hyperspectral image processing, and in particular to an intelligent structured hyperspectral image clustering method and apparatus. Background Technology

[0002] Hyperspectral images are typically obtained by sensors mounted on an imaging platform performing continuous spectral imaging of surface objects within a certain frequency range, providing rich spectral and spatial information about these objects. With the rapid development of sensor technology, hyperspectral images have become an important data source for various applications such as environmental monitoring, geological exploration, military reconnaissance, and agricultural production. As a fundamental task in hyperspectral image analysis, the classification of hyperspectral image pixels has received widespread attention. Based on the principle that different surface objects have different spectral reflectance and absorption characteristics, hyperspectral image classification can effectively identify different surface objects. However, hyperspectral image classification methods are usually supervised, requiring a large amount of labeled data to supervise the network in learning a good classifier. In real-world applications, obtaining hyperspectral image data with pixel-by-pixel annotations is time-consuming and labor-intensive. Therefore, unsupervised clustering of hyperspectral images to classify pixels has significant practical value. Specifically, hyperspectral image clustering aims to group surface object pixels belonging to the same category in a hyperspectral image into the same group without relying on category labels. Although hyperspectral image clustering has been extensively studied in recent years, it remains a challenging task due to the high data dimensionality, large spectral variability, and complex spatial distribution of hyperspectral images.

[0003] Early work on hyperspectral image clustering typically focused on using spectral information to learn pixel groupings. Due to the complex spectral variability of hyperspectral images, different land cover categories may have similar spectra. Therefore, relying solely on spectral information is insufficient to provide discriminative representations for clustering. In hyperspectral images, pixels belonging to the same category are often spatially regionally distributed, meaning that the central pixel and its neighbors are highly likely to belong to the same category. Building on this, researchers have proposed numerous spectral-spatial methods to jointly utilize spectral and spatial contextual information in hyperspectral images. For example, Zhai et al. explored spatial neighborhood correlations by constraining the central pixel to share similar representations with its vertical and horizontal neighbors. However, existing spectral-spatial methods generally focus on utilizing the correlation between spatially adjacent pixels, rarely considering the spectral-spatial correlation between non-adjacent pixels. In hyperspectral images, pixels of the same category are often distributed across multiple dispersed spatial regions. Therefore, capturing spectral-spatial information from highly correlated pixels distributed across different regions is crucial for improving the performance of hyperspectral image clustering.

[0004] In recent years, graph learning has attracted widespread attention in various data analysis applications. Due to its excellent ability to model structural information, graphs can effectively represent the correlations between data samples and have become an effective data representation learning method, with initial explorations in hyperspectral image clustering tasks. For example, Zeng et al. introduced a graph regularization term to make connected pixels in the graph share similar representations. Cai et al. enhanced the feature representation of the central pixel by aggregating information from pixels connected to the central pixel. However, these methods only focus on the first-order structural associations between connected pixels in the graph, ignoring the higher-order structural associations between unconnected pixels. Furthermore, they typically learn the features of the central pixel by introducing local graph structural information, neglecting the mutual promotion between local feature extraction and global clustering structure learning. Therefore, fully mining the structural information of hyperspectral images to achieve structured hyperspectral image clustering is of great significance. Summary of the Invention

[0005] This invention provides an intelligent structured hyperspectral image clustering method and apparatus. This invention fully explores the higher-order structural relationships between pixels in hyperspectral images and the mutually reinforcing relationship between local feature extraction and global clustering structure learning, thereby achieving more accurate hyperspectral image clustering. Ultimately, it can effectively classify and identify different surface objects without relying on category labels. See the description below for details:

[0006] A smart structured hyperspectral image clustering method, the method comprising:

[0007] A dual local graph structure is constructed to explore higher-order structural relationships between pixels;

[0008] The dual graph encoder uses a 3-layer graph convolutional network to progressively mine higher-order structural relationships between pixels and extract robust local spectral-spatial features.

[0009] Local feature learning and global clustering structure mining are integrated into a unified learning framework, with a shared self-expression layer embedded after the dual graph encoder to capture consistent global clustering structures of hyperspectral image pixels.

[0010] A dual graph decoder is used to reconstruct the input graph structure matrix, and a global structure-guided feature optimization mechanism is constructed to optimize the distribution of local spectral-spatial features by utilizing the global clustering structure captured by the self-expression layer.

[0011] The learning of spectral-spatial features and global clustering structure is jointly optimized in an end-to-end manner. The overall objective function is obtained by combining reconstruction loss, self-expression loss and distribution loss. The network model is trained using the overall loss function to obtain hyperspectral image clustering results. Based on the clustering results, different surface objects are classified and identified.

[0012] The graph convolutional network is as follows:

[0013]

[0014] in, This represents the features learned by the l-th layer of the graph convolutional network. This represents the learnable weight matrix. This represents a graph structure matrix with self-loops. Let σ represent the degree matrix of the diagonal, and σ represent the activation function.

[0015] Furthermore, the loss constraint of the self-expression layer is:

[0016]

[0017] Where C represents the self-expression coefficient matrix, ||.|| F This represents the Frobenius norm.

[0018] Among them, the student t-distribution is used to obtain its corresponding local feature distribution Q. (m) Local feature distribution Q (m) elements in The probability that pixel i is assigned to the u-th cluster is calculated from the distance between the pixel feature representation and the cluster center representation:

[0019]

[0020] in, This represents the feature of the i-th pixel. Let v represent the learnable cluster center representation of the u-th cluster, and v represent the degrees of freedom. Let u' represent the learnable cluster center representation of the u-th cluster, where u' represents the cluster label.

[0021] Furthermore, the self-expression layer progressively learns the global clustering structure among pixels, obtaining the global clustering distribution P by minimizing the following objective function:

[0022]

[0023] st P∈{0,1} n×c P1 = 1, rank(P) = c

[0024] Among them, w ij Let p represent the similarity between the i-th pixel and the j-th pixel in W. i Let p be a one-hot vector representing the i-th row of the global clustering distribution matrix P, indicating the cluster assignment of the i-th pixel. jIt is a one-hot vector that represents the j-th row of the global clustering distribution matrix P, where rank is the rank of the matrix, c is the number of clusters, and 1 represents a vector with all values ​​of 1.

[0025] A distribution constraint is constructed to utilize the global clustering distribution to constrain the local feature distribution. The constructed distribution constraint is defined as follows:

[0026]

[0027] Where KL represents the Kullback-Leibler divergence, Q (m) For local feature distribution, p iu Let be the element in the i-th row and u-th column of P. For Q (m) The element in the i-th row and u-th column.

[0028] A smart structured hyperspectral image clustering device, the device comprising: a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the device to perform the steps of the method described in any of the first aspects.

[0029] The beneficial effects of the technical solution provided by this invention are:

[0030] 1. This invention effectively mines the high-order structural correlations between pixels in hyperspectral images through a self-expressive embedded dual graph autoencoder. By designing a global structure-guided feature optimization mechanism, it achieves mutual promotion between local feature extraction and global clustering structure learning, thereby obtaining more compact spectral-spatial features and ultimately improving the clustering performance of hyperspectral images.

[0031] 2. The hyperspectral image clustering method proposed in this invention effectively explores the structural information of hyperspectral image pixels. Its experimental performance on public datasets is superior to existing hyperspectral image clustering methods. It can achieve accurate identification of imaged surface objects without category labels. Attached Figure Description

[0032] Figure 1 A flowchart of an intelligent structured hyperspectral image clustering method;

[0033] Figure 2 This is a comparison of the overall accuracy of hyperspectral image clustering. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below.

[0035] I. Constructing a dual graph structure

[0036] Given a hyperspectral image containing n pixels, this invention constructs a dual local graph structure to fully explore higher-order structural relationships between pixels. Specifically, pixel features from two pixel blocks of different scales are concatenated to obtain a dual pixel data matrix. The dual local graph structure of a pixel is represented by constructing two k-nearest neighbor graphs by connecting each pixel to its 30 most similar pixels. In this embodiment, the pixel block sizes are set to 9×9 and 11×11, respectively. (m) The corresponding graph structure matrix A (m) Defined as:

[0037]

[0038] in, A represents (m) The elements in. If If pixel nodes i and j are connected in the graph, then they are not connected in the graph. X represents (m) The data of the i-th pixel, X represents (m) The data of the j-th pixel, express k nearest neighbor, express k nearest neighbors.

[0039] II. Constructing a Dual Graph Encoder

[0040] Based on pixel data matrix And graph structure matrix In this embodiment of the invention, the dual graph encoder utilizes a 3-layer graph convolutional network to progressively mine higher-order structural relationships between pixels, thereby extracting robust local spectral-spatial features.

[0041] The operation of graph convolutional networks is defined as follows:

[0042]

[0043] in, This represents the features learned by the l-th layer of the graph convolutional network. This represents the learnable weight matrix. This represents a graph structure matrix with self-loops. This represents a diagonal degree matrix, whose diagonal elements express The element located in the i-th row and j-th column is σ, which represents the activation function. In this embodiment of the invention, the ReLU function is used as the activation function.

[0044] III. Constructing a shared self-expression layer

[0045] To integrate local feature learning and global cluster structure mining into a unified learning framework, a shared self-expressive layer is embedded after the dual graph encoder to capture a consistent global cluster structure of hyperspectral image pixels.

[0046] Specifically, the latent features extracted by the graph encoder are represented The data is fed into a self-expression layer and constrained by the following self-expression loss:

[0047]

[0048] Where C represents the self-expression coefficient matrix, ||.|| F This represents the Frobenius norm. Intuitively, the similarity matrix calculated from the self-expression coefficient matrix can effectively characterize the global clustering structure and will subsequently be used to guide the optimization of spectral-spatial features.

[0049] IV. Constructing a Dual Graph Decoder

[0050] To ensure that the learned spectral-spatial features retain the inherent graph structure information between the original pixels, this embodiment of the invention employs a dual graph decoder to reconstruct the input graph structure matrix. The decoder is implemented through the inner product of features, expressed by the formula:

[0051]

[0052] in, The reconstructed graph structure matrix is ​​represented by the following: This represents the output of the self-expression layer. After reconstructing the graph structure matrix, a reconstruction loss is used to constrain the consistency between the input and the reconstructed graph structure matrix:

[0053]

[0054] Here, BCE represents binary cross-entropy loss.

[0055] V. Design a feature optimization mechanism guided by global structure

[0056] To learn more separable features, a global structure-guided feature optimization mechanism was designed to optimize the distribution of local spectral-spatial features by leveraging the global clustering structure captured by the self-expression layer. This mechanism is applied to the local spectral-spatial features extracted by the graph encoder. The corresponding local feature distribution Q is obtained using the Student t-distribution. (m) Among them, the local feature distribution Q (m) elements in The probability that pixel i is assigned to the u-th cluster is calculated from the distance between the pixel feature representation and the cluster center representation, as shown in the formula:

[0057]

[0058] in, This represents the feature of the i-th pixel. Let v represent the learnable cluster center representation of the u-th cluster. v represents the degrees of freedom. Let v represent the learnable cluster center representation of the u′-th cluster, where u′ represents the cluster label. In this embodiment, v is set to 1.

[0059] Furthermore, during the network learning process, the self-expression layer can gradually learn the global clustering structure between pixels, thus enabling explicit modeling of the global clustering distribution. Specifically, the global similarity matrix is ​​calculated as W = |C| + |C| T The global clustering distribution P is obtained by minimizing the following objective function:

[0060]

[0061] stP∈{0,1} n×c P1 = 1, rank(P) = c (7)

[0062] Among them, w ij p represents the similarity between the i-th pixel and the j-th pixel in W. i Let p be a one-hot vector representing the i-th row of the global clustering distribution matrix P, indicating the cluster assignment of the i-th pixel. j Let be a one-hot vector representing the j-th row of the global clustering distribution matrix P, where rank is the rank of the matrix, c is the number of clusters, and 1 represents a vector with all values ​​of 1.

[0063] After obtaining the global clustering distribution P, distribution constraints are constructed to constrain local feature distributions using the global clustering distribution. The constructed distribution constraints are defined as follows:

[0064]

[0065] Where KL represents the Kullback-Leibler divergence, Q (m) For local feature distribution, p iu Let be the element in the i-th row and u-th column of P. For Q (m) The element in the i-th row and u-th column.

[0066] By optimizing the local spectral-spatial feature distribution through global clustering distribution, the distance between each hyperspectral pixel and its corresponding cluster center is shortened, resulting in more compact spectral-spatial features and improved clustering performance.

[0067] VI. Constructing the overall objective function

[0068] Finally, to jointly optimize the learning of spectral-spatial features and global clustering structure in an end-to-end manner, the overall objective function is obtained by combining reconstruction loss, self-expression loss, and distribution loss, which is expressed as:

[0069] L = L re +αL se +βL dc (9)

[0070] Here, α and β represent the weighting coefficients for balancing different losses. In this embodiment, α and β are set to 1 and 0.1, respectively.

[0071] VII. Hyperspectral Image Clustering

[0072] The network model is trained using the overall loss function to obtain the final self-expression coefficient matrix C. Based on the final self-expression coefficient matrix C, the corresponding similarity matrix W is calculated, and a spectral clustering algorithm is applied to the similarity matrix W to obtain hyperspectral image clustering results. Based on the hyperspectral image clustering results, different surface objects can be accurately classified and identified without relying on category labels.

[0073] Figure 2 The overall accuracy comparison results of hyperspectral image clustering are listed, with the comparison algorithms including the EKGCSC method and the GR-RSCNet method. EKGCSC is a traditional hyperspectral image clustering method, while GR-RSCNet is a deep hyperspectral image clustering method. Higher overall accuracy indicates better hyperspectral image clustering performance. As shown in the figure, the overall clustering accuracy of both the EKGCSC method and GR-RSCNet is lower than that of the method described in this invention, because they fail to fully exploit the structural information of hyperspectral image pixels. Figure 2 As can be seen, by fully exploring the higher-order structural associations between pixels in hyperspectral images and the mutually reinforcing relationship between local feature extraction and global clustering structure learning, the method of this invention can obtain better hyperspectral image clustering results and better achieve the division and recognition of surface objects.

[0074] A smart structured hyperspectral image clustering device includes a processor and a memory. The memory stores program instructions, and the processor invokes the program instructions stored in the memory to cause the device to execute any one of the method steps:

[0075] A dual local graph structure is constructed to explore higher-order structural relationships between pixels;

[0076] The dual graph encoder uses a 3-layer graph convolutional network to progressively mine higher-order structural relationships between pixels and extract robust local spectral-spatial features.

[0077] Local feature learning and global clustering structure mining are integrated into a unified learning framework, with a shared self-expression layer embedded after the dual graph encoder to capture consistent global clustering structures of hyperspectral image pixels.

[0078] A dual graph decoder is used to reconstruct the input graph structure matrix, and a global structure-guided feature optimization mechanism is constructed to optimize the distribution of local spectral-spatial features by utilizing the global clustering structure captured by the self-expression layer.

[0079] The learning of spectral-spatial features and global clustering structure is jointly optimized in an end-to-end manner. The overall objective function is obtained by combining reconstruction loss, self-expression loss and distribution loss. The network model is trained using the overall loss function to obtain hyperspectral image clustering results. Based on the clustering results, different surface objects are classified and identified.

[0080] The graph convolutional network is as follows:

[0081]

[0082] in, This represents the features learned by the l-th layer of the graph convolutional network. This represents the learnable weight matrix. This represents a graph structure matrix with self-loops. Let σ represent the degree matrix of the diagonal, and σ represent the activation function.

[0083] Furthermore, the loss constraint for the self-expression layer is:

[0084]

[0085] Where C represents the self-expression coefficient matrix, ||.|| F This represents the Frobenius norm.

[0086] Among them, the student t-distribution is used to obtain its corresponding local feature distribution Q. (m) Local feature distribution Q (m) elements in The probability that pixel i is assigned to the u-th cluster is calculated from the distance between the pixel feature representation and the cluster center representation:

[0087]

[0088] in, This represents the feature of the i-th pixel. Let v represent the learnable cluster center representation of the u-th cluster, and v represent the degrees of freedom. Let u' represent the learnable cluster center representation of the u-th cluster, where u' represents the cluster label.

[0089] Furthermore, the self-expression layer progressively learns the global clustering structure among pixels, obtaining the global clustering distribution P by minimizing the following objective function:

[0090]

[0091] stP∈{0,1} n×c P1 = 1, rank(P) = c

[0092] Among them, w ij Let p represent the similarity between the i-th pixel and the j-th pixel in W. i Let p be a one-hot vector representing the i-th row of the global clustering distribution matrix P, indicating the cluster assignment of the i-th pixel. j It is a one-hot vector that represents the j-th row of the global clustering distribution matrix P, where rank is the rank of the matrix, c is the number of clusters, and 1 represents a vector with all values ​​of 1.

[0093] A distribution constraint is constructed to utilize the global clustering distribution to constrain the local feature distribution. The constructed distribution constraint is defined as follows:

[0094]

[0095] Where KL represents the Kullback-Leibler divergence, Q (m) For local feature distribution, p iu Let be the element in the i-th row and u-th column of P. For Q (m) The element in the i-th row and u-th column.

[0096] Unless otherwise specified, the model numbers of the various devices in this embodiment of the invention are not limited, and any device that can perform the above functions is acceptable.

[0097] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A smart structured hyperspectral image clustering method, characterized in that, The method includes: A dual local graph structure is constructed to explore higher-order structural relationships between pixels; The dual graph encoder uses a 3-layer graph convolutional network to progressively mine higher-order structural relationships between pixels and extract robust local spectral-spatial features. Local feature learning and global clustering structure mining are integrated into a unified learning framework, with a shared self-expression layer embedded after the dual graph encoder to capture consistent global clustering structures of hyperspectral image pixels. A dual graph decoder is used to reconstruct the input graph structure matrix, and a global structure-guided feature optimization mechanism is constructed to optimize the distribution of local spectral-spatial features by utilizing the global clustering structure captured by the self-expression layer. The learning of spectral-spatial features and global clustering structure is jointly optimized in an end-to-end manner. The overall objective function is obtained by combining reconstruction loss, self-expression loss and distribution loss. The network model is trained using the overall loss function to obtain hyperspectral image clustering results. Based on the clustering results, different surface objects are classified and identified. The aforementioned feature optimization mechanism guided by a global structure, which utilizes the global clustering structure captured by the self-expression layer to optimize the distribution of local spectral-spatial features, is as follows: Using students The distribution obtains its corresponding local feature distribution. Local feature distribution elements in Represents pixels Assigned to the The probability of each cluster is calculated from the distance between the pixel feature representation and the cluster center representation: ； in, Indicates the first Features of each pixel Indicates the first Learnable cluster center representations for each cluster. Indicates degrees of freedom. Indicates the first Learnable cluster center representations for each cluster. The label indicating the cluster; The self-expression layer progressively learns the global clustering structure among pixels, and obtains the global clustering distribution by minimizing the following objective function. : ； in, express The Middle The pixel and the Similarity per pixel It is a one-hot vector representing the global clustering distribution matrix. The first in Line, indicating the first Cluster allocation relationship of pixels, It is a one-hot vector representing the global clustering distribution matrix. The first in The row represents the rank of the matrix, and c represents the number of clusters. Represents a vector whose values ​​are all 1; A distribution constraint is constructed to utilize the global clustering distribution to constrain the local feature distribution. The constructed distribution constraint is defined as follows: ； in, Indicates the Kullback-Leibler divergence. For local feature distribution, for The Middle row and number Column elements, for The Middle row and number The elements of the column.

2. The intelligent structured hyperspectral image clustering method according to claim 1, characterized in that, The graph convolutional network is: ； in, This represents the graph convolutional network. Features learned by layers This represents the learnable weight matrix. This represents a graph structure matrix with self-loops. Represents the degree matrix of the diagonal. This represents the activation function.

3. The intelligent structured hyperspectral image clustering method according to claim 2, characterized in that, The loss constraint for the self-expression layer is: ； in, Represents the self-expression coefficient matrix. express Frobenius Norm.

4. An intelligent structured hyperspectral image clustering device, characterized in that, The device includes a processor and a memory, the memory storing program instructions, the processor invoking the program instructions stored in the memory to cause the device to perform the method according to any one of claims 1-3.

Images