A remote sensing image clustering method and system

Abstract

The present application discloses a remote sensing image clustering method and system, which relates to the field of image processing. The method includes: acquiring a remote sensing image; performing superpixel segmentation on the remote sensing image to obtain a superpixel map; obtaining superpixel features and an adjacency matrix based on the superpixel map; constructing an image clustering model, and inputting the superpixel features and the adjacency matrix into the image clustering model to obtain clustering results; and completing the classification of objects in the remote sensing image based on the clustering results. Among them, the image clustering model includes a cross-attention map diffusion module, a discriminative spectral feature extraction module, a map self-contrast loss function, and a clustering output module. The present application can well realize efficient neighbor perception, adaptive extraction of advanced spatial features, and effective identification of discriminative information, thereby realizing accurate clustering of remote sensing images.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a remote sensing image clustering method and system. Background Art

[0002] Remote sensing image classification technology has received widespread attention in recent years. It aims to classify the category of each pixel. In recent years, many supervised and semi-supervised methods have achieved good classification results. However, these methods require manual annotation to obtain labels, which is often time-consuming, labor-intensive and cumbersome. In order to solve the above problems, unsupervised clustering methods have received more and more attention. Remote sensing image clustering does not rely on any training samples, but only realizes the natural division of pixels by mining the potential intrinsic structure inside the data. However, due to insufficient prior knowledge and spectral variability, unsupervised clustering becomes a challenging problem.

[0003] Traditional remote sensing image clustering methods often rely on handcrafted spectral features and shallow classifiers based on raw features. These features are often not representative and difficult to mine deep semantic information and adapt to the characteristics of spectral variation of remote sensing images. In recent years, with the development of deep learning, some deep clustering-based methods have been applied to remote sensing image clustering. They can update model parameters and learn discriminative spatial-spectral features for clustering through self-supervised training. Most deep clustering methods can be summarized into two steps: the design of feature extractors based on deep clustering methods and the use of traditional methods to cluster features. For example, in the clustering of hyperspectral remote sensing images, a new superpixel pooling autoencoder is used to extract the potential representation of features, and a neighborhood contrast learning deep subspace clustering network is proposed. Finally, feature clustering is achieved through spectral clustering; spatial similarity matrix and spectral similarity matrix are used to characterize the geometric structure, and a dual-graph autoencoder architecture is developed. Finally, clustering results are obtained through k-means; the graph regularization subspace clustering model can also be integrated into the autoencoder to extract high-level spectral features, and spectral clustering is used to obtain clustering results. Deep learning networks are usually unable to obtain a good deep expression of spatial structural information, and have not explored how to achieve efficient embedding of spatial and spectral information. Affected by spectral variability, clustering results are often not accurate enough. Insufficient use of structural information and interference from spectral noise also limit their clustering performance.

[0004] In the past few decades, a large number of methods have emerged for remote sensing image clustering. In the early stage, some traditional algorithms for natural image clustering were migrated to remote sensing image clustering, such as k-means (K-means clustering), spectral clustering (SC), fuzzy c-means (FCM) and independent component analysis (ICA) models. Despite some success, these methods usually only focus on spectral features and ignore the spatial structural relationship of objects, which limits their large-scale application. Later, some methods based on spatial-spectral joint classification were proposed to try to achieve higher clustering accuracy. For example, in order to flexibly use local spatial information and spectral information, the boundaries of different categories are better identified by using fuzzy similarity metrics based on pixel space attraction models, or, on the basis of sparse subspace clustering (SSC), the L2 norm regularization term is added to make more refined use of spatial information.

[0005] Unfortunately, the above networks are usually unable to obtain the deep expression of spatial structure information well, and have not explored in depth how to achieve efficient embedding of spatial and spectral information. In addition, the insufficient use of spatial-spectral information also leads to poor clustering performance. Recently, the development of graph clustering methods has made it possible to effectively encode spatial structure into spectral features, and extended CNN, which can only process Euclidean data, to non-Euclidean graph structure information processing. Taking hyperspectral remote sensing images as an example, the graph clustering process usually includes the following steps: first, obtain remote sensing image data and perform preprocessing, such as denoising and normalization; then, use feature extraction methods, such as principal component analysis or independent component analysis, to reduce the dimension of high-dimensional spectral data to reduce computational complexity; then, select spectral clustering or superpixel segmentation methods to construct the reduced-dimensional data into a graph structure; finally, cluster the data through graph clustering algorithms such as graph autoencoders or graph contrastive learning to identify different spectral feature regions, thereby realizing clustering of remote sensing images.

[0006] Many works based on graph coding have confirmed that it is effective to introduce structural information through graph embedding. For example, a low-pass graph convolution method is used to embed spatial spectral information to obtain smoother structural characteristics and reduce the amount of model training. In order to explore the goal-oriented graph clustering method, an attention network is used to aggregate the information of adjacent nodes, and a self-training module is developed to guide the clustering algorithm to achieve better performance. In order to extend GNN to general clustering tasks, a graph autoencoder network for general data can also be proposed. The model adaptively updates the graph according to the generated embedding content, so that deep information can be used and bad graphs caused by original features can be modified. GCN can also be used to learn the structural information of graph data and combine it with optimal transmission theory to achieve effective spectral clustering (OT), which performs well when processing high-dimensional data.

[0007] Graph clustering methods usually use a simple first-order adjacency matrix to integrate into the feature extraction network to characterize the relationship between different nodes. This cannot achieve high-order neighbor-awareness and is difficult to adaptively extract high-level spatial features. In addition, there is no effective response to spectral variation and noise interference, and discriminative information cannot be effectively identified, resulting in weak model robustness. Summary of the invention

[0008] The purpose of this application is to provide a remote sensing image clustering method and system, which can well realize efficient neighbor perception, adaptive extraction of high-level spatial features and effective identification of discriminative information, thereby realizing accurate clustering of remote sensing images.

[0009] To achieve the above objectives, this application provides the following solutions:

[0010] In a first aspect, the present application provides a remote sensing image clustering method, comprising:

[0011] Acquisition of remote sensing images;

[0012] Performing super-pixel segmentation on the remote sensing image to obtain a super-pixel map;

[0013] Obtaining superpixel features and an adjacency matrix based on the superpixel graph;

[0014] An image clustering model is constructed, and the superpixel features and the adjacency matrix are input into the image clustering model to obtain a clustering result; the image clustering model includes a cross-attention graph diffusion module, a discriminative spectral feature extraction module, a graph self-contrast loss function and a clustering output module; the cross-attention graph diffusion module is used to extract structural information in the remote sensing image based on the adjacency matrix; the discriminative spectral feature extraction module is used to extract spectral information in the remote sensing image based on the superpixel features, and is used to encode the structural information and the spectral information to obtain a sample similarity matrix; the graph self-contrast loss function is used to train the image clustering model; the clustering output module is used to obtain the clustering result based on the sample similarity matrix;

[0015] The classification of objects in the remote sensing image is completed based on the clustering result.

[0016] In a second aspect, the present application provides a remote sensing image clustering system, comprising:

[0017] An image acquisition unit, used for acquiring remote sensing images;

[0018] A pixel segmentation unit, used to perform super-pixel segmentation on the remote sensing image to obtain a super-pixel map;

[0019] A model building unit, used for building an image clustering model, and inputting the superpixel map into the image clustering model to obtain a clustering result; the image clustering model includes a cross-attention map diffusion module, a discriminative spectral feature extraction module and an image self-contrast loss function; the cross-attention map diffusion module is used to extract structural information in the superpixel map; the discriminative spectral feature extraction module is used to extract spectral information in the superpixel map, and is used to encode the structural information and the spectral information to obtain a sample similarity matrix; the image self-contrast loss function is used to train the image clustering model, and is used to obtain the clustering result based on the sample similarity matrix;

[0020] A ground object classification unit is used to complete the ground object classification in the remote sensing image based on the clustering result.

[0021] According to the specific embodiments provided in this application, this application has the following technical effects:

[0022] The present application provides a remote sensing image clustering method and system, which can realize efficient construction of graph structure and perception of high-order neighborhood information by setting a cross-attention graph diffusion module. By setting a discriminative spectral feature extraction module, efficient fusion of spatial information and spectral information is realized through structure-spectral coding, so that the two modules complement each other, realize joint extraction of spatial and spectral features, adaptively extract advanced spatial features, and effectively identify discriminative information, thereby realizing accurate clustering of remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 This is an application environment diagram of a remote sensing image clustering method in one embodiment of the present application;

[0025] Figure 2 A schematic diagram of a process of a remote sensing image clustering method provided in one embodiment of the present application;

[0026] Figure 3 This is a diagram of an implementation architecture of an image clustering model provided in one embodiment of the present application;

[0027] Figure 4 A cluster array diagram of the Salinas (SA) data set provided in another embodiment of the present application;

[0028] Figure 5A clustering array diagram of the Trento data set provided in one embodiment of the present application;

[0029] Figure 6 A clustering array diagram of the Pavia University (UP) data set provided in one embodiment of the present application;

[0030] Figure 7 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] In order to make the purpose, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0033] The remote sensing image clustering method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the remote sensing image to the server 104. After the server 104 receives the remote sensing image, for the remote sensing image, the server 104 performs superpixel segmentation on the remote sensing image to obtain a superpixel map. Superpixel features and adjacency matrix are obtained based on the superpixel map. An image clustering model is constructed, and the superpixel features and adjacency matrix are input into the image clustering model to obtain clustering results. The classification of objects in the remote sensing image is completed based on the clustering results. Among them, the constructed image clustering model includes a cross-attention graph diffusion module, a discriminative spectral feature extraction module, a graph self-contrast loss function, and a clustering output module. The cross-attention graph diffusion module is used to extract structural information in the remote sensing image based on the adjacency matrix. The discriminative spectral feature extraction module is used to extract spectral information in the remote sensing image based on superpixel features, and is used to encode structural information and spectral information to obtain a sample similarity matrix. The contrast loss function is used to train the image clustering model. The clustering output module is used to obtain clustering results based on the sample similarity matrix. The server 104 can feed back the obtained ground object classification results to the terminal 102. In addition, in some embodiments, the remote sensing image clustering method can also be implemented separately by the server 104 or the terminal 102, such as the terminal 102 can directly perform remote sensing image clustering on the remote sensing image to obtain the ground object classification result, or the server 104 can obtain the remote sensing image from the data storage system and perform remote sensing image clustering on the remote sensing image.

[0034] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0035] In an exemplary embodiment, Figure 2 As shown, a remote sensing image clustering method is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps 200 to 204. Among them:

[0036] Step 200: Acquire a remote sensing image, wherein the remote sensing image used here may be a hyperspectral image.

[0037] Step 201: Perform superpixel segmentation on the remote sensing image to obtain a superpixel map.

[0038] Step 202: Obtain superpixel features and an adjacency matrix based on the superpixel graph.

[0039] Step 203: construct an image clustering model, and input the superpixel features and the adjacency matrix into the image clustering model to obtain clustering results. The image clustering model includes a cross-attention graph diffusion module, a discriminative spectral feature extraction module, a graph self-contrast loss function, and a clustering output module. The cross-attention graph diffusion module is used to extract structural information in remote sensing images based on the adjacency matrix. The discriminative spectral feature extraction module is used to extract spectral information in remote sensing images based on superpixel features, and is used to encode structural information and spectral information to obtain a sample similarity matrix. The graph self-contrast loss function is used to train the image clustering model. The clustering output module is used to obtain clustering results based on the sample similarity matrix.

[0040] Step 204: completing the classification of objects in the remote sensing image based on the clustering results.

[0041] The implementation of the above steps 200 to 204 can well achieve efficient neighbor perception, adaptive extraction of high-level spatial features, and effective identification of discriminative information, thereby achieving accurate clustering of remote sensing images.

[0042] In another exemplary embodiment of the present application, Figure 3 As shown, the image clustering model provided by the present application is a two-stream network, which consists of a cross-attention graph diffusion module and a discriminative spectral feature extraction module. The cross-attention graph diffusion module extracts high-level structural information through multi-view cross-attention mechanism and graph diffusion. The discriminative spectral feature extraction module alleviates clutter through low-pass graph Laplacian and uses a discriminative feature scaling module for feature scaling. The graph structure encoding module implements structure-spectral feature encoding. Finally, the network is trained through the graph self-contrast loss function and the classification result is obtained.

[0043] Based on the above description of this embodiment, in actual application, if Figure 3 As shown in Figure 2, the construction process of the image clustering model can be described as:

[0044] Step S1: Given a remote sensing image h, set the number of training rounds (Epochs) and the number of clusters c, and the loss function weight factor and , Adaptive Moment Estimation (Adam) optimizer is used to update parameters and set the learning rate.

[0045] Step S2: Perform superpixel segmentation on the remote sensing image h through the Simple Linear Iterative Clustering (SLIC) algorithm, and take the average feature of each superpixel as the entire superpixel feature to establish a superpixel map and obtain the superpixel feature. And the adjacency matrix .

[0046] Step S3: Construct a symmetric normalized adjacency matrix and obtain a self-attention map through the neighborhood self-attention layer According to the adjacency matrix Constructing K-nearest neighbor (KNN) graph , K nearest neighbor graph With self-attention map Get the cross attention map through the multi-view cross attention layer ,Furthermore, through the cross attention map Get the structural similarity matrix . The symmetric normalized adjacency matrix is ​​expressed as :

[0047] In the formula, is the identity matrix, is the adjacency matrix The degree matrix of for The field of real numbers.

[0048] Step S4: Structural similarity matrix Get cross-attention features through Multilayer Perceptron (MLP) , .

[0049] Step S5: Cross-Attention Features The graph diffusion module is used to realize nonlinear diffusion to aggregate graph information and obtain a stable state of the graph. . The cross-attention feature The stationary state of the graph The output of the graph diffusion module, i.e., structural information, is obtained by information fusion through the normalization factor, which is denoted as , .

[0050] Step S6: Superpixel features The spectral smoothing feature is obtained by K-layer graph Laplacian filter , .

[0051] Step S7: smoothing the spectrum Divided into N characteristic spectrum segments , , calculate the feature difference matrix , , the feature difference matrix and characteristic spectrum After multiplication, concatenation is performed to obtain new discriminative features .

[0052] Step S8: Diffusion module output and new discriminative features Input to the graph structure encoding module to get the output features , and according to the output characteristics Calculate sample similarity matrix .

[0053] Step S9: Perform back propagation based on the loss of the loss function calculated by the graph self-contrast loss function.

[0054] Step S10: After L rounds of training, a trained network model is obtained, and the test sample is predicted according to the trained network to obtain the predicted value of the test sample. The trained network model corresponding to the predicted value that meets the set conditions is used as the image clustering model.

[0055] Furthermore, in order to improve the perception ability of the graph structure, this embodiment adopts the K-nearest neighbor (KNN) graph Aggregate high-order feature neighborhood information, different from n-hop adjacency matrix, K nearest neighbor graph It not only represents the connection relationship between nodes, but also encodes the similarity of features.

[0056] Based on the above description, the process of extracting structural information in remote sensing images based on the adjacency matrix by the cross attention graph diffusion module provided in this application may include:

[0057] (1) Construct a symmetric normalized adjacency matrix based on the adjacency matrix.

[0058] (2) The neighborhood self-attention layer is used to process the constructed symmetric normalized adjacency matrix using the self-attention mechanism to obtain a self-attention graph.

[0059] (3) Construct a K-nearest neighbor graph based on the adjacency matrix.

[0060] (4) A multi-view cross-attention layer is used to perform a cross-attention mechanism on the K nearest neighbor graph and the self-attention map to obtain a cross-attention map, and a structural similarity matrix is ​​obtained based on the cross-attention map.

[0061] (5) A multi-layer perceptron is used to obtain cross-attention features based on the structural similarity matrix.

[0062] (6) The graph diffusion module is used to perform graph information aggregation operations on the cross-attention features to obtain the stable state of the graph.

[0063] (7) The cross-attention features are fused with the stationary state of the graph through a normalization factor to obtain structural information.

[0064] The discriminative spectral feature extraction module extracts spectral information in the remote sensing image based on superpixel features, and the process of encoding the structural information and the spectral information to obtain the sample similarity matrix may include:

[0065] (1) A multi-layer graph Laplacian filter is used to filter the super-pixel features to obtain spectral smoothing features. The spectral smoothing features are the spectral information.

[0066] (2) Divide the spectral smoothing features into multiple characteristic spectral segments and determine the characteristic difference matrix.

[0067] (3) Concatenate the characteristic spectral segments and the characteristic difference matrix to obtain the discriminative features.

[0068] (4) The graph structure encoding module is used to perform graph structure encoding on the structural information and discriminative features to obtain output features.

[0069] (5) Determine the sample similarity matrix based on the output features.

[0070] In another exemplary embodiment of the present application, predefined graphs are often used in the graph diffusion process, which limits their performance. Therefore, a wider structural receptive field can be obtained more flexibly through multi-view information, and a graph self-attention mechanism can be used to embed feature and structural information into the latent space to learn a view. Figure 1 Based on this, the cross-attention graph diffusion module set in this application can include a neighborhood self-attention layer and a multi-view cross-attention layer, which has:

[0071] .

[0072] .

[0073] In the formula, , and is a learnable parameter, is the normalized adjacency matrix, . is the normalized adjacency matrix The transpose of .

[0074] The self-attention layer can better aggregate high-level information, and the cross-attention layer realizes the fusion of multiple views and obtains a consistent structure graph. Get the structural similarity matrix It is expressed as:

[0075] .

[0076] in, is the cross attention map The transpose of Represents the structural similarity matrix The Row, the structural similarity matrix After the two norm Perform normalization.

[0077] Furthermore, a two-layer MLP is designed to analyze the structural similarity matrix Processing to obtain cross-attention features , ,have:

[0078] .

[0079] In the formula, represents the trainable parameters of two linear processing layers, Represents the cross-attention feature Dimension.

[0080] Through the graph attention layer and the MLP layer, the high-order information of the graph can be encoded and a consistent view can be obtained, which improves the representation ability of the graph and is conducive to improving the performance of graph diffusion tasks.

[0081] In another exemplary embodiment of the present application, the graph convolutional neural network (GCN) has the phenomenon of over-smoothing in the feature extraction process, which limits the graph structure representation ability of GCN. In addition, GCN can only capture limited spatial dependencies through fixed finite layers. Therefore, a graph diffusion network based on discrete Beltrami flow can be developed to perform graph feature learning and topological structure evolution, while avoiding feature over-smoothing by adaptively aggregating graph information through a nonlinear graph diffusion network. Based on this, in the graph diffusion module adopted in this application, the stable state of the graph in the graph space , , which usually contains structural and feature information. For the graph node at time t For example, the time-dependent graph diffusion can be expressed as:

[0082] .

[0083] .

[0084] Among them, the function is the diffusion rate that controls the diffusion intensity between nodes and their neighbors, is the i-th graph node at time t, is the jth graph node at time t, is the i-th graph node at time t=0, is the original graph node without diffusion, Represents a collection of graph nodes.

[0085] For the stable diffusion state of the graph, it is necessary to derive the time-dependent stable diffusion state of the graph to achieve nonlinear diffusion and aggregate graph information. Based on the above description, the stable state of the graph at time t is :

[0086] .

[0087] in, is the time step, is the initial graph stationary state, is the time step The graph is in stable state.

[0088] To obtain the integral term, we must calculate , some work uses neural networks to represent ordinary differential equations Inspired by this, this embodiment can use Neural ordinary differential equation (ODE) to calculate , the advantage of Neural ODE is that ordinary differential equations can be embedded into neural networks as a module. Based on this, It can be expressed as:

[0089] .

[0090] in, and It is a neural network The learnable parameters in .

[0091] Further, based on the above description, the cross attention feature The stationary state of the graph The output of the graph diffusion module is obtained by information fusion through the normalization factor. It is expressed as:

[0092] .

[0093] in, represents the activation function, is the weighting factor, , is to utilize the degree information of each graph node. R is the cross attention feature.

[0094] In another exemplary embodiment of the present application, due to the influence of spectral anomalies, remote sensing images are prone to noise, which brings uncertainty factors to accurate clustering. Due to the influence of various limiting factors such as sensor sensitivity and environment, remote sensing images acquired by imaging sensors will inevitably be damaged by various noises, such as Gaussian noise, impulse noise, etc. These noises not only greatly reduce the visual effect of remote sensing images, but also have adverse effects in subsequent applications such as classification, mineral exploration and target detection. Based on this, the present application can introduce a low-pass graph Laplace filter to achieve low-pass denoising. Among them, the construction of the low-pass graph Laplace filter is as follows:

[0095] .

[0096] in, is the identity matrix, represents the normalized Laplacian matrix, is a low-pass graph Laplacian filter. In order to achieve better filtering effect, K layers of graph Laplacian filters can be stacked, expressed as:

[0097] .in, is a stack of K layers of graph Laplacian filters, The Laplacian filter is represented by layer.

[0098] The low-pass denoising module filters out high-frequency noise in the spectrum, while improving clustering performance and model operation efficiency.

[0099] In another exemplary embodiment of the present application, the key to high-precision spectral clustering is to convert spectral features into more discriminative features through a feature extraction module, and then use a distance function to measure proximity. Therefore, this embodiment extracts discriminative features and scales features with more "information value" to increase the similarity between features of the same object and expand the differences between features of different objects.

[0100] Based on the above description, discriminative feature scaling is mainly divided into three main steps: spectral segment division, feature difference matrix construction and feature scaling processing.

[0101] In order to more efficiently process and compare the differences between spectra, the spectral image can be Divided into N spectral segments, there are:

[0102]

[0103] in, Represents the nth dimensional spectrum, Representing superpixel features The total number of pixels contained.

[0104] In order to extract discriminative features, the spectral similarity between pixels in each spectral band is measured by the spectral angle metric (SAM) similarity measure, which is:

[0105] .

[0106] In the formula, represents the similarity between the pth and qth pixels in the spectral segment, The value range of is between 0 and Π. The smaller the value of is, the more similar the spectra are. Based on this, is the similarity matrix. is the spectral angle metric (SAM) similarity measure, is the pth pixel dimensional spectral features, is the qth pixel dimensional spectral features, is the inverse cosine function.

[0107] Furthermore, the feature difference factor is constructed through the similarity matrix:

[0108] .

[0109] In the formula, Represents the difference factor of the nth spectral segment. The greater the difference, the stronger the representation of the feature. Feature difference matrix In order to amplify these discriminative features, the difference factor is multiplied by the spectral segment to obtain more discriminative spectral features, namely, discriminative features, which are expressed as:

[0110] In the formula, is the feature difference matrix The transpose of .

[0111] In another exemplary embodiment of the present application, the graph diffusion module realizes the extraction of high-level structural semantic information, while the low-pass filtering and feature scaling module is responsible for spectral noise filtering and discriminative feature extraction. In order to achieve efficient fusion representation of structural information and spectral features, the designed graph structure encoding module is expressed as:

[0112]

[0113] in, , and Represent the input features, Layer features and Layer features, . is the symmetric normalized structural similarity matrix, , is the degree matrix, . and Represent the weight and bias of the lth layer respectively.

[0114] Subsequently, a task-oriented neighborhood clustering loss function is designed to ensure the consistency of the graph structure. Specifically, the output features of the graph encoding structure are first calculated The similarity matrix of is:

[0115] .in, Represents the cosine similarity between the i-th graph node and the j-th graph node, thus obtaining the sample similarity matrix . is the output feature of the i-th graph node, is the output feature of the jth graph node, is the output feature of the jth graph node The transpose of .

[0116] In another exemplary embodiment of the present application, in order to better perform self-supervised learning, the image self-contrast loss function used in the present application can be expressed as:

[0117] .

[0118] In the formula, represents the self-contrastive loss, represents the feature self-contrastive loss, represents the structural self-contrast loss, represents the self-constrained clustering loss, represents the weight factor of the structural self-contrast loss, Represents the weight factor for the self-constrained clustering loss.

[0119] Among them, the feature self-contrastive loss is defined as follows:

[0120] .

[0121] In the formula, represent situation, represent situation, is the symmetric normalized adjacency matrix between the i-th graph node and the j-th graph node. Represents the total number of nodes.

[0122] The neighbor nodes of the same node are regarded as positive samples, and the non-neighbor nodes are regarded as negative samples, so as to maintain the consistency of the graph structure. The advantage of this is that the distance between nodes of the same type can be reduced, and the distance between nodes of different types can be increased, thereby improving the clustering performance by extracting the discriminative features of the network by maintaining a consistent view.

[0123] Similarly, the structural self-contrastive loss is defined as follows:

[0124] .

[0125] In the formula, It is the structural similarity matrix of the graph diffusion module.

[0126] Feature self-contrastive loss and structural self-contrastive loss The learned node features and structural representations are only for better network reconstruction and have no intuitive connection with clustering tasks. Therefore, this application uses self-constrained loss to constrain features to make them more suitable for clustering tasks. Based on this, self-constrained clustering loss is defined as:

[0127] .

[0128] In the formula, represents the similarity that the i-th graph node belongs to category u, represents the target distribution, represents the KL divergence, Represented by The matrix composed of Represented by The t distribution is used to measure the similarity between the target node and the category center, which is:

[0129] .

[0130] in, represents the category center of category u, represents the category center of category k, Represents the features of the i-th graph node. It can be regarded as a soft distribution of nodes.

[0131] In order to achieve cluster-oriented node representation, similar nodes need to be closer and different nodes need to be farther away, so the target distribution Defined as:

[0132] .

[0133] In the formula, represents the similarity that the i-th graph node belongs to category c, and C represents the total number of categories.

[0134] The target distribution can be made sharper by quadratic calculation. and The KL divergence between the two distributions implements mutual constraints.

[0135] As another embodiment of the present application, experimental verification was carried out on the following public challenge datasets, using three real benchmark datasets: Salina (SA) dataset, Trento dataset and Pavia University (UP) dataset for verification. The scale of the Salina (SA) dataset is 512×217 and contains 204 bands, with a total of 16 types of ground objects. The image size of the Trento dataset is 600×166, containing 63 spectral channels, the spectral range is 0.40 ~ 0.98 μm, and includes six types of ground objects. The Pavia University (UP) dataset is an image composed of 103 spectral bands, and the size of the data is 610×340. This dataset contains a total of 9 types of ground objects.

[0136] 1. Evaluation criteria. Overall classification accuracy (OA), kappa coefficient (κ), normalized mutual information (NMI), adjusted rand index (ARI) and purity (Purity) are used as quantitative evaluation indicators. OA represents the overall classification accuracy. κ is an indicator used to evaluate the consistency of judgments, NMI is used to measure the similarity between two clustering results, ARI represents the degree of consistency between the clustering results and the actual situation, and Purity is a simple and transparent evaluation indicator. , the closer it is to 1, the better the clustering result.

[0137] 2. Baseline method.

[0138] In order to comprehensively analyze the effectiveness of the algorithm, the method (LFGD) provided in this application is compared with 8 selected baseline methods. The 8 baseline methods include K-means clustering (k-means), fuzzy c-means clustering (FCM), Possibilistic C-means clustering (PCM), Neighborhood Clustering (NCSC), Deep Fuzzy Clustering Network (DFCN), Structured Deep Clustering Network (SDCN), Enhanced Graph Autoencoder (EGAE), Adaptive Graph Autoencoder (AdaGAE) and Deep Autoencoder Graph Clustering (DAEGC). Among them, k-means, FCM, and PCM are traditional clustering methods, NCSC and DAEGC represent subspace clustering methods and graph clustering methods respectively, DFCN and SDCN represent deep clustering methods, and EGAE and AdaGAE represent graph autoencoder methods.

[0139] 3. Parameter settings and implementation details.

[0140] For fair comparison, all experiments were conducted on NVIDIA Titan RTX using the Pytorch framework. In the image clustering model constructed in this application, Epoch=40, the training optimizer uses the Adam optimizer, and the learning rate is 5×10 -4 During the training process, the hyperparameters are set as follows: The clustering results after quantitative comparison of the three data sets are shown in Tables 1 to 3. The qualitative comparison results of the three data sets are shown in Figures 4 to 6 As shown. Among them, Figure 4 Part (a) to Figure 4 Part (j) respectively represents the schematic diagrams of clustering results of K-means, FCM, PCM, NCSC, DFCN, SDCN, EGAE, AdaGAE, DAEGC and the image clustering model provided by this application for the SA dataset. Figure 5 Part (a) to Figure 5 Part (j) respectively represents the schematic diagrams of clustering results of K-means, FCM, PCM, NCSC, DFCN, SDCN, EGAE, AdaGAE, DAEGC and the image clustering model provided by this application for the Trento dataset. Figure 6 Part (a) to Figure 6 Part (j) respectively represents the schematic diagrams of clustering results of K-means, FCM, PCM, NCSC, DFCN, SDCN, EGAE, AdaGAE, DAEGC and the image clustering model provided by this application for the UP dataset.

[0141] Based on the above description, taking the clustering of hyperspectral images as an example, the experimental conclusions that can be obtained are:

[0142] 1) Quantitative analysis of SA dataset (targeted comparison of methods).

[0143] Table 1 shows the clustering accuracy of the SA dataset under different methods. From the clustering results, it can be seen that LFGD has achieved the best clustering effect. OA even almost reached 90%, which is very close to the result of supervised clustering, and the five indicators are the highest among all methods. Compared with the best results in the comparison methods, LFGD has improved OA, K, NMI, ARI and purity by 11.64%, 12.96%, 7.68%, 19.22% and 10.76% respectively. Furthermore, from a methodological point of view, LFGD does not simply treat HSI data as high-dimensional data, but extracts features from spatial object correlation and spectral correlation. The advantage of this is that it can give play to the characteristics of HSI and combine the characteristics of the image itself. In addition, compared with clustering using a subset of the dataset (SA-A), the clustering results using the entire HSI dataset have a large fluctuation, which shows that verification only on a small-scale dataset is not reliable.

[0144] Table 1 Quantitative clustering results on the SA dataset

[0145]

[0146] The graph diffusion model provided in this application is the first application in the HSI clustering task. The graph diffusion model can encode high-level structural features, thereby establishing good spatial relationships and helping to achieve better clustering results.

[0147] 2) Quantitative analysis of UP dataset.

[0148] Compared with the SA dataset, the objects in the UP dataset are more scattered, which brings great challenges to clustering.

[0149] In Table 2, LFGD achieves the best performance among the 10 methods. Especially, the clustering accuracy of Painted metal sheets and Bitumen reaches 100%.

[0150] Compared with the best result among the comparison methods, LFGD improves OA, K, NMI, ARI and purity by 10.15%, 14.33%, 13.41%, 3.49% and 7.48% respectively.

[0151] For specific methods, subspace clustering outperforms graph coding and deep clustering methods. However, their OA is difficult to exceed 70% on the UP dataset because they cannot fully establish the nonlinear relationship between the spatial and spectral features of HSI images.

[0152] It is worth noting that LFGD achieved a performance breakthrough of OA=70.25%.

[0153] Table 2 Quantitative clustering results on the Trento dataset

[0154]

[0155] The original intention of LFGD is to start from the characteristics of HSI data, deeply mine the structure and spectral relationship, and establish the relationship between them through the graph coding module. Experiments have proved that this method is simple and efficient and has good application prospects.

[0156] 3) Quantitative analysis of Trento dataset.

[0157] The Trento dataset contains only 6 types of objects, and the objects are relatively concentrated. However, the clustering effect of most models is still unsatisfactory.

[0158] In Table 3, the clustering effect of LFGD proposed in this application has achieved commendable results, with OA exceeding 90%. It is worth noting that on the two challenging ground features, Ground and Vineyard, the accuracy of LFGD is 90.91% and 99.45%, which is one of the reasons for its high clustering performance.

[0159] One of the advantages of LFGD is that it uses a task-oriented loss function, which makes the extracted features more suitable for clustering tasks. At the same time, the structural self-supervision and feature self-supervision losses realize the self-representation extraction of spatial spectral features.

[0160] In addition, data processing is also an important part of improving performance. The low-pass Laplacian filter and the discriminative feature scaling module filter out noise while improving the spectral recognition of different categories.

[0161] Table 3 Quantitative clustering results on the UP dataset

[0162]

[0163] 4. Visual clustering effect analysis.

[0164] In order to intuitively display the clustering results, the labeled objects in the data set are visually clustered. Figures 4 to 6 It can be seen that traditional methods such as k-means, FCM and PCM have salt and pepper noise, which means that they are easily affected by spectral variability and feature extraction is not sufficient. The deep learning method benefits from its nonlinear representation ability, and the colormap obtained is smoother and has less noise. However, for some details, such as edge objects, it is easy to misclassify. LFGD realizes the perceptual expression of the HSI topological structure through the graph diffusion model, so as to better perceive the spatial features, which makes its classification map smoother and the visualization effect more vivid. In addition, for spectral variability, the low-pass Laplace filter also plays a role in reducing classification noise. It can be seen that the classification noise is relatively small. LFGD processes texture details more finely, which makes it have excellent classification results.

[0165] Based on the above description, compared with the known technology, this application has the following characteristics:

[0166] 1. In the cross-attention graph diffusion module, unlike the traditional graph clustering method that simply obtains the adjacency relationship of nodes, this application encodes high-order structural information into the latent space through the graph cross-attention mechanism to obtain a better graph structure representation. The higher structural receptive field can also facilitate the graph diffusion model to learn graph knowledge faster. Through the graph attention layer and the MLP layer, the high-order information of the graph can be encoded and a consistent view can be obtained, which improves the representation ability of the graph and is conducive to improving the performance of the graph diffusion task.

[0167] 2. This application regards the graph diffusion module as a discrete representation of Beltrami flow, and the cross-attention feature reaches a steady state after a certain period of time in the graph diffusion process. Compared with graph convolutional neural networks, graph diffusion uses fewer parameters to capture the dynamic properties of graphs. The graph diffusion network performs graph feature learning and topological structure evolution, and at the same time, it aggregates graph information through an adaptive nonlinear graph diffusion network to avoid feature over-smoothing.

[0168] 3. In this application, the low-pass denoising module can effectively filter out high-frequency noise and aggregate neighbor information through stacked GraphLaplacian Filters, while improving clustering performance and model operation efficiency.

[0169] 4. In this application, the discriminative spectral segment scaling module enhances the network recognition ability by identifying and highlighting "valuable" spectral segments to obtain features. The high-level structural features and discriminative spectral features extracted by the cross-attention graph diffusion module and the discriminative spectral feature extraction module are respectively extracted by the graph structure encoding module to perform joint evolution extraction of spatial-spectral information, and a joint encoding feature can be obtained.

[0170] 5. The graph structure encoding module set up in this application realizes the efficient fusion of spatial information and spectral information through structure-spectral encoding, so that the two sub-networks complement and promote each other to achieve joint extraction of spatial and spectral features.

[0171] 6. This application also designs a task-oriented neighborhood clustering loss function, including feature self-contrast loss, structure self-contrast loss and self-constrained clustering loss, to ensure the consistency of the graph structure.

[0172] Based on the same inventive concept, the embodiment of the present application also provides a remote sensing image clustering system for implementing the remote sensing image clustering method involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more remote sensing image clustering system embodiments provided below can refer to the limitations on the remote sensing image clustering method above, and will not be repeated here.

[0173] In an exemplary embodiment, a remote sensing image clustering system is provided, comprising:

[0174] An image acquisition unit, used for acquiring remote sensing images;

[0175] A pixel segmentation unit, used to perform super-pixel segmentation on the remote sensing image to obtain a super-pixel map;

[0176] A model building unit, used for building an image clustering model, and inputting the superpixel map into the image clustering model to obtain a clustering result; the image clustering model includes a cross-attention map diffusion module, a discriminative spectral feature extraction module and an image self-contrast loss function; the cross-attention map diffusion module is used to extract structural information in the superpixel map; the discriminative spectral feature extraction module is used to extract spectral information in the superpixel map, and is used to encode the structural information and the spectral information to obtain a sample similarity matrix; the image self-contrast loss function is used to train the image clustering model, and is used to obtain the clustering result based on the sample similarity matrix;

[0177] A ground object classification unit is used to complete the ground object classification in the remote sensing image based on the clustering result.

[0178] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store remote sensing image clustering data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a remote sensing image clustering method is implemented.

[0179] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0180] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0181] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0182] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0183] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0184] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0185] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0186] The present application uses specific examples to illustrate the principles and implementation methods of the present application. The above examples are only used to help understand the method and core ideas of the present application. At the same time, for those skilled in the art, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A remote sensing image clustering method, characterized in that: The remote sensing image clustering method comprises: Acquisition of remote sensing images; Performing super-pixel segmentation on the remote sensing image to obtain a super-pixel map; Obtaining superpixel features and an adjacency matrix based on the superpixel graph; An image clustering model is constructed, and the superpixel features and the adjacency matrix are input into the image clustering model to obtain a clustering result; the image clustering model includes a cross-attention graph diffusion module, a discriminative spectral feature extraction module, a graph self-contrast loss function and a clustering output module; the cross-attention graph diffusion module is used to extract structural information in the remote sensing image based on the adjacency matrix; the discriminative spectral feature extraction module is used to extract spectral information in the remote sensing image based on the superpixel features, and is used to encode the structural information and the spectral information to obtain a sample similarity matrix; the graph self-contrast loss function is used to train the image clustering model; the clustering output module is used to obtain the clustering result based on the sample similarity matrix; the graph self-contrast loss function is expressed as: ; In the formula, represents the self-contrastive loss, represents the feature self-contrastive loss, represents the structural self-contrast loss, represents the self-constrained clustering loss, represents the weight factor of the structural self-contrast loss, Represents the weight factor of the self-constrained clustering loss; completing classification of objects in the remote sensing image based on the clustering result; The process of extracting spectral information from the remote sensing image based on the superpixel feature by the discriminative spectral feature extraction module and encoding the structural information and the spectral information to obtain a sample similarity matrix includes: A multi-layer graph Laplacian filter is used to filter the super-pixel feature to obtain a spectral smoothing feature; the spectral smoothing feature is the spectral information; Dividing the spectral smoothing feature into a plurality of characteristic spectrum segments, and determining a characteristic difference matrix; splicing the characteristic spectrum segment and the characteristic difference matrix to obtain a discriminative feature; Using a graph structure encoding module to perform graph structure encoding on the structural information and the discriminative features to obtain output features; A sample similarity matrix is ​​determined according to the output features.

2. The remote sensing image clustering method according to claim 1, characterized in that: The process of the cross attention graph diffusion module extracting structural information in the remote sensing image based on the adjacency matrix includes: Constructing a symmetric normalized adjacency matrix based on the adjacency matrix; The neighborhood self-attention layer is used to process the constructed symmetric normalized adjacency matrix with a self-attention mechanism to obtain a self-attention graph; Constructing a K nearest neighbor graph according to the adjacency matrix; A multi-view cross-attention layer is used to perform a cross-attention mechanism on the K nearest neighbor graph and the self-attention graph to obtain a cross-attention graph, and a structural similarity matrix is ​​obtained based on the cross-attention graph; A multi-layer perceptron is used to obtain cross-attention features based on the structural similarity matrix; A graph diffusion module is used to perform graph information aggregation operation on the cross attention features to obtain a stable state of the graph; The cross-attention feature is fused with the steady state of the graph through a normalization factor to obtain the structural information.

3. The remote sensing image clustering method according to claim 1, characterized in that: The spectral smoothing feature is divided into a plurality of characteristic spectrum segments, and a characteristic difference matrix is ​​determined, including: The spectral similarity between pixels in each characteristic spectrum band is measured by spectral angle similarity measurement to obtain a similarity matrix; A feature difference factor is constructed based on the similarity matrix to obtain the feature difference matrix.

4. The remote sensing image clustering method according to claim 1, characterized in that: Performing super-pixel segmentation on the remote sensing image to obtain a super-pixel map includes: A simple iterative clustering algorithm is used to perform super-pixel segmentation on the remote sensing image to obtain the super-pixel map.

5. A remote sensing image clustering system, characterized in that: The remote sensing image clustering system is used to implement the remote sensing image clustering method according to any one of claims 1 to 4; the remote sensing image clustering system comprises: An image acquisition unit, used for acquiring remote sensing images; A pixel segmentation unit, used to perform super-pixel segmentation on the remote sensing image to obtain a super-pixel map; A model building unit, used for building an image clustering model, and inputting the superpixel map into the image clustering model to obtain a clustering result; the image clustering model includes a cross-attention map diffusion module, a discriminative spectral feature extraction module and an image self-contrast loss function; the cross-attention map diffusion module is used to extract structural information in the superpixel map; the discriminative spectral feature extraction module is used to extract spectral information in the superpixel map, and is used to encode the structural information and the spectral information to obtain a sample similarity matrix; the image self-contrast loss function is used to train the image clustering model, and is used to obtain the clustering result based on the sample similarity matrix; A ground object classification unit is used to complete the ground object classification in the remote sensing image based on the clustering result.

Images