A clustering method and system for incomplete views

Abstract

The embodiment of the present invention provides a clustering method and system for incomplete views, which belongs to the field of information technology. The clustering method includes: obtaining incomplete multiple views to be clustered, and constructing an adjacency matrix to determine the graph structure information of the incomplete multiple views; combining the adjacency matrix, using a pre-constructed clustering model to encode each incomplete view to obtain a potential feature representation; using an attention mechanism to fuse each potential feature representation to generate a comprehensive potential feature representation; performing deep clustering on the comprehensive potential feature representation, and outputting the clustering results of the incomplete multiple views. By inputting the incomplete multiple views to be clustered and the adjacency matrix into the clustering model, the clustering model performs multiple neighborhood aggregation and feature learning on the incomplete multi-view nodes to obtain the potential feature representation of each node, which is more conducive to clustering, and uses the attention mechanism to fuse the potential feature representation to generate a comprehensive potential feature representation, and performs deep clustering to obtain the clustering results.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a clustering method and system for incomplete views. Background Art

[0002] With the development of science and technology and the widespread use of sensors, a large amount of data is generated in actual production and life, and these data are often stored in different forms. For example, images display their features in different forms such as HOG, SIFT, SURF, ORB, LBP, HAAR, etc., and videos can contain time-synchronized pictures and audio information. The same information may be stored in these data with different structures, which constitutes multi-view data. Compared with a single view, multi-view data can often display its feature information from different angles. In clustering tasks, the feature information that can be extracted will also be more comprehensive.

[0003] Multi-view clustering is a clustering method that integrates multiple views of data, where each view represents a different feature or expression of the data. By integrating information from multiple views, multi-view clustering aims to improve the understanding of complex data structures and to more accurately divide sample points into similar groups by simultaneously considering information from different views, so as to achieve the purpose of improving clustering performance. This method is widely used to process multimodal data, where each modality provides a unique perspective on the data, prompting the algorithm to more comprehensively capture the intrinsic structure of the data.

[0004] Some incomplete multi-view clustering algorithms have been proposed in the prior art, such as methods based on non-negative matrix factorization, subspace-based clustering methods, spectral clustering-based, kernel-based and deep neural network-based methods. They can be roughly divided into two categories. One is to use existing data to directly perform multi-view clustering without special processing of missing data. The other is to use strategies to infer and supplement missing data. For example, recover data from shared subspaces, use autoencoders to recover missing data, use adversarial generative networks to generate missing data, use graph encoders to encode missing data, etc. However, most of the existing incomplete multi-view clustering methods based on self-representation have the following limitations: using existing data to directly perform multi-view clustering is not conducive to multi-view due to the lack of view data. Figure 1 The learning of consistent representation affects the clustering performance. Summary of the invention

[0005] In view of the continuous expansion of the application scope of clustering algorithms in actual production and life, the problem of how to continue to maintain the accuracy of clustering algorithms in the presence of image missing is raised.

[0006] In order to clearly demonstrate the specific implementation method of this application, some related concepts are introduced first:

[0007] Graph Convolutional Networks (GCN) is a type of neural network specifically designed to process graph-structured data. First, the graph convolutional neural network represents data in the form of a graph, where each node represents an object and the edge represents the relationship between nodes. Second, the graph convolutional neural network creates an adjacency matrix and a feature matrix, and uses a convolutional layer to continuously iterate its node representation. Finally, a fully connected layer is used to output the node feature vector.

[0008] There are many options for feature fusion mechanisms between different views, such as simple averaging or weighted averaging. The attention fusion mechanism is a more complex but clever fusion method. The attention fusion mechanism is a deep learning technology that increases the influence of important features by assigning different weights to input features, and combines these weighted features to more effectively represent and process data, thereby improving the performance and accuracy of the model.

[0009] The process of clustering algorithm involves dividing the objects in the data set into multiple groups or "clusters" based on similarity, usually by calculating the distance or similarity between data points (such as Euclidean distance or cosine similarity) and applying specific algorithm logic (such as iterative centroid update in K-means or eigenvector decomposition in spectral clustering) to determine the attribution of data points, thereby achieving unsupervised classification of data. Traditional clustering algorithms such as K-means or spectral clustering are widely used, but most of them are sensitive to initial values ​​or noise. Therefore, in practical applications, choosing a suitable clustering algorithm needs to consider the characteristics of the data and the specific requirements of the clustering task.

[0010] Mutual Information: Mutual Information (MI) is a measure of the amount of shared information between two random variables. It is based on the concept that knowing one of the two variables is helpful for predicting the other variable, so it is often used to measure the mutual dependence between two random variables.

[0011] Its mathematical expression is as follows: Mutual information can be defined as two random variables and The joint distribution of , and their independent distributions The differences:

[0012]

[0013] In the case of continuous variables, the sum is replaced by the integral. The remaining mutual information related concepts are as follows:

[0014] Entropy: Entropy is a measure of the random variable A measure of uncertainty.

[0015] Conditional Entropy: Conditional Entropy is a known random variable In the case of uncertainty.

[0016] The relationship between mutual information and entropy: Mutual information can be regarded as the amount of uncertainty reduction, that is, .

[0017] The purpose of the embodiments of the present invention is to provide a method and system for clustering incomplete views, which are used to completely or at least partially solve the technical problems existing in the above-mentioned prior art.

[0018] In order to achieve the above-mentioned object, an embodiment of the present invention provides a clustering method for incomplete views, comprising: obtaining incomplete multiple views to be clustered, and constructing an adjacency matrix for each incomplete view to determine graph structure information of the incomplete multiple views to be clustered, wherein the adjacency matrix represents the connection relationship between nodes in each incomplete view or the strength of the connection between nodes;

[0019] Combined with the adjacency matrix constructed for each incomplete view, a pre-built clustering model is used to encode each incomplete view to obtain the potential feature representation of each incomplete view;

[0020] The attention mechanism is used to fuse the latent feature representations of each incomplete view to generate a comprehensive latent feature representation;

[0021] Deep clustering is performed on the comprehensive latent feature representation, and a clustering result of incomplete multiple views is output.

[0022] Optionally, obtain incomplete multiple views to be clustered, and construct an adjacency matrix for each incomplete view, including:

[0023] Determining nodes in each incomplete view to be clustered;

[0024] For complete nodes, if there is a direct connection between the nodes, the corresponding matrix element is 1, if there is no direct connection between the nodes, the corresponding matrix element is 0;

[0025] For missing nodes, neighboring nodes within the target range of the missing nodes are selected to construct an adjacency matrix.

[0026] Optionally, the latent feature representation of each incomplete view is calculated according to the following formula:

[0027]

[0028] In the formula, For the The potential feature representation of the encoded view, softmax is the activation function, , , is the adjacency matrix, for The unit diagonal matrix of is the value of the i-th row and j-th column of the diagonalized adjacency matrix A, for The matrix obtained by summing up the columns is is the trainable parameter of the mth layer of the encoder of the stacked graph convolutional neural network model, Represents the graph convolution operation of the m-1th layer.

[0029] Optionally, generate a comprehensive latent feature representation according to the following formula:

[0030]

[0031] Where V is the number of views, is element-wise multiplication with broadcasting, The attention distribution is determined by the following formula:

[0032]

[0033] , For the Kernel learning of views, and are the parameters of the fully connected layer, is the deviation matrix.

[0034] Optionally, the pre-built clustering model includes: a stacked graph convolutional neural network autoencoder module, a fusion module and a deep clustering guidance module, wherein the deep clustering guidance module includes a main clustering module and an auxiliary clustering module.

[0035] Optionally, the performing deep clustering on the comprehensive potential feature representation and outputting a clustering result of an incomplete multi-view includes:

[0036] inputting the comprehensive latent feature representation into the main clustering module to obtain a soft assignment of clusters;

[0037] The comprehensive latent feature representation is input into the auxiliary clustering module to obtain a soft assigned label.

[0038] Optionally, the soft allocation of the cluster is calculated according to the following formula:

[0039]

[0040] In the formula, represents the soft allocation of clusters, represents the comprehensive latent feature representation, Represents the cluster centers in the main clustering module.

[0041] Optionally, the clustering method further includes:

[0042] The incomplete multi-views to be clustered are subjected to deletion processing to obtain a reconstructed graph structure corresponding to the incomplete multi-views to be clustered.

[0043] Optionally, before fusing the latent feature representations of the incomplete views using an attention mechanism to generate a comprehensive latent feature representation, the clustering method further includes:

[0044] Use JS divergence to align incomplete multi-views of clustering:

[0045]

[0046] In the formula, are the latent feature representations after encoding the 1st and 2nd views, , They are The value of row i and column j.

[0047] On the other hand, the present invention also provides an incomplete view clustering system, comprising:

[0048] An acquisition unit, configured to acquire incomplete multiple views to be clustered, and construct an adjacency matrix for each incomplete view to determine graph structure information of the incomplete multiple views to be clustered, wherein the adjacency matrix represents a connection relationship between nodes in each incomplete view or a strength of connection between nodes;

[0049] A decoding unit, configured to encode each incomplete view by combining the adjacency matrix constructed for each incomplete view using a pre-constructed clustering model to obtain a potential feature representation of each incomplete view;

[0050] A fusion unit is used to fuse the latent feature representations of each incomplete view using an attention mechanism to generate a comprehensive latent feature representation;

[0051] The output unit is used to perform deep clustering on the comprehensive potential feature representation and output a clustering result of the incomplete multi-view.

[0052] Through the above technical scheme, the incomplete multi-views to be clustered and the adjacency matrix constructed by each incomplete view are input into the clustering model, so that the clustering model obtains the potential feature representation of each node after multiple neighborhood aggregation and feature learning on the incomplete multi-view nodes, which is more conducive to clustering, and uses the attention mechanism to fuse the potential feature representation to generate a comprehensive potential feature representation, and then performs deep clustering based on the comprehensive potential feature representation to obtain the clustering result.

[0053] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0055] Figure 1 is an implementation flow chart of an incomplete view clustering method provided by an embodiment of the present invention;

[0056] Figure 2 It is an overall framework diagram of an algorithm provided by an embodiment of the present invention;

[0057] Figure 3 is an example diagram of an incomplete view provided by an embodiment of the present invention;

[0058] Figure 4 This is a comparative experimental data diagram of an embodiment of the present invention on different data sets with existing methods;

[0059] Figure 5 This is a graph of experimental results of different missing rates on the same data set provided by an embodiment of the present invention;

[0060] Figure 6 It is a structural schematic diagram of an incomplete view clustering system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0062] See also Figure 1 As shown, it is a flowchart of an implementation of an incomplete view clustering method provided by an embodiment of the present invention, including the following execution steps:

[0063] Step 100: Obtain incomplete multiple views to be clustered, and construct an adjacency matrix for each incomplete view to determine graph structure information of the incomplete multiple views to be clustered.

[0064] The adjacency matrix represents the connection relationship between nodes or the strength of the connection between nodes in each incomplete view.

[0065] In some embodiments, there is a predefined The multi-view data of an instance is ,and Indicates The feature matrix in the view is Indicates The graph structure of the data is represented by the adjacency matrix , express and For convenience, we only discuss the dual view (missing view) Figure 1 , missing vision Figure 2 ) (see Figure 3 ), this situation can be easily extended to multiple views.

[0066] In some implementations, when executing step 100, the following steps may be specifically performed:

[0067] S1000: Determine nodes in each incomplete view to be clustered.

[0068] It should be understood that these nodes are usually key feature points of the image.

[0069] S1001: For complete nodes, if there is a direct connection relationship between the nodes, the corresponding matrix element is 1 (or a weight value representing the connection strength); if there is no direct connection relationship between the nodes, the corresponding matrix element is 0.

[0070] S1002: For a missing node, select neighboring nodes within a target range from the missing node to construct an adjacency matrix.

[0071] It should be understood that the adjacency matrix is ​​an N×N square matrix, where N is the number of nodes.

[0072] In some embodiments, weights also need to be considered: in some cases, the connections between nodes are not just binary (present or not present), but have different strengths or importance. In this case, the elements in the adjacency matrix can be a weight value, indicating the strength of the connection between the nodes. In order to make the training process more stable, it is usually necessary to perform some normalization on the adjacency matrix. For example, the adjacency matrix can be normalized using a degree matrix. Finally, the effect of the adjacency matrix needs to be tested in an actual graph convolutional network and optimized as needed, which may involve adjusting the selection of nodes, the definition of connections, or the way weights are assigned.

[0073] Step 101: Combining the adjacency matrix constructed for each incomplete view, using a pre-constructed clustering model to encode each incomplete view, and obtaining a potential feature representation of each incomplete view.

[0074] In some embodiments, see Figure 2 As shown, the pre-built clustering model includes a stacked graph convolutional neural network autoencoder module, a fusion module and a deep clustering guidance module, wherein the deep clustering guidance module includes a main clustering module and an auxiliary clustering module.

[0075] In some embodiments, after performing multiple neighborhood aggregation and feature learning on nodes through GCN, new feature representations of each node can be obtained, and these new feature representations capture the complex interactions and relationships between facial key points.

[0076] In some embodiments, the stacked graph convolutional neural network autoencoder not only captures the attribute information of the nodes, but also captures the structural information of the graph. In order to learn useful representations from the data, a stacked GCN layer is used as an encoder. The encoding of face image information is achieved using a stacked graph convolutional neural network (Stacked GCN). The steps for building and training the stacked graph convolutional neural network (Stacked GCN) are as follows:

[0077] (1) Design the GCN layer: Step 1: Select the appropriate graph convolution operation to combine the features of the node with the features of its neighbors. Step 2: Define feature transformation. In each GCN layer, the features of the node are linearly transformed through a weight matrix. This weight matrix is ​​learnable and determines how to transform the original features of the node into a higher-level representation. Step 3: Integrate neighbor information and use the normalized adjacency matrix to guide the aggregation of node features. Step 4: Apply nonlinear activation functions, such as ReLU, to enhance the expressiveness of the model.

[0078] (2) Stacking GCN layers: First, build a deep network by adding more GCN layers based on a single GCN layer. The output of each layer becomes the input of the next layer. Second, adjust the feature dimension of each layer. The primary layer maintains a higher dimension to capture detailed information, while higher layers reduce the dimension to extract more abstract representations. Finally, to avoid overfitting, regularization layers can be added between GCN layers.

[0079] Through steps (1) and (2), an effective stacked GCN model can be constructed and trained to obtain the final encoding output.

[0080] In some embodiments, the training process of the clustering model is as follows:

[0081] (1) Initialize network parameters: Initialize the parameters of the stacked GCN.

[0082] (2) Select loss function: Define the loss function to guide both feature extraction and clustering.

[0083] It includes view reconstruction loss and clustering loss.

[0084] (3) Select an optimizer: Select an appropriate optimizer. The present invention uses the Adam optimizer.

[0085] And set an appropriate learning rate.

[0086] (4) Batch training: Use small batches of data for iterative training. In each iteration, execute

[0087] The following steps: forward propagation: pass data through stacked GCN to obtain feature representation and clustering output; loss calculation: calculate the loss of the current model according to the predefined loss function; back propagation: calculate the gradient of the loss function with respect to the network parameters; parameter update: update the network parameters according to the gradient and optimizer.

[0088] (5) Monitor the training process: monitor changes in the loss function and possible overfitting.

[0089] In some embodiments, the graph autoencoder captures not only the attribute information of the nodes, but also the structural information of the graph. In order to learn useful representations from the data, a stacked GCN layer is used as the encoder. The mapping function of a view can be expressed as a function ,in It is The potential representation of the view, Represents the encoder parameters. Specifically, the graph convolution operation of the mth layer:

[0090]

[0091] In the formula, , . for The unit diagonal matrix of is the value of the i-th row and j-th column of the diagonalized adjacency matrix A, for The matrix obtained by summing up the columns is is the trainable parameter of the encoder layer m, is an activation function, such as relu. ,In the last layer of the encoder, softmax is used as the activation function.

[0092] In some implementations, the potential feature representation of each incomplete view may be calculated according to the following formula:

[0093]

[0094] In the formula, For the The potential feature representation of the encoded view, softmax is the activation function, , , is the adjacency matrix, for The unit diagonal matrix of is the value of the i-th row and j-th column of the diagonalized adjacency matrix A, for The matrix obtained by summing up the columns is is the trainable parameter of the mth layer of the encoder of the stacked graph convolutional neural network model, Represents the graph convolution operation of the m-1th layer.

[0095] In some embodiments, see Figure 2 As shown in the figure, in order to prevent the encoder from excessively distorting the graph structure of the incomplete views to be clustered when extracting information, a A graph decoder composed of view-specific decoders is used to reconstruct the graph structure The mapping function of a specific view decoder can be expressed as ,in More specifically, the graph structure decoding operation can be described as follows:

[0096]

[0097] in, For the The latent feature representation of the encoded view is It is The weights of the view-specific decoder, express The transposed matrix of .

[0098] In some embodiments, after obtaining the encoded information of multiple images from different perspectives or under different conditions, they are merged to obtain a more comprehensive and accurate potential representation. Directly concatenating or averaging the information of multiple views may not produce good clustering performance, so an attention mechanism fusion method (corresponding to Figure 2 In this way, the view fusion module and the deep clustering module can promote each other and further improve the clustering performance. The relevant steps are as follows:

[0099] Step 102: The latent feature representations of each incomplete view are fused using an attention mechanism to generate a comprehensive latent feature representation.

[0100] In some embodiments, first, define the attention mechanism: design an attention model that can learn which features in different views are more important. Secondly, feature fusion preparation: prepare features for use by the attention model, which involves splicing or merging the features into a unified format. Thirdly, calculate attention weights: calculate an attention weight for each feature, which reflects the contribution of each feature to the final fusion result. Finally, weighted feature fusion: according to the calculated attention weights, weighted fusion of features from different views into a comprehensive feature representation.

[0101] In some embodiments, assuming that Different information sources (or feature sets), each source There is a corresponding eigenvector The purpose of the attention fusion mechanism is to learn a weight coefficient These feature vectors are combined (usually determined by some form of attention score).

[0102] Calculate the attention weights:

[0103]

[0104] in, is a function that calculates the attention score for each feature set.

[0105] Fuse feature vectors:

[0106]

[0107] in, is the fused feature vector, which is the weighted sum of each feature vector.

[0108] In some implementations, the comprehensive latent feature representation is generated according to the following formula:

[0109]

[0110] In the formula, is element-wise multiplication with broadcasting, The attention distribution is determined by the following formula:

[0111]

[0112] , For the Kernel learning of views, and are the parameters of the fully connected layer, is the deviation matrix.

[0113] In some implementations, in order to better fuse views and thus learn the consistency of different views, a dual consistency loss is designed. Specifically, one is to maximize and The other is to minimize and JS differences (corresponding to Figure 2 In , min ).because and are obtained after softmax, so they can be considered as probability distributions over d classes. We can define for:

[0114]

[0115] In the formula, , are the potential feature representations after encoding for the 1st and 2nd views, respectively. for The transposed matrix of .

[0116] set up and They represent the marginal probability distribution and , they can be obtained by d Line and d’ In order to make z and z' equally important and P symmetric, we further Calculate P. Therefore, it can be calculated by the following formula and The mutual information between:

[0117]

[0118] In the formula, and They represent the marginal probability distribution and , is a specific probability value.

[0119] According to information theory, information entropy From self-information Therefore, in order to obtain the same form, the regularization term is added and . Therefore, the consistency loss is:

[0120]

[0121] In the formula, , Respectively for The latent feature representation of each view after encoding.

[0122] Viewpoint alignment is very important for subsequent multi-view fusion. In order to align the views, you need to use and The distribution of each instance of is as identical as possible. and can be regarded as a probability distribution over D classes, so Figure 1 Consistency is used while JS divergence is used to align views.

[0123] In some implementations, before executing step 102, a step is further executed to align the incomplete multiple views to be clustered using JS divergence:

[0124]

[0125] In the formula, , are the potential feature representations after encoding for the 1st and 2nd views, respectively. , They are The value of row i and column j.

[0126] The second consistency loss is:

[0127]

[0128] In the formula, They are the latent feature representations after encoding of the 1st and 2nd views respectively.

[0129] Step 103: performing deep clustering on the comprehensive latent feature representation, and outputting a clustering result of an incomplete multi-view.

[0130] In some embodiments, the deep clustering module is mainly divided into two modules, a main clustering module and an auxiliary clustering module. The main clustering module based on deep embedding clustering: Step 1: Initialize the cluster centers. Initialize the cluster centers in the feature space through the K-means clustering method. Step 2: Design a loss function and define a loss function, which is usually based on the clustering target. DEC commonly uses an objective function based on the probability of cluster assignment. The cross entropy loss function based on the KL (Kullback-Leibler Divergence) divergence is defined in this application. Step 3: Assign clusters: After the training is completed, use the final network model to assign each data point to the nearest cluster center. Auxiliary clustering module based on Cauchy-Schwarz (CS) divergence: The design of a deep clustering module based on CS (Cauchy-Schwarz) divergence involves building a deep learning framework that can minimize the CS divergence, thereby achieving more effective data clustering. CS divergence is a method to measure the difference between two probability distributions. It is used in deep clustering to optimize the feature space, making data points of the same category closer and data points of different categories more dispersed. The design of the auxiliary clustering module is similar to that of the main clustering module. The difference lies in the selection of the loss function. The auxiliary clustering module minimizes the defined CS divergence loss function by optimizing the network parameters, thereby adjusting the feature space to achieve the optimal clustering result.

[0131] In some implementations, the incomplete multi-view clustering result output in step 103 includes:

[0132] First, the comprehensive latent feature representation is input into the main clustering module to obtain a soft assignment of clusters.

[0133] In some embodiments, for the main clustering module, inspired by DEC, the student-t distribution is used as the kernel to measure the latent representation With cluster center The similarity between them is used to obtain the cluster soft assignment. The initial cluster center is obtained by performing kmeans on the fusion representation after pre-training. The equation is as follows:

[0134]

[0135] In the formula, represents the soft allocation of clusters, represents the comprehensive latent feature representation, Represents the cluster centers in the main clustering module.

[0136] In order to optimize the data representation by learning from high-confidence assignments, the following target distribution P is designed:

[0137]

[0138] in, is the soft cluster frequency used to normalize the loss function for each centroid. Therefore, the clustering loss is defined as:

[0139]

[0140] In the formula, is the specific probability value in the target distribution P, Indicates soft allocation of clusters.

[0141] Secondly: the comprehensive latent feature representation is input into the auxiliary clustering module to obtain a soft assigned label.

[0142] In some embodiments, an auxiliary clustering guidance module is used, a classifier consisting of fully connected layers, and finally a softmax to supplement the above clustering module. After the latent representation passes through this module, a discrete assignment of D classes can be obtained. Specifically, the Cauchy-Schwarz (CS) divergence is used as a measure of the quality of the cluster distribution. The definition of CS divergence is as follows:

[0143]

[0144] In the formula, and Node and The probability density distribution function of .

[0145] According to DDC, which is a data-driven approach, the use of The Parzen window estimator with a Gaussian kernel is used to approximate the probability density function. Therefore, the CS divergence can be expressed as:

[0146]

[0147] In the formula, is the node estimated by the Gaussian kernel and The probability density between .

[0148] From the above formula, we can see that the similarity of instances belonging to the same cluster is large, while the similarity of instances belonging to other clusters is small, so the clustering can be made compact and different clusters can be separated. It can also be regarded as the cosine similarity between different clusters.

[0149] At the same time, the soft assigned labels can be obtained through the DDC clustering guidance module of the model (correspond Figure 2 In the (over-clustering pseudo-labels) in, it can be seen as the probability of over-clustering classes on more clusters than the true clusters. Therefore, minimizing the CS divergence is equivalent to minimizing the following formula:

[0150]

[0151] In the formula, is the soft assigned label, yes No. List, for The transposed matrix of is the node estimated by the Gaussian kernel and The probability density between Yes Storage The matrix of .

[0152] Furthermore, to avoid degenerate clustering, the output is forced to be close to the simplex angle. ,in is a matrix No. List, Represents the simplex angles. Therefore, the geometry is expressed as CS divergence, which is as follows:

[0153]

[0154] In the formula, is the label matrix after forcing the output simplex angles, For the matrix The i-th column of is its transposed matrix, K is the same as above.

[0155] In order to obtain different clusters that are orthogonal to each other, mathematically speaking, ,in, represents the strictly upper triangular element sum. Therefore, the total loss of the DDC cluster module is as follows:

[0156]

[0157] In the formula, represents the total loss of the DDC cluster module, represents the CS divergence constraint loss, represents the constraint loss forcing the output simplex angle, A constraint loss that represents the orthogonality of clustering results.

[0158] It is important to note that D is a hyperparameter, not necessarily the number of clusters, as it may conflict with the main clustering module. This is because DEC emphasizes high-confidence instances but blurs instances at the edge of the cluster. However, DDC makes the clusters more compact, which means that instances at the edge of the cluster will be forced to move to the center of the cluster. One solution proposed for this problem is to set D larger when there are many instances at the edge of the cluster. In this way, the two clustering modules both conflict and complement each other.

[0159] In some implementations, a step is further performed to perform deletion processing on the incomplete multiple views to be clustered to obtain a reconstructed graph structure corresponding to the incomplete multiple views to be clustered.

[0160] On the one hand, when constructing the adjacency matrix of an incomplete view, if there is missing node information, GCN is used to estimate its feature representation based on the nearest k neighbor nodes in its adjacency matrix. On the other hand, when decoding the view, a view-specific decoder network is designed to convert the completed feature representation back to the form of the original view.

[0161] In some embodiments, deletion processing is interpreted as: for each , select k nearest neighbors based on the similarity between data to establish edge relationships. Specifically, calculate the similarity matrix of data on each view separately There are many methods to calculate similarity, such as Heat Kernel: And Dotproduct: ,in express and In the In this case, the similarity-based adjacencies of these missing instances are empty. However, there is a consistency assumption between views, so the adjacencies of the other view of these instances are used as their adjacencies. For the first view Some instances are missing in the second view Find its instance in the The graph structure of ,in and Respectively and No. Similarly, for the second view The same operation is performed for the missing instances in . Expand to multiple views, for the Some instances are missing in one view and the graph structure of these instances is used in other views.

[0162] In order not to destroy the structural information, a view-specific decoder is used to reconstruct the graph structure of each view. Mathematically, the designed reconstruction loss is as follows:

[0163]

[0164] In the formula, represents the equilibrium parameter, is the latent feature representation after encoding of the m-th view, is the feature matrix of the m-th view, is the adjacency matrix of the m-th view, For Adjacency matrix after applying the diagonalization constraint.

[0165] Multi-view data usually has a view with the most information to control the clustering performance, so a balance parameter is introduced in the formula .

[0166] In some embodiments, the DEC module is first removed, the network is pre-trained, and then Kmeans clustering is performed using the latent variables obtained from the pre-training to initialize the centroid of the DEC module. Then, the network is trained end-to-end, where the centroid and network parameters are updated alternately. According to the above definition, the total loss of the model is:

[0167] In the formula, Represents the loss of view reconstruction, i.e., the loss calculated by the missing processing module equation, represents the loss calculated by the equation of the information fusion module between views, Represents the main clustering module equation to calculate the loss, Represents the auxiliary clustering module equation to calculate the loss, , , , , They represent the weights corresponding to different losses.

[0168] The Adam optimizer is used and the learning rate is set to 0.001. In each round of training, all the data are sent to the network. The data is pre-trained for 30 epochs and then jointly trained with DEC for 20 epochs. It should be noted that during the joint training process, the centroid is likely to deviate too much, so the update frequency of the target distribution is a hyperparameter that needs to be adjusted. µ5 is set to 0.01 and the update level is set to 1 for all datasets. The rest of the hyperparameters are set between [0.01, 1]. To evaluate the performance of the model on incomplete multi-view data, one view is randomly deleted as the incomplete data of the complete data. The missing rate η is defined as η = (n−m) / n, where m is the number of complete instances and n is the number of all instances.

[0169] The algorithm is described in detail as follows:

[0170]

[0171] In some embodiments, experiments are conducted on three scene datasets Caltech101-20, LandUse-21, and Scene-15 in complete cases and under different missing conditions. In order to evaluate the performance of the model, three widely used clustering indicators are used, namely clustering accuracy (ACC), normalized mutual information (NMI), and average rand index (ARI). The larger the three indicators, the better the clustering performance. The experimental results are shown in Figure 2. Figure 4 , Figure 5 As shown in Figure 2, the proposed method is compared with 6 multi-view clustering methods, including Deep Canonical Correlation Analysis (DCCA) and Perturbation-Oriented Incomplete Multi-View Clustering (PIC). For all the compared methods, the recommended network structure and parameters are used for fair comparison. In short, Figure 4 As shown in Figure 2, the present invention outperforms many current mainstream incomplete multi-view clustering algorithm models on many public data sets. In order to further explore the model's ability and advantages in handling missing data, the missing rate is set to 0.1 to 0.3 with an interval of 0.1 on Caltech101-20. Figure 5 As shown in Figure 3, when the missing rate is low, the clustering performance of the model only decreases slightly, but still maintains good clustering performance.

[0172] The encoder is used to obtain the latent representation and set the double consistency constraint. In addition, there are two clustering guidance modules to make the learned latent representation more conducive to clustering. Different from the traditional method, the clustering model of this application is clustering-oriented from beginning to end. Specifically, based on multi-view Figure 1We first use graph embedding to handle missing views and obtain latent representations of the data, including missing data. We then use the distribution and mutual information between views to emphasize dual consistency and learn clustering-friendly latent representations through dual clustering.

[0173] See also Figure 6 As shown, it is a schematic diagram of the structure of an incomplete view clustering system provided by an embodiment of the present invention, including:

[0174] An acquisition unit 600 is used to acquire incomplete multiple views to be clustered, and construct an adjacency matrix for each incomplete view to determine graph structure information of the incomplete multiple views to be clustered, wherein the adjacency matrix represents the connection relationship between nodes in each incomplete view or the strength of the connection between nodes;

[0175] A decoding unit 601 is used to encode each incomplete view by combining the adjacency matrix constructed for each incomplete view and using a pre-constructed clustering model to obtain a potential feature representation of each incomplete view;

[0176] A fusion unit 602 is used to fuse the potential feature representations of each incomplete view using an attention mechanism to generate a comprehensive potential feature representation;

[0177] The output unit 603 is used to perform deep clustering on the comprehensive potential feature representation and output a clustering result of the incomplete multi-view.

[0178] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0179] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0180] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0181] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0182] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0183] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0184] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0185] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0186] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A clustering method for incomplete views, characterized in that: include: Acquire incomplete multiple views to be clustered, and construct an adjacency matrix for each incomplete view to determine graph structure information of the incomplete multiple views to be clustered, wherein the adjacency matrix represents a connection relationship between nodes in each incomplete view or a strength of connection between nodes; Combined with the adjacency matrix constructed for each incomplete view, a pre-built clustering model is used to encode each incomplete view to obtain the potential feature representation of each incomplete view; The attention mechanism is used to fuse the latent feature representations of each incomplete view to generate a comprehensive latent feature representation; performing deep clustering on the comprehensive latent feature representation and outputting a clustering result of incomplete multiple views; Obtain incomplete multiple views to be clustered, and construct an adjacency matrix for each incomplete view, including: Determining nodes in each incomplete view to be clustered; For complete nodes, if there is a direct connection between the nodes, the corresponding matrix element is 1, if there is no direct connection between the nodes, the corresponding matrix element is 0; For missing nodes, select neighboring nodes within the target range of the missing nodes to construct an adjacency matrix; The latent feature representation of each incomplete view is calculated according to the following formula: In the formula, For the The potential feature representation of the encoded view, softmax is the activation function, , , is the adjacency matrix, for The unit diagonal matrix of is the value of the i-th row and j-th column of the diagonalized adjacency matrix A, for The matrix obtained by summing up the columns is is the trainable parameter of the mth layer of the encoder of the stacked graph convolutional neural network model, Represents the graph convolution operation of the m-1th layer; The comprehensive latent feature representation is generated according to the following formula: Where V is the number of views, is element-wise multiplication with broadcasting, The attention distribution is determined by the following formula: , For the Kernel learning of views, and are the parameters of the fully connected layer, is the deviation matrix; The pre-built clustering model includes: a stacked graph convolutional neural network autoencoder module, a fusion module and a deep clustering guidance module, wherein the deep clustering guidance module includes a main clustering module and an auxiliary clustering module; The step of performing deep clustering on the comprehensive potential feature representation and outputting a clustering result of an incomplete multi-view comprises: inputting the comprehensive latent feature representation into the main clustering module to obtain a soft assignment of clusters; Inputting the comprehensive latent feature representation into the auxiliary clustering module to obtain soft assigned labels; The soft allocation of a cluster is calculated according to the following formula: In the formula, represents the soft allocation of clusters, represents the comprehensive latent feature representation, Represents the cluster centers in the main clustering module.

2. The clustering method according to claim 1, characterized in that: The clustering method further comprises: The incomplete multi-views to be clustered are subjected to deletion processing to obtain a reconstructed graph structure corresponding to the incomplete multi-views to be clustered.

3. The clustering method according to claim 1, characterized in that: Before fusing the latent feature representations of the incomplete views using an attention mechanism to generate a comprehensive latent feature representation, the clustering method further includes: Use JS divergence to align incomplete multi-views of clustering: In the formula, are the latent feature representations after encoding the 1st and 2nd views, , They are The value of row i and column j.

4. A clustering system for incomplete views, characterized in that: include: An acquisition unit, configured to acquire incomplete multiple views to be clustered, and construct an adjacency matrix for each incomplete view to determine graph structure information of the incomplete multiple views to be clustered, wherein the adjacency matrix represents a connection relationship between nodes in each incomplete view or a strength of connection between nodes; A decoding unit, configured to encode each incomplete view by combining the adjacency matrix constructed for each incomplete view using a pre-constructed clustering model to obtain a potential feature representation of each incomplete view; A fusion unit is used to fuse the latent feature representations of each incomplete view using an attention mechanism to generate a comprehensive latent feature representation; An output unit, used for performing deep clustering on the comprehensive latent feature representation and outputting a clustering result of an incomplete multi-view; Obtain incomplete multiple views to be clustered, and construct an adjacency matrix for each incomplete view, including: Determining nodes in each incomplete view to be clustered; For complete nodes, if there is a direct connection between the nodes, the corresponding matrix element is 1, if there is no direct connection between the nodes, the corresponding matrix element is 0; For missing nodes, select neighboring nodes within the target range of the missing nodes to construct an adjacency matrix; The latent feature representation of each incomplete view is calculated according to the following formula: In the formula, For the The potential feature representation of the encoded view, softmax is the activation function, , , is the adjacency matrix, for The unit diagonal matrix of is the value of the i-th row and j-th column of the diagonalized adjacency matrix A, for The matrix obtained by summing up the columns is is the trainable parameter of the mth layer of the encoder of the stacked graph convolutional neural network model, Represents the graph convolution operation of the m-1th layer; The comprehensive latent feature representation is generated according to the following formula: Where V is the number of views, is element-wise multiplication with broadcasting, The attention distribution is determined by the following formula: , For the Kernel learning of views, and are the parameters of the fully connected layer, is the deviation matrix; The pre-built clustering model includes: a stacked graph convolutional neural network autoencoder module, a fusion module and a deep clustering guidance module, wherein the deep clustering guidance module includes a main clustering module and an auxiliary clustering module; The step of performing deep clustering on the comprehensive potential feature representation and outputting a clustering result of an incomplete multi-view comprises: inputting the comprehensive latent feature representation into the main clustering module to obtain a soft assignment of clusters; Inputting the comprehensive latent feature representation into the auxiliary clustering module to obtain soft assigned labels; The soft allocation of a cluster is calculated according to the following formula: In the formula, represents the soft allocation of clusters, represents the comprehensive latent feature representation, Represents the cluster centers in the main clustering module.

Images