A Missing Multi-view Clustering Method Based on Integrated Heterogeneous Graph Attention Network

Abstract

The present invention discloses a missing multi-view clustering method based on an integrated heterogeneous graph attention network, and the method mainly includes the following steps: learning a common latent representation based on non-missing views; constructing a k-nearest neighbor graph based on the common latent representation, and constructing a specific missing pattern graph based on the missing conditions of different views. An integrated heterogeneous graph is obtained by integrating the k-nearest neighbor graph and the missing pattern graphs of different views, representing the relationship between samples under different missing patterns; the interaction between samples is further strengthened by using the graph attention mechanism, so as to learn a set of structured latent space representations; the consistency of the probability distribution is embedded in the network, so as to learn a consistent distribution for clustering. Experimental results on multiple real data sets verify the effectiveness and robustness of the method proposed in the present invention in missing multi-view clustering.

Description

Technical Field

[0001] The present invention relates to the field of multi-view learning, and in particular to a missing multi-view clustering method based on an integrated heterogeneous graph attention network. Background Art

[0002] With the rapid development of information technology, data collection methods are becoming more and more diverse, resulting in massive multi-view data. For example, news can be reported in different languages, or in the form of text, images, and videos; an image can be represented by different features, such as GIST, LBP, Garbor, SIFT, and HoG. In general, multi-view data contains richer information than a single view. Multi-view clustering adaptively divides data into their respective categories by reasonably fusing information from different views without any label information. Multi-view clustering has received increasing attention because it reduces the time and labor cost of multi-view data annotation.

[0003] Most traditional multi-view clustering methods assume that multi-view data is complete. However, in real life, due to problems such as machine downtime or sensor failure, some views of the collected data are missing. This kind of data is called missing multi-view data. The missing view not only leads to information loss, but also increases the difficulty of mining complementary information. These factors make missing multi-view clustering a challenging problem. Existing missing multi-view clustering can be divided into three categories: some methods ignore missing data and only use complete data. This method will lose a lot of information and cannot achieve ideal results when the missing rate is high or the sample size is small; some methods first group data with different missing conditions and then train them independently, but this method ignores the complex relationship between data and the grouping is more troublesome; some methods learn hidden layer representations for clustering through elastic models. This method focuses on utilizing the complementarity between views and ignores the structural information between samples, so it is difficult to achieve ideal clustering results. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present invention provides a missing multi-view clustering method based on an integrated heterogeneous graph attention network. The method simultaneously considers the complementarity between views, the structural information between samples, and the structural information of the missing views, and can achieve good clustering results under different missing conditions.

[0005] A missing multi-view clustering method based on an integrated heterogeneous graph attention network, the method comprising the following steps:

[0006] Step 1: Learn a common latent space representation for all available views.

[0007] The reconstruction loss function of the public latent space representation is:

[0008]

[0009] Among them, L r For the reconstruction loss, its goal is to learn the mapping between the original data and the common latent representation. Is The parameterized reconstruction network for the vth view, is the nth sample of the vth view, N and V represent the number of samples and views respectively, s nv Flag indicating whether the v-th view in the n-th sample is missing.

[0010] Step 2: Construct a k-nearest neighbor graph based on the common latent representation, and construct a missing pattern graph for each view. Combining the k-nearest neighbor graph and the missing pattern graph can obtain an integrated heterogeneous graph, which represents the proximity relationship between samples under different missing patterns.

[0011] The k-nearest neighbor graph refers to mining the relationship between different samples in the latent space based only on the distance information between samples. For samples i and j, their latent space representations are h i and h j , the distance between them can be calculated using the Gaussian kernel function. Therefore, a k-nearest neighbor graph G is constructed based on the public latent representation. n ∈R N×N , and its expression formula is as follows:

[0012]

[0013] Among them, N k (h i ) and N k (h j ) represent the k nearest neighbor samples of sample i and sample j respectively.

[0014] The missing pattern graph refers to the fact that for samples i and j in different views, they may have different missing conditions. Therefore, considering the missing conditions of all samples in view v, a missing pattern graph is constructed It can be calculated as follows:

[0015]

[0016] in, and denote the i-th and j-th samples in the v-th view respectively.

[0017] By considering the neighbor relationship between samples in the latent space and the missing patterns in different views, we can get the neighbor relationship between samples under different missing patterns. Combining the k-nearest neighbor graph and the missing pattern graph, we can get a latent space heterogeneous graph. It is defined as follows:

[0018]

[0019] Step 3: Use the graph attention mechanism to strengthen the interaction between samples and learn a set of structured latent representations.

[0020] There are V different types of graph structures in the heterogeneous graph G, representing the neighbor relationships of samples under different missing modes. By using the graph attention mechanism, we can explore the importance of each sample point to its neighbors in the latent space, and further aggregate the features of these important neighbors to obtain a structured latent representation. The graph attention mechanism can be defined as:

[0021]

[0022] Among them, h n is the public implicit representation, G v is the adjacency graph matrix, are the learnable parameters of the graph attention mechanism.

[0023] By learning the graph attention mechanism, we can obtain a set of structured latent representations:

[0024]

[0025] Step 4: For all views, learn a consistent distribution for clustering using the consistent distribution constraint.

[0026] Through the graph attention mechanism, the interactive relationship between samples under different missing patterns can be mined, and the structured latent representations corresponding to V different missing patterns can be obtained. In these V views, the distribution within each view should be consistent. Therefore, by considering the probability distribution of different views, we embed the consistency of probability distribution into the integrated heterogeneous graph network. Specifically, we first learn a set of probability distributions based on the latent representations of all views, and then make the probability distribution of each view as consistent as possible through constraints, and learn a consistent probability distribution.

[0027] For the vth view, we use Student's t-distribution as the kernel to measure the node z i ' (v) and cluster center μ j The similarities between. and They are the initial distribution Q (v) and the target distribution P (v) The elements in . Among them, It can be defined as:

[0028]

[0029] Among them, α is the degree of freedom of Student's t-distribution, C is the number of sample categories, represents the probability of node i being assigned to node j in view v. In our experiments, the initial cluster centers can be obtained by using K-means on the structured latent representation corresponding to the vth view, and the cluster centers of all views are set to the same to better follow the consistency principle.

[0030] By Raising to the second power and normalizing by the frequency of each cluster yields

[0031]

[0032] in, is the soft assignment frequency. In order to compare the similarity of two probability distributions, the loss function is defined as the probability distribution consistency loss L c The clustering loss is defined as minimizing the KL divergence between the original distribution and the target distribution of all views. Therefore, L c It can be defined as:

[0033]

[0034] Among them, KL divergence is used to measure the asymmetric difference between two probability distributions.

[0035] Finally, take The mean of is taken as the final consistent distribution and used for clustering.

[0036] The present invention aims at the clustering problem of missing multi-view data and proposes an integrated heterogeneous graph attention network model. Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0037] 1. The present invention simultaneously considers the complementarity between different views, the structural information between different samples, and the structural information of the missing view, achieves a balance between the consistency relationship and information complementarity between multi-view data, and improves the accuracy of missing multi-view clustering. The present invention can also be flexibly applied to different missing situations and has good robustness;

[0038] 2. The present invention uses the graph attention mechanism to aggregate the features of the heterogeneous neighbors of each node, thereby obtaining a more structured latent space representation;

[0039] 3. By constraining the probability distribution consistency of different views through distribution consistency, the distribution of learned features on different views is made as close as possible, thereby utilizing the consistency information between multiple views. At the same time, the distribution consistency constraint also promotes the learned features to have a better cluster structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0041] Figure 1 A flowchart of a missing multi-view clustering method based on an integrated heterogeneous graph attention network of the present invention;

[0042] Figure 2 It is a schematic diagram of the overall network structure of the present invention;

[0043] Figure 3 The clustering visualization effects of the present invention and the latest method on two data sets are compared.

[0044] Figure 4 It is the convergence curve in the training process of the present invention. DETAILED DESCRIPTION

[0045] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.

[0046] In order to solve the problems existing in the background technology, the embodiment of the present invention proposes a missing multi-view clustering method based on an integrated heterogeneous graph attention network, the highlight of which is that the model can flexibly utilize the complementarity between views and the structural information between samples, so as to learn a more structured implicit representation. In different missing situations, good clustering effects can be achieved.

[0047] Example 1

[0048] In the missing multi-view clustering method based on structured implicit representation learning proposed in this embodiment, the method includes the following steps: Figure 1 The following steps are shown:

[0049] 101: Construct a reconstruction network and learn a common latent space representation. The reconstruction network is implemented by a group of multi-way neural networks. The common latent space is linearly or nonlinearly mapped through a multi-view multi-way neural network, so that the common latent space representation can be reconstructed to obtain data from multiple perspectives through linear or nonlinear mapping. In this step, the multi-way neural network model is implemented by a multi-layer perceptron. In this embodiment, the number of layers of the multi-layer perceptron is between 1-3. The input of each multi-view multi-way neural network model is the latent space representation, and the corresponding output is the data of the corresponding view reconstructed by the latent space representation; the reconstruction loss function represents the degree of approximation between the common latent space and the missing multi-view training sample data.

[0050] In this embodiment, the reconstruction loss function is used to constrain the reconstruction output represented by the latent space to be as close as possible to the real multi-view data. The reconstruction loss function is defined as the square of the mean of the Euclidean distance between the real multi-view data and the reconstructed multi-view data, where the reconstruction loss is not calculated for the missing view data. The reconstruction loss function is:

[0051]

[0052] in, Is The parameterized reconstruction network for the vth view, is the nth sample of the vth view, N and V represent the number of samples and views respectively, s nv Flag indicating whether the v-th view in the n-th sample is missing.

[0053] 102: Construct a k-nearest neighbor graph and a missing pattern graph to obtain an integrated heterogeneous graph.

[0054] The k-nearest neighbor graph is used to mine the structural relationship between samples, which is measured by the similarity between samples. In this embodiment, the sample similarity is measured by the Gaussian kernel distance. For each sample, the k samples closest to it are selected as adjacent nodes.

[0055] The missing pattern graph is used to mine the structural information of missing views. In this embodiment, the missing pattern graph is constructed according to the missing conditions of the views.

[0056] By combining the k-nearest neighbor graph and the missing pattern graph, an integrated heterogeneous graph can be obtained, which represents the interaction relationship between samples under different missing patterns.

[0057] 103: Construct a graph attention network. In this example, the graph attention mechanism can be defined as:

[0058]

[0059] Among them, T is the transposition and ‖ is the connection operation. Finally, by concatenating the outputs of multiple attention mechanisms, we can get the updated node features:

[0060]

[0061] Among them, σ is the activation function, is the normalized attention coefficient calculated for the mth attention mechanism, M is the number of attention heads, and W m It is the linear transformation weight matrix of the input features under the mth attention mechanism.

[0062] 104: Using probability distribution to constrain the consistency between views. In the example, we use a two-step alternating unsupervised algorithm to improve clustering. In the first step, we calculated the soft assignment between each sample point and the cluster centroid in the latent representation. In this example, we used t distribution as the kernel to measure the similarity between the data representation and the cluster center vector. Each cluster center is initialized by K-means. By minimizing the KL divergence loss between the initial distribution and the target distribution, the network can be promoted to learn a structured latent space representation that is more suitable for clustering tasks, that is, the data representation around the cluster center is closer, thus having better clustering performance.

[0063] In the second step, we update the structured latent space representation and refine the cluster centroids by learning from the current high-confidence assignment using an auxiliary target distribution. This process is repeated until convergence is reached.

[0064] The detailed training steps of the network are as follows:

[0065] Step 1: Input the number of iterations T and the number of nearest neighbors k. Randomly initialize the public hidden representation Reconstructing network parameters Graph Attention Mechanism Parameters Initialize the current number of iterations iter;

[0066] Step 2: Repeat steps 3 and 4 until the number of iterations reaches T;

[0067] Step 3: Use the formula L r Calculate the reconstruction loss using the formula L c Calculate clustering loss;

[0068] Step 4: Use SGD (Stochastic Gradient Descent) algorithm to update alternately and Update iter=iter+1.

[0069] In summary, the present invention proposes a missing multi-view clustering method based on structured latent representation learning. The method first learns a common latent space representation for the available views. In order to mine the interactive information between samples in the latent space, a k-nearest neighbor graph is constructed based on the latent space representation. Then, a set of view-specific missing pattern graphs are constructed for the missing conditions of different views. Then, the similarity graph is fused with the missing pattern graph of a specific view to construct an integrated heterogeneous graph to mine the interactive relationship between samples under different missing patterns. In order to obtain a more structured representation, the graph attention mechanism is used to learn the attention of the nodes in each missing mode, so that the features of nodes of the same type are closer. Finally, by embedding the consistency of the probability distribution into the network, the probability distribution of the latent space representation corresponding to each view is constrained, and a consistent distribution is learned for clustering.

[0070] Example 2

[0071] In the specific implementation process, the multi-view clustering method mentioned above is applied to carry out simulation experiments.

[0072] 1. Dataset

[0073] Five public multi-view datasets are used in our experiments, and a brief description of each dataset is summarized as follows:

[0074] (1) ORL dataset: It consists of 400 face images from 40 different categories, with 10 images in each category. For each image, three features, LBP, Gabor, and Intensity, are extracted respectively.

[0075] (2) 3Sources dataset: 3Sources is a well-known multi-view text dataset. It collects 416 different news from three online news sources: BBC, Reuters, and The Guardian, covering 6 different topics. Among them, 169 news were reported by three news sources at the same time. The article from each news source can be regarded as a view. In the experiments of this paper, 169 news reported by three news websites at the same time were selected, and features of 3560 dimensions, 3631 dimensions, and 3068 dimensions were extracted respectively.

[0076] (3) Caltech101-7 dataset: The Caltech101 dataset is an image recognition dataset consisting of 9146 images in 101 categories. Caltech101-7 is a subset of the Caltech101 dataset. It contains 1474 images in 7 categories, including Garfield, Motorcycle, Stop Sign, Snoopy, Dollar, Expression, and Windsor Chair. Six features are extracted from each image: Gabor, Wavelet, CENTRIST, HOG, GIST, and LBP.

[0077] (4) COIL20 dataset: This dataset is a commonly used image dataset, containing 1440 images from 20 categories, with 72 images in each category. For each image, three different features are extracted, namely intensity, LBP, and Gabor, with corresponding feature dimensions of 1024, 3304, and 6750, respectively.

[0078] (5) Handwritten dataset: This dataset consists of 2000 handwritten digits (i.e., 0-9), with 200 images in each category. Six features are extracted from each image: PIX, FOU, FAC, ZER, KAR, and MOR.

[0079] 2. Comparison method

[0080] We compared the performance of IHGAT with the following methods:

[0081] (1) BSV: This method first fills the missing instances with the average value of the available instances in each view. Then, K-means is used to cluster the filled views. For each dataset, the view with the best clustering result is selected.

[0082] (2) Concat: Similar to BSV, Concat first fills the missing views with the average value of the available instances of each view. However, this method concatenates all the filled views into a feature vector and then uses K-means for clustering.

[0083] (3) MultiNMF: MultiNMF first fills the missing instances with the average of the available instances in each view and learns a non-negative low-dimensional consistent representation for all views. K-means is then performed on the learned consistent representation to obtain the clustering results.

[0084] (4) MIC: This method is based on L2,1-norm regularized weighted non-negative matrix factorization to learn consistent representations for all views, where instance missing information is constrained as weights to avoid the negative impact of view missingness.

[0085] (5) OMVC: This method designs a weighted non-negative matrix factorization framework to learn consistent representations of missing views.

[0086] (6) DAIMC: This method utilizes weighted matrix factorization based on instance information alignment and sparse regression based on basis matrix alignment to learn consistent representations for all views.

[0087] (7)UEAF: This method uses the error matrix of graph regularization constraints to simulate missing instances, thereby learning a low-dimensional consistent representation of all views, and finally performs K-means on the consistent representation to obtain the clustering result.

[0088] 3. Experimental Setup

[0089] To generate missing multi-view datasets, we randomly delete p% (p∈{10,30,50}) instances from each view while ensuring that all samples have at least one view. Then, each missing multi-view data is randomly divided into 80% for training and 20% for testing. For the methods proposed in this chapter, the range of hyperparameter λ is {0.0001, 0.001, 0.01, 0.1, 1, 10, 100, 1000}, the range of k is k∈{3, 15}, and the default value of k is 10 for most cases. The dimension D of the latent space representation is set to {16, 32, 64, 128, 256}, and the default value is 64. Due to the randomness caused by data partitioning, each method is run 10 times and the average of all indicators is calculated. To compare the actual performance of each algorithm, we use clustering accuracy (ACC) and normalized mutual information (NMI) to evaluate the clustering effect of these algorithms on missing multi-view data.

[0090] 4. Experiment and result analysis

[0091] We evaluate the proposed method on five multi-view datasets. Table 1 shows the clustering results of different methods.

[0092] Table 1: Comparison of clustering performance of different methods on five multi-view datasets.

[0093]

[0094]

[0095] According to the experimental results in Table 1, the following conclusions can be drawn:

[0096] (1) Compared with other methods, BSV and Concat have the worst clustering effect in most cases. This shows that simply filling the missing views with the average value of available instances is difficult to achieve ideal results and lacks rationality. By utilizing the complementary information of multiple views, MIC, OMVC, DAIMC, UEAF and IHGAT can achieve better clustering performance in most cases.

[0097] (2) Compared with the comparison methods, our method achieves the best results on all datasets. For example, on the ORL and COL20 datasets, when the missing rate is 0.5, IHGAT improves ACC by 12.18% and 7.91% respectively, and improves NMI by 13.48% and 12.59% respectively compared with the best comparison methods. This shows that considering the structural information of missing data and the available information of non-missing data at the same time to learn a more compact common representation is beneficial for missing multi-view clustering.

[0098] (3) As the missing view rate increases, the clustering effect of all methods decreases. For example, on the ORL dataset, when the missing view rate changes from 0.1 to 0.5, the ACC of DAIMC and IHGAT decreases by 12.25% and 13.13%, respectively, and the NMI decreases by 10.35% and 5.6%, respectively. This shows that the missing view seriously affects the mining of multi-view data information.

[0099] In order to more intuitively demonstrate the advantages of the proposed method, we use t-SNE to visualize the consistent representations learned by different methods. Figure 3 The visualization results of different methods on the Handwritten (HW) and COIL20 datasets when the missing rate is 0.3 are shown. From left to right are the results of MIC, OMVC, DAMIC, and IHGAT. It can be clearly seen that the structured latent representation learned by this method is better in structure, very compact within the class, and has a larger distance between classes, which has a better clustering effect.

[0100] Figure 4 The convergence of the present invention on four datasets (a) Handwritten, (b) COIL20, (c) ORL and (d) Caltech7 is shown. From the convergence curve, it can be seen that on the four datasets, as the number of iterations increases, the objective function value gradually decreases and converges to a stable point within 50 iterations, indicating that the proposed optimization method has good convergence.

[0101] Obviously, the above embodiments of the present invention are only examples for clearly explaining the present invention, and are not intended to limit the implementation methods of the present invention. At the same time, for those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A missing multi-view clustering method based on integrated heterogeneous graph attention network, characterized in that: The method comprises the following steps: Step 1: learn a common latent space representation for all available views; Step 2: construct a k-nearest neighbor graph based on the common latent representation learned in step 1, and construct a missing pattern graph for each view. Combining the k-nearest neighbor graph and the missing pattern graph, we can get an integrated heterogeneous graph that represents the proximity relationship between samples under different missing patterns. Step 3: Based on the heterogeneous graph obtained in step 2, the graph attention mechanism is used to strengthen the interaction between samples and learn a set of structured latent representations; Step 4: Based on the structured latent representation learned in step 3, a consistent distribution is learned for clustering using a consistent distribution constraint. The consistent distribution constraint learning step includes: For the vth view, we use Student's t-distribution as the kernel to measure the node z i ′(v) and cluster center μ j The similarities between and They are the initial distribution Q (v) and the target distribution P (v) The elements in , where It can be defined as: Among them, α is the degree of freedom of Student's t-distribution, C is the number of sample categories, represents the probability of node i being assigned to node j in view v. In our experiments, the initial cluster center can be obtained by using K-means on the structured latent representation corresponding to the vth view, and the cluster centers of all views are set to the same to better follow the consistency principle. By Raising to the second power and normalizing by the frequency of each cluster yields in, is the soft assignment frequency. In order to compare the similarity of two probability distributions, the loss function is defined as the probability distribution consistency loss L c , the clustering loss is defined as minimizing the KL divergence between the original distribution and the target distribution of all views, so L c It can be defined as: Among them, KL divergence is used to measure the asymmetric difference between two probability distributions. Finally, take The mean of is taken as the final consistent distribution and used for clustering.

2. The missing multi-view clustering method based on integrated heterogeneous graph attention network according to claim 1, characterized in that: In step 1, the reconstruction loss function of the public latent space representation is: Among them, L r The reconstruction loss aims to learn the mapping between the original data and the common latent representation. Is The parameterized reconstruction network for the vth view, is the nth sample of the vth view, N and V represent the number of samples and views respectively, s nv Flag indicating whether the v-th view in the n-th sample is missing.

3. The missing multi-view clustering method based on structured representation learning according to claim 1, characterized in that: In step 2, a k-nearest neighbor graph G is constructed based on the common latent representation. n ∈R N×N , and its expression formula is as follows: Among them, N k (h i ) and N k (h j ) represent the k nearest neighbor samples of sample i and sample j respectively.

4. The missing multi-view clustering method based on integrated heterogeneous graph attention network according to claim 1, characterized in that: In step 2, a missing pattern graph is constructed for each view. It can be calculated as follows: in, and denote the i-th and j-th samples in the v-th view respectively.

5. The missing multi-view clustering method based on integrated heterogeneous graph attention network according to claim 1, characterized in that: In step 2, by combining the k nearest neighbor graph and the missing pattern graph, a latent space heterogeneous graph can be obtained, which is defined as follows:

6. The missing multi-view clustering method based on integrated heterogeneous graph attention network according to claim 1, characterized in that: In step 3, the graph attention mechanism can be defined as: Among them, h n is the public implicit representation, G v is the adjacency graph matrix, is the learnable parameter of the graph attention mechanism, By learning the graph attention mechanism, we can obtain a set of structured latent representations:

Images