Multi-view clustering method for uncertainty perception

The uncertainty-aware multi-view clustering method addresses data quality differences across perspectives by estimating sample uncertainty and fusing cluster distributions, resulting in robust clustering results that enhance recommendation system accuracy and user satisfaction.

CN119066451APending Publication Date: 2024-12-03TIANJIN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411195744.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing multi-view clustering methods fail to effectively address the differences in data quality across various perspectives due to unequal treatment of data from different viewpoints, limiting the performance of clustering tasks such as recommendation systems.

Method used

An uncertainty-aware multi-view clustering method that estimates sample uncertainty in an unsupervised manner by measuring the consistency of cluster distributions using a graph-based strategy, performs weighted fusion of view-specific features, and fuses cluster distributions using the Dempster combination rule to obtain robust clustering results.

Benefits of technology

Improves the accuracy of recommendations in recommendation systems by providing content that meets user needs through credible multi-view clustering, enhancing user satisfaction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066451A_ABST
    Figure CN119066451A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-view clustering method for uncertainty perception. The method comprises the following steps: constructing a graph auto-encoder for each view based on a multi-view feature extraction module, wherein the graph auto-encoder is used for learning specific features of each view; based on the uncertainty estimation strategy of the graph, calculating the entropy of the clustering distribution between adjacent samples to measure the consistency of the clustering distribution of each sample and the adjacent samples in the graph, and estimating the uncertainty of each sample in each view angle; performing weighted fusion on the visual angle specific features of the sample by using uncertainty to obtain discriminative multi-visual angle common features; according to the method, clustering distribution of different perspectives is fused based on uncertainty, so that a robust clustering result is obtained, and a recommendation system can provide content meeting requirements for a user based on the clustering result. The method can be applied to a recommendation system to improve the recommendation quality, so that the user experience is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of multi-view, and in particular to an uncertainty-aware multi-view clustering method. Background Art

[0002] With the development of information collection technology, data in practical applications usually have various forms of expression. For example, the same object or content can be comprehensively represented by data in different forms such as text, image, audio, etc. These multiple forms of data are called multi-view data. In multi-view data, data from different perspectives describe the same object and therefore have consistent semantic information. At the same time, since multi-view data reflects the different characteristics of objects or content from different aspects, data from different perspectives are also complementary. Multi-view clustering aims to comprehensively utilize information from different perspectives to achieve classification of a large amount of unlabeled data, and has important application value in many fields such as social media analysis, recommendation systems, and medical diagnosis. However, there are complex correlations in multi-view data in reality, which makes multi-view clustering still a challenging task.

[0003] In recent years, thanks to the ability of graph structures to model the relationship between data, researchers have proposed a series of graph-based multi-view clustering methods. Among them, Wang et al. designed a multi-graph attention fusion encoder structure, which introduced a self-attention mechanism to mine the correlation between samples to learn robust multi-view features for clustering. Xia et al. proposed a self-supervised graph convolutional network to capture the common subspace representation of multi-view data, and applied a spectral clustering algorithm to obtain the final clustering results. However, due to the influence of sensor differences or environmental factors, the quality of data under different perspectives varies. Existing methods usually treat data from different perspectives equally, ignoring the differences in data quality under different perspectives, which limits the further improvement of clustering performance. At present, uncertainty learning, as a way to model data quality, has achieved remarkable results in many visual tasks, but existing methods often rely on task-related labels to provide prior knowledge for uncertainty learning. In the multi-view clustering task, effectively evaluating sample uncertainty in an unsupervised manner helps to alleviate the negative impact of data quality differences from different perspectives, thereby obtaining robust clustering results. Based on the clustering results, it can be applied to recommendation systems to effectively improve the accuracy of recommendations and user satisfaction. Therefore, exploring uncertainty-aware multi-view graph clustering methods has important research significance. Summary of the invention

[0004] The present invention provides an uncertainty-aware multi-view clustering method. The present invention can estimate the uncertainty of different samples at different viewpoints in an unsupervised manner, and finally realizes credible multi-view clustering by modeling the sample data quality at different viewpoints and learning discriminative multi-view common features and robust clustering distribution under the guidance of uncertainty. The method can be applied in the recommendation system to improve the quality of recommendation, thereby improving the user experience. See the following description for details:

[0005] An uncertainty-aware multi-view clustering method, the method comprising:

[0006] Based on the multi-view feature extraction module, a graph autoencoder is constructed for each view to learn the specific features of each view;

[0007] Based on the uncertainty estimation strategy of the graph, the consistency of the cluster distribution of each sample in the graph with its adjacent samples is measured to estimate the uncertainty of each sample at each perspective;

[0008] Uncertainty is used to perform weighted fusion on the view-specific features of the samples to obtain discriminative multi-view common features;

[0009] Based on uncertainty, clustering distributions from different perspectives are fused to obtain robust clustering results. Based on the clustering results, the recommendation system provides users with content that meets their needs.

[0010] The method further includes: using reconstruction loss to constrain the learning process of different perspective features.

[0011] Among them, the graph-based uncertainty estimation strategy measures the consistency of the cluster distribution of each sample in the graph with its adjacent samples to estimate the uncertainty of each sample at each perspective: calculate the entropy of the cluster distribution of each sample and its adjacent samples at each perspective, and obtain the uncertainty of the sample at each perspective. The formula is:

[0012]

[0013] and

[0014]

[0015] in, represents the uncertainty of the i-th sample in the v-th perspective, represents the set of the i-th sample and its adjacent samples in the v-th view, K represents the number of categories, Representing a collection The probability that sample j in is assigned to the kth cluster.

[0016] Among them, the multi-view common features are:

[0017]

[0018] Among them, z i represents the common features of the i-th sample, represents the specific features of sample i in the vth view, v is the label of the view, V is the total number of views, and L is the number of layers of the graph autoencoder.

[0019] The fusion of cluster distributions from different perspectives based on uncertainty is as follows:

[0020] Based on the Demspter combination rule, the fusion operation is expressed as:

[0021]

[0022] in, represents the set of cluster distribution and uncertainty after the fusion of the i-th sample, represents the set of cluster distribution and uncertainty of the i-th sample in the v-th perspective, Represents the distributed fusion operation. Taking the m-th and n-th perspectives as examples, the fusion operation is expressed as follows:

[0023]

[0024] Among them, Q ik and u i Respectively represent the cluster distribution and uncertainty of the ith sample after fusion, represents the parameter used to measure the amount of conflict between two distributions, is the regularization of this parameter.

[0025] The beneficial effects of the technical solution provided by the present invention are:

[0026] 1. This paper proposes an uncertainty-aware multi-view graph clustering network, which estimates the uncertainty for each sample in an unsupervised manner, effectively models the data quality of different samples at each viewpoint, thereby improving the performance of multi-view clustering;

[0027] 2. The present invention designs an uncertainty-guided common feature learning mechanism, which estimates the uncertainty of each sample by measuring the entropy of the average cluster distribution between adjacent samples, thereby dynamically learning more discriminative multi-view common features;

[0028] 3. The present invention proposes a cross-perspective credible distribution fusion module, which uses uncertainty to fully explore the complementarity of cluster distributions of different perspectives, and realizes the credible fusion of multiple perspective cluster distributions through the Dempster combination rule, thereby obtaining robust clustering results, which can improve the quality of recommendations and user satisfaction in the recommendation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Flowchart of an uncertainty-aware multi-view clustering method. DETAILED DESCRIPTION

[0030] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.

[0031] The uncertainty-aware multi-view graph clustering method provided in an embodiment of the present invention is described below by way of example.

[0032] 1. Constructing a multi-view feature extraction module

[0033] For multi-view graph data containing V views Where X (v) Represents the sample data matrix of the vth perspective, A (v) Representing the graph structure matrix of the vth perspective, the embodiment of the present invention first constructs a graph autoencoder for each perspective to learn the specific features of each perspective.

[0034] In the embodiment of the present invention, in order to encode the sample data and the graph structure into a unified representation, an L-layer graph autoencoder based on a self-attention mechanism is used for feature extraction, where for the l-th encoder layer of the v-th view, the learned feature Z (v,l) The formula is:

[0035] Z (v,l) =C (v,l) Z (v,l-1) W (v,l)

[0036] Among them, C (v,l) represents a learnable attention coefficient matrix, the calculation method of which is well known to those skilled in the art and will not be described in detail in the present embodiment. (v,l) represents the learnable weight matrix. When l = 1, Z (v,0) =X (v) After L encoder layers, the sample feature at the vth view is represented by Z (v,L) .

[0037] 2. Designing a Graph-Based Uncertainty Estimation Strategy

[0038] Since the graph structure can reflect the similarity relationship between samples, adjacent samples in the graph belong to the same category with a greater probability. Therefore, for each sample in the graph, if its cluster distribution is inconsistent with that of adjacent samples, it can be considered that the uncertainty of the sample is high. On the contrary, if its cluster distribution tends to be consistent with that of adjacent samples, it can be considered that the uncertainty of the sample is low. In order to quantitatively estimate this uncertainty in an unsupervised manner, an embodiment of the present invention designs a graph-based uncertainty estimation strategy, which estimates the uncertainty of each sample in the graph at each perspective by measuring the consistency of the cluster distribution of each sample with its adjacent samples.

[0039] First, calculate the cluster distribution Q of each perspective (v) Specifically, based on the learned view-specific features Z (v,L) , the clustering distribution Q is obtained by measuring the similarity between the features of each sample and each cluster center (v) , expressed as:

[0040]

[0041] in, It's Q (v) The elements in represent the probability of assigning the i-th sample to the k-th cluster in the v-th view. Represents Z (v,L) The feature representation of the i-th sample in μ. k and μ k′ represents a learnable cluster center representation, k and k′ represent cluster labels, τ represents the degree of freedom, and τ is set to 1 in the embodiment of the present invention.

[0042] After obtaining the cluster distribution of each viewpoint, the uncertainty of each sample is calculated by calculating the average cluster distribution of the sample and its adjacent samples The entropy is obtained and expressed as:

[0043]

[0044] and

[0045]

[0046] in, represents the uncertainty of the i-th sample in the v-th perspective, represents the set of the i-th sample and its adjacent samples in the v-th view, and K represents the number of categories. In the above formula, the average cluster distribution The larger the entropy, the higher the uncertainty of the i-th sample, and vice versa. The smaller the entropy, the lower the uncertainty of the i-th sample. In this way, the uncertainty of each sample in each perspective is effectively estimated.

[0047] 3. Constructing a common feature learning mechanism guided by uncertainty

[0048] After estimating the uncertainty of all samples at each viewing angle, the embodiment of the present invention constructs an uncertainty-guided common feature learning mechanism to obtain robust multi-view common features.

[0049] Specifically, for different samples at different viewing angles, different weight coefficients are used to weight the sample features at different viewing angles. Intuitively, the lower the sample uncertainty at a certain viewing angle, the better the feature quality of the sample at that viewing angle. Conversely, the higher the uncertainty, the worse the feature quality of the sample at that viewing angle. Therefore, the embodiment of the present invention uses uncertainty to perform weighted fusion on the view-specific features of the sample to obtain discriminative multi-view common features, and the calculation formula is as follows:

[0050]

[0051] Among them, z i Represents the common features of the i-th sample. Since uncertainty is continuously updated during the training process, the embodiment of the present invention can dynamically update the sample weights in the feature fusion process under the guidance of uncertainty, thereby making full use of the complementarity of multi-view data to obtain discriminative multi-view common features.

[0052] 4. Design of a cross-perspective trusted distribution fusion module

[0053] Since the sample quality of different perspectives is different, different perspectives usually present cluster structures with different degradation degrees. In order to obtain a more reliable cluster distribution, the embodiment of the present invention proposes a cross-perspective credible distribution fusion module, which uses the estimated uncertainty to achieve credible fusion of cluster distributions of different perspectives.

[0054] First, in order to effectively learn the cluster distribution of each perspective, the embodiment of the present invention introduces a distribution optimization strategy. Specifically, based on the learned high-quality common features z i , generating a joint clustering distribution shared by all perspectives The formula is:

[0055]

[0056] in, is the joint cluster distribution The elements in represent the probability of assigning the i-th sample to the k-th cluster. Based on the joint cluster distribution Generate a target distribution with high confidence To constrain the learning of clustering distributions from different perspectives. Specifically, the target distribution Elements in Calculated by the following formula:

[0057]

[0058] In the network learning process, the target distribution Dynamic updates are performed to better optimize the learning of cluster distributions from different perspectives. The distribution optimization loss is defined as:

[0059]

[0060] Among them, ||·|| F Denotes the Frobenius norm. In order to obtain a credible cluster distribution of multi-view data, the estimated uncertainty is used to fuse the cluster distributions of different views.

[0061] Specifically, based on the Demspter combination rule, the fusion operation is expressed as follows:

[0062]

[0063] in, represents the set of cluster distribution and uncertainty after the fusion of the i-th sample, represents the set of clustering distribution and uncertainty of the i-th sample in the v-th perspective. Represents the distributed fusion operation. Taking the m-th and n-th perspectives as examples, the fusion operation is expressed as follows:

[0064]

[0065] Among them, Q ik and u i They represent the clustering distribution and uncertainty after fusion of the i-th sample respectively. represents the parameter used to measure the amount of conflict between two distributions, is the regularization of this parameter. Through the above distribution fusion operation, the cluster distribution information of low uncertainty samples is effectively retained, while the cluster distribution information of high uncertainty samples is suppressed, thereby obtaining a more reliable clustering result.

[0066] Specifically, the clustering results of each sample are obtained by the following formula:

[0067]

[0068] Among them, y iRepresents the cluster label corresponding to the i-th sample.

[0069] In the recommendation system of e-commerce platforms, this method can integrate information from multiple dimensions such as the price and brand attributes of the products browsed and purchased by users, and perform cluster analysis to more accurately explore users' potential interests and preferences. Based on the clustering results obtained by this method, the recommendation system can provide users with richer content that better meets their needs, thereby improving the accuracy of recommendations and user satisfaction.

[0070] 5. Constructing the overall objective function

[0071] In addition to the distribution optimization loss, the embodiment of the present invention also introduces the reconstruction loss L rec To constrain the learning process of different perspective features, it can be expressed as:

[0072]

[0073] in, and Respectively represent the reconstructed sample data matrix and graph structure matrix, ||·||2 represents the L2 norm, and λ1 represents a hyperparameter for balancing the data reconstruction loss and the graph structure reconstruction loss. In the embodiment of the present invention, λ1 is set to 0.01. In general, the objective function of the network is defined as:

[0074] L overall =L rec +λ2L clu

[0075] Among them, L overall represents the overall loss, and λ2 represents a hyperparameter for balancing the reconstruction loss and the distribution optimization loss. In the embodiment of the present invention, λ2 is set to 10.

[0076] Those skilled in the art will appreciate that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0077] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An uncertainty-aware multi-view clustering method, characterized in that: The method comprises: Based on the multi-view feature extraction module, a graph autoencoder is constructed for each view to learn the specific features of each view; Based on the uncertainty estimation strategy of the graph, the consistency of the cluster distribution of each sample in the graph with its adjacent samples is measured to estimate the uncertainty of each sample at each perspective; Uncertainty is used to perform weighted fusion on the view-specific features of the samples to obtain discriminative multi-view common features; Based on uncertainty, clustering distributions from different perspectives are fused to obtain robust clustering results. Based on the clustering results, the recommendation system provides users with content that meets their needs.

2. The uncertainty-aware multi-view clustering method according to claim 1, characterized in that: The method also includes: reconstructing the loss to constrain the learning process of different perspective features.

3. The uncertainty-aware multi-view clustering method according to claim 1, characterized in that: The graph-based uncertainty estimation strategy measures the consistency of the cluster distribution of each sample in the graph with its adjacent samples. The uncertainty of each sample at each perspective is estimated as follows: the entropy of the cluster distribution of each sample and its adjacent samples at each perspective is calculated to obtain the uncertainty of the sample at each perspective. The calculation formula is: and in, represents the uncertainty of the i-th sample in the v-th perspective, represents the set of the i-th sample and its adjacent samples in the v-th view, K represents the number of categories, Representing a collection The probability that sample j in is assigned to the kth cluster.

4. The uncertainty-aware multi-view clustering method according to claim 1, characterized in that: The multi-view common features are: Among them, z i represents the common features of the i-th sample, represents the specific features of sample i in the vth view, v is the label of the view, V is the total number of views, and L is the number of layers of the graph autoencoder.

5. The uncertainty-aware multi-view clustering method according to claim 1, characterized in that: The fusion of cluster distributions from different perspectives based on uncertainty is: Based on the Demspter combination rule, the fusion operation is expressed as: in, represents the set of cluster distribution and uncertainty after the fusion of the i-th sample, represents the set of cluster distribution and uncertainty of the i-th sample in the v-th perspective, Represents the distributed fusion operation. Taking the m-th and n-th perspectives as examples, the fusion operation is expressed as follows: Among them, Q ik and u i Respectively represent the cluster distribution and uncertainty of the ith sample after fusion, represents the parameter used to measure the amount of conflict between two distributions, is the regularization of this parameter.