Multi-view data clustering method based on neighborhood correction and VMF hybrid model

Abstract

The invention discloses a multi-view data clustering method based on a neighborhood correction and VMF hybrid model. The method comprises the following steps: firstly, collecting a data set containing a multi-view sample; secondly, multi-view data consistency correction is extracted by adopting a cross-view consistency loss function; then, a feature correction module based on neighborhood driving is built, correction of consistency features is completed, fusion processing is carried out on all view features after correction, and a visual view is generated; then, a von Mises-Fisher (VMF) hybrid model is adopted to carry out clustering distribution modeling on the target view and the feature views respectively; and finally, quantifying the similarity of the VMF mixed distribution corresponding to each view through a probability product kernel function, designing a cross-view distribution alignment constraint, aligning the clustering distribution of the same sample in different views, and finally outputting a high-precision clustering label. The method provided by the invention is verified by a plurality of public standard data sets, has remarkable effectiveness, and can generate a clustering structure with high intra-class compactness and excellent inter-class separation degree. Compared with an existing multi-view clustering technical scheme, the method has the advantage that the accuracy and reliability of a clustering result are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of data mining and machine learning, specifically a multi-view data clustering method based on a neighborhood correction and VMF hybrid model. It aims to achieve effective clustering analysis of high-dimensional, multi-source, and multi-modal data by fusing multi-view features and combining spatial neighborhood information, thereby improving the accuracy and robustness of clustering. Background Technology

[0002] Multi-view clustering, a core technology in unsupervised learning, aims to fully explore complementary and consistent information from multiple data source views without relying on label information, thereby achieving accurate sample grouping. With the rapid development of data acquisition and storage technologies, real-world data often possesses multi-view characteristics. For example, image data includes views such as color and texture, while social network data covers dimensions such as social relationships and behavioral habits. This type of multi-view data has significant application value in fields such as data mining, image segmentation, and recommendation systems.

[0003] In recent years, deep learning technology has been widely used in multi-view clustering due to its powerful feature representation capabilities. By jointly training multi-view encoders, it takes into account both shared information and differences between views, significantly improving clustering results. However, existing deep multi-view clustering methods still have significant limitations: most methods overemphasize feature representation and fusion, neglecting the distribution characteristics within views, making it difficult to fully explore the potential structural information of each view; at the same time, multi-view data often suffers from inconsistent feature distribution. Unreasonable feature distribution within a single view or excessive distribution deviation between views can lead to cluster structure degradation, resulting in problems such as scattered samples within a cluster and blurred boundaries between clusters, seriously affecting clustering performance. Summary of the Invention

[0004] This invention proposes a multi-view data clustering method based on a hybrid model of neighborhood correction and VMF. This method designs a view... Figure 1 A robust feature extraction module is used to obtain robust features for each view. A neighborhood nonparametric feature correction module optimizes the local structural representation of features, improving intra-class compactness. Simultaneously, a probability product kernel function based on the VMF distribution is introduced to measure the distribution similarity between different views. Cross-view distribution alignment constraints are designed to achieve effective alignment of the multi-view distribution structure, thereby improving the overall clustering performance. Validation on multiple public datasets shows that this method can generate compact and well-separated clustering results.

[0005] 1. A multi-view data clustering method based on a hybrid model of neighborhood correction and VMF, characterized by comprising the following steps: S1: Collect a multi-view public dataset, input the data of each view into a symmetric structure autoencoder; extract the latent feature representation of each view through the encoder, and then reconstruct the output through the decoder to maintain the integrity of the feature representation; S2: Input the latent features obtained in S1 into the feature encoder, and output high-dimensional feature representations for each view; construct and compute cross-view... Figure 1 The consistency loss is used to optimize the model parameters, and a preliminary consistency feature representation of each view is obtained. S3: Construct a neighborhood-driven feature correction module. Through the feature correction module, the initial consistent features of each view are guided to gather into the high-density feature region to obtain the corrected features of each view. S4: The corrected features of each view obtained in step S3 are fused to obtain a unified fused feature representation; the fused feature representation and the corrected feature representation of each view in step S3 are respectively input into the VMF hybrid distribution model, and the clustering distribution model is completed through the VMF hybrid distribution model to obtain the preliminary clustering distribution; S5: Measure the similarity of VMF mixture distributions by using the probability product kernel function, construct distribution consistency constraints across views, and align the VMF mixture distributions of the same sample in different views.

[0006] S6: Iterative optimization of cross-view Figure 1 Consistency loss, VMF mixed distribution model and cross-view distribution consistency constraints are used to finally output high-precision clustering labels; In one embodiment, S1 further includes a specific implementation of the edge detection module. The encoder expression can be represented as:

[0007] The decoder expression can be represented as:

[0008] in For the original sample, For the restored sample, reconstruct the loss expression. for:

[0009] in, This represents the total number of samples in the view. The optimization objective at this stage comprehensively considers consistency feature learning and reconstruction constraints, thereby obtaining a discriminative and consistent representation while preserving the informational capacity of the original features of the view.

[0010] In one embodiment, the expression for the feature encoder in step S2 can be represented as:

[0011] in, This represents the trainable parameters; MLP stands for Multilayer Perceptron.

[0012] The similarity function is defined as the cosine similarity expression:

[0013] Where a and b are the sample vectors for which similarity needs to be calculated. Based on the constructed positive and negative sample pairs, cross-view Figure 1 The consistency loss function can be expressed as:

[0014] in The temperature coefficient is fixed at 0.5, used to adjust the gradient magnitude, and N represents the number of samples.

[0015] In one embodiment, the corrected feature representation is obtained in step S3:

[0016] Among them, the one with the highest cosine similarity is selected. The expression for a sample as its neighborhood can be: )) in Fixed at 0.5, It equals the number of clusters.

[0017] In one embodiment, the first step S4 is obtained Fusion features of individual samples:

[0018] The VMF hybrid distribution model is used, and its form is as follows:

[0019] Where μ is the cluster center direction vector; k is the concentration parameter; and m represents the format where the number of density centers equals the number of cluster types. It is the normalization constant, where Let d represent the first kind of modified Bessel function, and d represent the dimension.

[0020] Based on the VMF mixture model, the i-th sample The probability of belonging to the m-th cluster in the v-th view can be expressed as:

[0021] in The VMF component mixing coefficients are estimated independently for each view. Let represent the mean direction vector of the m-th cluster in the v-th view. This represents the concentration parameter of the m-th cluster in the v-th view. In the VMF mixture model, u and k are model parameters. The maximum likelihood estimation function for the VMF mixture model is:

[0022] In one embodiment, the probability kernel alignment constraint in step S5 can be expressed as:

[0023] The probability kernel similarity between different views can be expressed as:

[0024] Where D represents the feature dimension; , representing the mean vector component of the d-th dimension; (k represents the concentration degree on the d-th dimension). This represents the feature representation of the i-th sample after all views have been fused. This represents the modified feature of the i-th sample in the v-th view.

[0025] In one embodiment, the objective function for jointly optimizing the VMF clustering models on all views using a joint maximum likelihood estimation framework in S5 can be expressed as:

[0026] λ is used to control the degree of effect of cross-view distribution alignment constraints.

[0027] In one embodiment, the acquisition of cluster labels in step S6 involves the final clustering result of each sample being determined by the fused posterior distribution Z, obtained by selecting the cluster label with the highest posterior probability, expressed as:

[0028] in, Let be the final cluster label for the i-th sample. Let be the posterior probability that the i-th sample belongs to the m-th cluster.

[0029] As can be seen from the above technical solution, the present invention has the following beneficial effects: By designing a domain feature-based correction module, nonparametric optimization of each sample feature is achieved, causing features to cluster towards the density peak direction. This enhances the local structure representation ability during clustering and effectively reduces the negative impact of inconsistent feature distributions between views. Addressing the issue of large differences in multi-view data distribution, this invention proposes a clustering method based on a VMF hybrid model. This method fully utilizes the directional characteristics of the VMF distribution on a unit sphere to reasonably characterize the distribution features of data in different views. Simultaneously, a probability product kernel function is used to measure and align the similarity of VMF distributions in different views, and cross-view distribution consistency constraints are designed, effectively improving the joint modeling capability of multi-view data. 3. Figure Descriptions Figure 1 This is a schematic diagram of the overall process of the method of the present invention; Figure 2 This is a model diagram of the multi-view data clustering method based on a hybrid model of neighborhood correction and VMF in this invention; Figure 3 Visualization of T-SNE for fused features on the hdigit dataset as the training epochs increase. Detailed Implementation

[0030] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0031] Example 1 S1: The autoencoder in this implementation consists of an encoder and a decoder, both of which use a sequence network structure built with fully connected layers (FC), and the activation function is uniformly ReLU. The specific network parameter configuration is as follows: the encoder's network structure progresses sequentially in the dimension sequence "FC500→FC500→FC2000→FC256", where the final output FC256 corresponds to the latent feature dimension; the decoder adopts a network structure that is completely symmetrical to the encoder.

[0032] S2: Input the latent features of each view obtained in step S1 into a multilayer perceptron (MLP) to map the features to a high-dimensional space, with ReLU as the activation function; then construct positive and negative sample pairs, specifically: define the high-dimensional features of the same sample in any two different views as a positive sample pair, and define any combination of high-dimensional features of different samples as a negative sample pair; through positive and negative sample pairs, achieve consistent alignment of the same type of sample features between different views, and finally obtain the preliminary consistent feature representation of each view.

[0033] S3: Feature correction, guiding features to cluster towards high-density regions. The specific steps are as follows: S31) For the initial consistency features of samples in each view, calculate its characteristics compared with those of all other samples in the same view. Using the cosine similarity, construct a neighborhood similarity matrix; S32) Select the sample with the highest similarity. Each sample is considered as its neighborhood, denoted as .

[0034] S33) Based on the neighborhood selection, the following feature correction formula is designed to optimize the features:

[0035] In the formula ; This represents the number of clusters.

[0036] S4: Feature fusion and acquisition of the preliminary clustering distribution process are as follows: S41) Feature fusion adopts an average fusion strategy. For the i-th sample, the fused features are... The formula for calculation is:

[0037] S42) Construct a VMF hybrid model, integrating features Features after correction of each view In a VMF mixture model, each view corresponds to an independent VMF mixture distribution component. The probability density function of the VMF mixture model is defined as:

[0038] in Let v be the direction vector of the m-th cluster of the v-th view. For the distribution concentration parameter (The larger the value, the stronger the feature clustering). The normalization factor is expressed as:

[0039] For feature dimension, It is a first-order modified Bessel function, and its numerical stability is ensured by calculation using the MPMath high-precision mathematical library.

[0040] S43) Based on the VMF hybrid model, the first Sample In the The view belongs to the first The probability of a cluster can be expressed as:

[0041] The maximum likelihood estimation function for the S44)VMF mixture model is:

[0042] The specific process of distribution alignment and clustering optimization in S5 is as follows: S51) Introducing the probability product kernel function to measure the similarity between the fused feature distribution and the feature distributions of each view, the kernel function is defined as:

[0043] Where D represents the feature dimension; : Represents the mean vector component of the d-th dimension; : represents the concentration k on the d-th dimension; : The feature representation of the i-th sample after fusing all views; This represents the corrected feature representation of the i-th sample in the v-th view.

[0044] S52) Design the cross-view distribution consistency constraint loss, the expression of which is:

[0045] S6: The steps involve constructing a model that includes reconstruction loss and cross-view loss. Figure 1 The combined loss function of consistency loss, fusion clustering similarity term, and cross-view distribution alignment constraint loss The Adam optimizer was used for iterative training with a learning rate of 0.0001. The training process consisted of two phases: first, each encoder was trained independently for 200 iterations. Then, in the fine-tuning phase, the entire network framework was jointly trained for 100 epochs, with the temperature parameter τ set to 0.5. Continuous optimization across vision was performed. Figure 1 The model is saved after the consistency loss parameter, VMF mixture distribution model parameter and cross-view distribution consistency constraint related parameter are processed until the model converges or reaches the preset number of training rounds. Experimental results: Tables 1–3 systematically compare the proposed method with 11 mainstream algorithms in the field of multi-view clustering (including K-means, PLCMF, LMVSC, FPMVS-CAG, FastMICE, DSMVC, SiMVC, CoMVC, MFLVC, GCFAggMVC, and CAMVC) on three core clustering evaluation metrics: ACC (clustering accuracy), NMI (normalized mutual information), and PUR (clustering purity). Experimental results show that the proposed method consistently achieves optimal performance on all metrics and all datasets, demonstrating a significant comprehensive advantage over the 11 mainstream comparison algorithms. In the ACC metric, the proposed method achieves scores of 0.4047, 0.9947, and 0.7278 on the CCV, Hdigit, and Prokaryotic datasets, respectively, representing a maximum improvement of 19.1% compared to the suboptimal algorithm. In the NMI metric test, the proposed method leads the best comparison algorithm by 14.4% with a score of 0.7813 on the Voc dataset and reaches an exceptionally high mutual information value of 0.9792 on the Digit-Product dataset. Under the PUR metric validation, the proposed method achieves the best purity of 0.7720 and 0.8164 on the Prokaryotic and Voc datasets, respectively, significantly outperforming the comparison algorithm. These three metrics demonstrate that the proposed method exhibits stable clustering performance in both heterogeneous and homogeneous view data, and its cross-view... Figure 1 The iterative optimization of consistency loss and the design of the VMF hybrid distribution model effectively improved the clustering accuracy and generalization ability, verifying the technical superiority of the proposed method. Figure 3 The Hdigit clustering process during the fine-tuning phase was visualized, from a random mixed distribution in Epoch 0, to the initial clustering of multiple clusters starting in Epoch 25, to the gradually clearer boundaries of the clusters in Epoch 50, and finally to the clear separation of various clusters in Epoch 100. This process intuitively demonstrates that the algorithm achieves effective clustering of data step by step through iterative optimization, verifying the convergence and stability of the cluster structure during model training.

[0046] Table 1

[0047] Table 2

[0048] Table 3

Claims

1. A multi-view data clustering method based on a hybrid model of neighborhood correction and VMF, characterized in that, Includes the following steps: S1: Extract the latent feature representations of each view and input the multi-view data into the symmetric structure autoencoder respectively; S2: Design a cross-view consistency loss function to obtain a preliminary consistency feature representation of each view; S3: Construct a neighborhood-driven feature correction module. Through the feature correction module, the initial consistent features of each view are guided to gather into the high-density feature region to obtain the corrected features of each view. S4: The VMF hybrid model is used to model the multi-view. First, the features of each corrected view are fused to form a new fused view. Then, the fused feature representation and the corrected feature representation of each single view obtained in step S3 are input into the VMF hybrid distribution model. The VMF hybrid distribution model is used to complete the cluster distribution model and obtain the preliminary cluster distribution. S5: Introduce a probability product kernel function to measure the similarity of VMF mixture distributions, construct distribution consistency constraints across views, and optimize the clustering distribution of fused views; S6: By iteratively optimizing the cross-view consistency loss, combining the modeling optimization of the VMF hybrid distribution model with the conditional strengthening of the cross-view distribution consistency constraint, high-precision clustering labels are finally generated.

2. The multi-view data clustering method based on a hybrid model of neighborhood correction and VMF as described in claim 1, characterized in that, The steps to construct the cross-view consistency loss function include: The latent features of each view obtained in S21 are mapped to a high-dimensional space, as shown in the following expression: ； in This represents the i-th sample in the v-th view. The trainable parameters of the high-level encoder of view v are represented, and MLP stands for Multilayer Perceptron Network. This represents the autoencoder in S1. S22) The cross-view consistency loss function expression is as follows: ； Where N represents the number of samples. The temperature coefficient is fixed at 0.5, used to adjust the gradient magnitude, and sim() represents the cosine similarity.

3. The multi-view data clustering method based on a hybrid model of neighborhood correction and VMF as described in claim 1, characterized in that, The corrected features obtained in step S3 are represented as follows: ； in This represents the i-th sample after correction. The uncorrected i-th sample, Fixed at 0.5, is the number of clusters; z represents the features before correction. The set of neighborhood samples of sample i is represented as: 。 4. The multi-view data clustering method based on a hybrid model of neighborhood correction and VMF as described in claim 1, characterized in that, Step S4, which involves modeling the cluster distribution using the VMF hybrid distribution model to obtain the preliminary cluster distribution, includes the following steps: S41) Modeling is performed using a VMF mixture distribution, and its form is as follows: ； Where μ is the cluster center direction vector; k is the concentration parameter; and m represents the format where the number of density centers equals the number of cluster types. It is the normalization constant, where This represents a modified Bessel function of the first kind, where d represents the dimension; S42) Based on the VMF mixture model, the i-th sample The probability of belonging to the m-th cluster in the v-th view can be expressed as: ； in The VMF component mixing coefficients are estimated independently for each view. Let represent the mean direction vector of the m-th cluster in the v-th view. This represents the concentration parameter of the m-th cluster in the v-th view. In the VMF mixture model, u and k are model parameters; S43) The joint maximum likelihood estimation function for optimizing the VMF model is: 。 5. The multi-view data clustering method based on a hybrid model of neighborhood correction and VMF as described in claim 1, characterized in that, Step S5 constructs the distribution consistency constraint across views, expressed as follows: ； in This is used to measure the distribution similarity between views, thereby achieving cross-view structural alignment. For the difference in distribution structure between sample i in the merged view and the v-th view, a probability product kernel similarity function of the following form is constructed: ； Where D represents the feature dimension; Let represent the mean vector component of the d-th dimension; Indicates the first Concentration degree k on dimension). The feature representation of the i-th sample after fusing all views; The corrected feature representation of the i-th sample in the v-th view.

Images