Incompletely aligned multi-view clustering method based on structural consistency comparative learning

Abstract

The invention discloses an incomplete alignment multi-view clustering method based on structural consistency comparative learning, and belongs to the technical field of computer vision. The method comprises semantic consistency comparative learning and view consistency comparative learning, the semantic consistency comparative learning realizes implicit class alignment of samples by capturing global semantic consistency information, and the view consistency comparative learning learns consistency information between views while retaining local structure information of the views by hierarchically selecting positive samples. Therefore, an accurate sample corresponding relation is established.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to a non-fully aligned multi-view clustering method based on structural consistency contrastive learning. Background Technology

[0002] In recent years, with the rapid development of information technology, social media platforms, sensor networks, and other sources generate massive amounts of multi-source heterogeneous data every day. This type of data typically contains diverse representations of the same entity, such as images, text, and videos of the same sample; these different representations are collectively referred to as multi-view data. Multi-view clustering aims to improve clustering performance by mining the inherent consistency and complementarity information of multi-view data. Although multi-view clustering methods have made significant progress, most existing methods implicitly assume that all samples are perfectly aligned. However, in practical applications, due to the complexity and inconsistency of time and space, collecting perfectly aligned samples is extremely challenging, leading to the problem of incompletely aligned multi-view clustering.

[0003] Traditional multi-view clustering methods typically achieve good performance when processing fully aligned multi-view data by fully mining and fusing consistency and complementarity information between views. However, in incompletely aligned clustering problems, due to the missing correspondences between some samples in the views, the model struggles to accurately capture the inherent structural relationships across views, leading to a significant drop in clustering performance when processing incompletely aligned multi-view data. Therefore, the key to incompletely aligned multi-view clustering tasks lies in how to effectively construct the missing sample correspondences between views to learn cross-view clustering relationships. Figure 1 Sexual information.

[0004] Traditional non-perfectly aligned multi-view clustering methods typically employ a contrastive learning strategy, learning consistency information among views by designing a noise-robust contrastive loss function. Specifically, by selecting high-quality positive samples, the influence of noisy labels is effectively mitigated, feature quality is improved, and matching errors are reduced. Furthermore, to enhance consensus representation capabilities, an additional encoder reconstruction loss is added. This loss reconstructs each original view from the integrated common representation, aiming to preserve high-dimensional semantic information and thus improve downstream clustering performance. Through this strategy, researchers can achieve effective multi-view clustering even with non-perfect alignment, effectively avoiding clustering biases and limitations caused by data misalignment.

[0005] A search revealed the application number: CN202510770956.8, application date: June 10, 2025, invention title: "A Double-Aligned Partial Multi-View Clustering Method Based on Anchor Point Graph." Traditional incompletely aligned multi-view clustering methods typically rely on a large number of aligned instances for training and require reconstructing the relationships between each view separately, resulting in high computational complexity and difficulty in accurately recovering the alignment relationships between views. To address these issues, this invention proposes using aligned samples as anchor points to construct an anchor point graph. The relationship between unaligned samples and anchor points is measured using bipartite graph similarity, forming a re-representation matrix to achieve initial alignment between views. This method further employs a double-alignment strategy: firstly, based on the re-representation matrix, the Hungarian algorithm is used to achieve initial alignment at the sample level; subsequently, the Hungarian algorithm is used again in the graph embedding space to perform secondary alignment of latent features, forming a double-alignment mechanism that effectively improves the accuracy of view alignment. However, this method still has certain limitations. It is highly dependent on the quality of anchor points; if the anchor point selection is not representative or insufficient in number, alignment errors may propagate in subsequent steps. Furthermore, although this method reduces the need for aligned instances, the repeated use of the Hungarian algorithm still incurs high computational overhead in multi-view, large-sample scenarios, affecting overall efficiency. Therefore, this invention proposes a non-fully aligned multi-view clustering method based on structural consistency contrastive learning. Summary of the Invention

[0006] The technical problem this invention aims to solve is: how to effectively explore the implicit structural information contained in unaligned samples while fully mining the known relationships between aligned samples, so as to reduce the model's over-reliance on aligned samples and thus construct more accurate and robust sample correspondences. This invention provides a non-fully aligned multi-view clustering method based on structural consistency contrastive learning. This invention includes semantic consistency contrastive learning and visual... Figure 1 Semantic consistency contrastive learning achieves implicit class alignment of samples by capturing global semantic consistency information. Figure 1 Consistency-based contrastive learning establishes accurate sample correspondences by hierarchically selecting positive samples and learning consistency information between views while preserving local structural information.

[0007] The present invention solves the above-mentioned technical problems through the following technical solution, and the present invention includes the following steps:

[0008] S1: Obtain the original dataset, which is a non-fully aligned multi-view dataset;

[0009] S2: Use the encoder to extract latent features from the incompletely aligned multi-view dataset, and use the alignment indicator matrix to indicate whether the samples are aligned, with 1 for aligned and 0 for misaligned.

[0010] S3: The extracted latent features are reconstructed through a decoder that is symmetrical to the encoder, and a reconstruction loss is introduced to reduce the difference between the original features and the reconstructed features, thereby optimizing the latent features;

[0011] S4: Based on latent features, initialize the cluster centers for each view and the global cluster center using K-means and the Hungarian algorithm;

[0012] S5: Introduce cluster center contrast loss and semantic consistency loss to optimize global cluster centers;

[0013] S6: Based on global semantic consistency learning, samples are encouraged to move closer to their global cluster centers in the feature space, thus achieving implicit class alignment of samples;

[0014] S7: View-based Figure 1 Sexual impairment, learning vision Figure 1 Sexual information;

[0015] S8: Construct a total loss function based on each loss, update the parameters of the backbone network through backpropagation, and iteratively update the cluster centers of each view and the global cluster center;

[0016] S9: Combine the aligned latent features and use the k-means algorithm to obtain the final clustering result.

[0017] Furthermore, in step S2, via the encoder Extract to the first The potential characteristics of each view are as follows:

[0018] ;

[0019] in, It is the first Potential features of a view It is the first Encoder for each view, It is the first The original features of each view It is the first View encoder parameters.

[0020] Furthermore, in step S3, based on the first The latent features of each view are reconstructed using a decoder that is symmetrical to the encoder, as follows:

[0021] ;

[0022] in, It is the first Reconstruction features of individual views It is the first Decoder for each view It is the first Each view decoder parameter.

[0023] Furthermore, in step S3, the reconstruction loss The definition is as follows:

[0024] ;

[0025] Where V is the total number of views.

[0026] Furthermore, in step S4, the specific processing procedure is as follows:

[0027] S41: After obtaining the latent features of the samples, the K-means algorithm is used to initialize the cluster centers of each view:

[0028] ;

[0029] in, It is the first The cluster centers of each view, where k is the number of cluster centers;

[0030] S42: Based on the latent feature matrices of the aligned samples in each view, concatenate them along the feature dimension and use the K-means algorithm to generate the initial global cluster centers:

[0031] ;

[0032] in, It is the global cluster center. It is the fusion feature after splicing;

[0033] S43: Use the Hungarian algorithm to match cluster centers of different views to achieve cross-view cluster center alignment.

[0034] Furthermore, in step S5, the specific processing procedure is as follows:

[0035] S51: By comparing the cluster centers of each view with the global cluster centers, the discriminative power of the cluster structure is enhanced and the differences in cluster structure between views are eliminated. The similarity measure between cluster centers is defined as follows:

[0036] ;

[0037] in, It is the i-th cluster center of the m-th view. It is the j-th global cluster center;

[0038] S52: Define the cluster center contrastive loss as follows:

[0039] ;

[0040] Where K is the number of categories;

[0041] S53: The soft assignment probability of a sample is obtained by calculating the similarity between the sample in each view and each global cluster center. , representing the probability that sample i in the v-th view belongs to class j, and the soft assignment probability that sample i in the v-th view is assigned to class j. Defined as:

[0042] ;

[0043] in, It's a temperature parameter. It is the latent feature of the i-th sample in the v-th view;

[0044] S54: Apply consistency constraints to global cluster centers using aligned samples, and define the semantic consistency loss as follows:

[0045] ;

[0046] The first term is used for cross-view semantic consistency learning, and the second term is the entropy of cluster assignment probability, which is used for cluster regularization. N is the number of samples. It represents the semantic information of the i-th sample in the m-th view.

[0047] Furthermore, in step S6, the global semantic consistency loss function is constructed as follows:

[0048] ;

[0049] in, For temperature parameters, , , It is the global semantic consistency loss of the v-th view. It is the similarity between the i-th sample in the v-th view and the j-th cluster center in that view. It is the weight of the i-th sample in the v-th view that is closest to the j-th global cluster center.

[0050] Furthermore, in step S7, the specific processing procedure is as follows:

[0051] S71: Cosine distance is used to measure the similarity between samples across views, as defined below:

[0052] ;

[0053] S72: When constructing positive sample pairs for contrastive learning, for aligned samples, the corresponding samples in different views will be directly regarded as positive samples:

[0054] ;

[0055] in, It is the positive sample of the i-th sample in view m;

[0056] For unaligned samples, select the 'a' samples with the highest similarity based on the similarity between samples, and then use the corresponding samples of the aligned samples among these 'a' samples as the soft positive samples of that sample. Meanwhile, to ensure validity, each unaligned sample must select at least one aligned sample. The soft positive samples of unaligned samples are defined as follows:

[0057] ;

[0058] in, It is a soft positive sample of the i-th sample in view m. This means taking the largest 'a' values, and argmax means taking the maximum value.

[0059] S73: Select high-confidence samples as pseudo-positive samples based on the similarity between the sample and the global cluster center.

[0060] ;

[0061]

[0062] in, The confidence threshold. It is a pseudo-positive sample of the i-th sample in view m. It is the set of high-confidence samples for view m. It is the set of high-confidence samples for view n. This is view m containing all sample categories. It is view n, encompassing all sample categories;

[0063] S74: For the i-th sample in view m, its positive and negative sample set is defined as follows:

[0064]

[0065] in, It is the set of positive samples for the i-th sample in view m. It is the set of negative samples of the i-th sample in view m;

[0066] S75: Learning vision through cross-view contrast learning Figure 1 Consistent information, defining the view Figure 1 The consistency loss function is as follows:

[0067] ;

[0068] in, .

[0069] Furthermore, in step S8, the specific processing procedure is as follows:

[0070] S81: Construct the total loss function for training. The total loss function is as follows:

[0071] ;

[0072] in, , , and It's a hyperparameter;

[0073] S82: Based on the total loss function above, optimize the latent features of each view after a fixed number of rounds, use the k-means algorithm to update the cluster centers of each view, and establish a preliminary sample correspondence based on sample similarity to update the global cluster centers.

[0074] Furthermore, in step S9, by calculating the similarity between samples in the view, the most similar cross-view unaligned sample is matched for the unaligned sample, thereby establishing the sample correspondence; then, the aligned latent features are spliced ​​together, and the k-means algorithm is used to obtain the final clustering result.

[0075] The present invention has the following advantages over the prior art:

[0076] 1. This invention introduces semantic consistency contrastive learning and visual... Figure 1 Consistency-based contrastive learning significantly improves the model's adaptability to low-alignment data. Specifically, the model not only fully mines the overall semantic distribution information of each view, effectively reducing its dependence on known aligned samples, but also effectively learns the consistency information among misaligned samples by designing a hierarchical, high-quality sample pair selection mechanism. Therefore, even when sample alignment relationships are severely lacking, the model can still maintain good clustering performance.

[0077] 2. This invention employs a global semantic consistency learning mechanism, guiding each view sample to converge towards its respective global cluster center in the latent feature space. This achieves implicit class alignment of misaligned samples, significantly reducing the dependence of the matching process on explicit alignment relationships. Further, it combines view... Figure 1 The consistency-based contrastive learning method dynamically constructs positive and negative sample pairs to learn consistency information between misaligned samples, effectively improving the accuracy and stability of view alignment and achieving more accurate and reliable alignment results at the semantic level.

[0078] 3. This invention introduces a dynamic collaborative update mechanism for cluster centers, simultaneously optimizing the cluster centers of each view and the global cluster center during training. This guides samples to cluster towards their respective global centers in the latent feature space, thereby forming cluster structures with stronger discriminative power and clearer boundaries. Furthermore, through contrastive learning of cluster centers and semantic consistency constraints, the invention effectively suppresses cluster center drift, significantly improving the reliability and stability of the clustering results. Attached Figure Description

[0079] Figure 1 This is a flowchart illustrating the incompletely aligned multi-view clustering method based on structural consistency contrastive learning in this embodiment of the invention.

[0080] Figure 2 This is a schematic diagram of the model framework corresponding to the incompletely aligned multi-view clustering method based on structural consistency contrastive learning in this embodiment of the invention. Detailed Implementation

[0081] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0082] Example 1

[0083] This embodiment provides a technical solution: a non-fully aligned multi-view clustering method based on structural consistency contrastive learning, aiming to reduce the model's dependence on aligned samples. This method fully leverages the consistency and complementarity information between views by synergistically utilizing global semantic information and local structural information. By organically integrating multi-view representation learning, self-supervised high-confidence guidance, and view sample alignment, a unified deep learning framework is constructed to effectively improve clustering performance in non-fully aligned scenarios. The specific core strategies are as follows:

[0084] 1. Semantic Consistency Contrastive Learning: Traditional methods mainly rely on aligned samples to learn consistency information between views, which significantly limits performance in low alignment scenarios. To address this, this invention proposes a semantic consistency contrastive learning mechanism. This mechanism extracts and fuses the structural information of each view's cluster center, guiding all samples in the latent feature space to gradually approach their respective global cluster centers. This not only effectively alleviates interference from boundary samples but also significantly reduces reliance on cross-view aligned samples while achieving implicit class alignment of unaligned samples in the latent feature space.

[0085] 2. Collaborative Optimization of Cluster Centers: Traditional methods struggle to establish accurate sample correspondences, resulting in an inability to learn effective clustering structures. To address this, this invention employs an alternating optimization strategy, combining comparative learning of cluster centers with comparative learning of semantic consistency. This dynamically and collaboratively updates the cluster centers of each view and the global cluster center, guiding sample representations to align with the global center. This mutually reinforcing process effectively enhances the discriminativeness and consistency of the clustering structure, thereby jointly optimizing and obtaining clearer, higher-quality clustering results.

[0086] 3. View Figure 1 Consistency-based contrastive learning: Traditional methods are susceptible to false negatives when learning consistency information between views. This problem is particularly prominent in incompletely aligned multi-view clustering tasks, making it difficult for the model to effectively capture cross-view consistency. Figure 1 To address this issue, this invention designs a robust contrastive learning loss function. Through a hierarchical positive and negative sample selection strategy, combined with high-confidence clustering information, a high-quality positive sample set is dynamically constructed. This effectively learns the consistency information between misaligned samples, thereby establishing a more accurate sample correspondence. This method effectively promotes consistency learning between views while also fully preserving the specific structural information of each view.

[0087] The core steps of the method of this invention are as follows:

[0088] Step 1: Obtain the original dataset.

[0089] Step 2: Extract latent features from the non-fully aligned multi-view dataset using the encoder, and use the alignment indicator matrix to indicate whether the samples are aligned, with 1 indicating alignment and 0 indicating misalignment.

[0090] Step 3: The extracted latent features are reconstructed using a decoder that is symmetrical to the encoder. A reconstruction loss is introduced to reduce the difference between the original features and the reconstructed features, thereby optimizing the latent features.

[0091] Step 4: Use the k-means algorithm to obtain the cluster centers of each view and the global cluster center.

[0092] Step 5: To learn a good clustering structure, a cluster center contrastive loss L is introduced. CC and semantic consistency loss L SC This is used to optimize the cluster centers.

[0093] Step 6: Apply the aforementioned latent features to global semantic consistency learning, prompting samples to move closer to the global cluster center in the latent feature space, thereby achieving implicit class alignment of samples. Use the global semantic consistency loss L... GC To optimize.

[0094] Step 7: To fully learn the video Figure 1To establish reliable sample correspondences, the system retains local structural information while maintaining consistency information. A hierarchical positive sample selection strategy is employed, and high-quality sample pairs are screened using cluster centers. Finally, latent features are applied to the visual information. Figure 1 Consistent learning, using visual Figure 1 Loss of sexual function L FC To optimize.

[0095] Step 8: Construct a total loss function model based on the above loss, update the parameters of the backbone network through backpropagation, and iteratively update the cluster centers to reduce the gap between the predicted clustering results and the actual clustering results, thereby improving the accuracy of the final clustering task.

[0096] Step 9: Combine the aligned latent features and use the k-means algorithm to obtain the final clustering result.

[0097] Step 10: Repeat the above process until the model reaches the predetermined performance index or meets the termination condition.

[0098] Example 2

[0099] like Figure 1 As shown, this embodiment further explains the incompletely aligned multi-view clustering method based on structural consistency contrastive learning in Embodiment 1.

[0100] The specific steps are as follows:

[0101] Step 1: Obtain the original dataset (Handwritten, Scene-15, BDGP, ​​Reuters, Fashion).

[0102] Step 2: Use the encoder to extract latent features from the non-fully aligned multi-view dataset. Use the alignment indicator matrix to indicate whether the samples are aligned, 1 for aligned and 0 for misaligned. The size is the number of samples × 1. Divide the samples into two subsets, the aligned subset A and the misaligned subset U.

[0103] Step 3: First, pass through the encoder The latent features are extracted, and then the extracted latent features are processed by a decoder that is symmetrical to the encoder. We obtain the reconstructed features and then introduce a reconstruction loss. Optimize latent features by minimizing the difference between the original features and the reconstructed features.

[0104] i. The first thing learned The potential characteristics of each view are:

[0105] (1)

[0106] in, It is the first Potential features of a view It is the first Encoder for each view, It is the first The original features of each view It is the first View encoder parameters;

[0107] ii. Based on the learned latent features, use a view-specific decoder Perform feature reconstruction on each view data to obtain the reconstructed features:

[0108] (2)

[0109] in, It is the first Reconstruction features of individual views It is the first Decoder for each view It is the first View decoder parameters;

[0110] iii. By minimizing the reconstruction loss Maintain view-specific information:

[0111] (3)

[0112] Where V is the total number of views;

[0113] By minimizing the reconstruction loss, the model can learn compact and information-rich latent features. This method effectively preserves the unique information of each view while also addressing the issue of dimensional inconsistency between views, laying a solid foundation for subsequent learning tasks.

[0114] Step 4: Initialize the cluster centers for each view and the global cluster center using K-means and the Hungarian algorithm.

[0115] i. After obtaining the latent features of the samples, the K-means algorithm is used to initialize the cluster centers of each view:

[0116] (4)

[0117] in, It is the first The cluster centers of each view, where k is the number of cluster centers;

[0118] ii. The initial global cluster centers are generated by concatenating the latent feature matrices of the aligned samples from each view along the feature dimension and using the K-means algorithm:

[0119] (5)

[0120] in, It is the global cluster center. It is the fusion feature after splicing;

[0121] iii. Using the Hungarian algorithm to match cluster centers of different views, and by calculating the distance between the cluster centers of two views, construct a K-row, K-column cost matrix M, where... ,pass To minimize the total cost, a correspondence π is determined. This permutation π defines the optimal one-to-one correspondence between the two views. Based on the permutation π, the cluster center set of the second view is reordered to obtain an aligned center set. This achieves alignment of cluster centers across views, laying the foundation for subsequent collaborative optimization.

[0122] Step 5: Calculate the loss function , Optimize the global cluster center.

[0123] i. By comparing the cluster centers of each view with the global cluster centers, the discriminative power of the cluster structure is enhanced and the differences in cluster structure between views are eliminated. The similarity measure between cluster centers is defined as follows:

[0124] (6)

[0125] in, It is the i-th cluster center of the m-th view. It is the j-th global cluster center;

[0126] ii. The cluster center contrast loss is defined as follows:

[0127] (7)

[0128] Where K is the number of categories;

[0129] iii. The soft assignment probability of a sample is obtained by calculating the similarity between the sample in each view and each global cluster center. , representing the probability that sample i in the v-th view belongs to class j, and the soft assignment probability that sample i in the v-th view is assigned to class j. Defined as:

[0130] (8)

[0131] in, It's a temperature parameter. It is the latent feature of the i-th sample in the v-th view;

[0132] iv. Apply consistency constraints to global cluster centers using aligned samples. The semantic consistency loss is defined as follows:

[0133] (9)

[0134] (10)

[0135] The first term is used for cross-view semantic consistency learning, while the second term is the entropy of cluster assignment probabilities, used for cluster regularization to avoid obtaining meaningless solutions. N is the number of samples. It represents the semantic information of the i-th sample in the m-th view (the probability of belonging to a different class).

[0136] Step 6: Calculate the loss function This causes samples to move closer to their respective global cluster centers in the latent feature space, achieving implicit class alignment of samples and ultimately forming a compact and highly discriminative global clustering structure.

[0137] i. Construct a global semantic consistency loss function to encourage different view samples to move closer to their respective global cluster centers in the latent feature space:

[0138] (11)

[0139] (12)

[0140] in, For temperature parameters, It is the global semantic consistency loss of the v-th view. It is the similarity between the i-th sample in the v-th view and the j-th cluster center in that view. It is the weight of the i-th sample in the v-th view that is closest to the j-th global cluster center;

[0141] ii. Therefore, the global semantic consistency loss is:

[0142] (13)

[0143] in, It is a global semantic consistency loss.

[0144] Step 7: Construct high-quality positive and negative sample pairs and calculate robust view contrast loss. Using the consistency representation between learning views as an example, we select two views, m and n, to illustrate the general method for calculating the loss function.

[0145] i. Cosine distance is used to measure the similarity between samples across views, and it is defined as follows:

[0146] (14)

[0147] ii. When constructing positive sample pairs for contrastive learning, for aligned samples, their corresponding samples in different views will be directly regarded as positive samples:

[0148] (15)

[0149] in, It is the positive sample of the i-th sample (aligned sample) in view m.

[0150] iii. For unaligned samples, select the 'a' samples with the highest similarity based on the similarity between samples, and then use the corresponding samples of the aligned samples among these 'a' samples as the soft positive samples of the unaligned sample; at the same time, to ensure validity, each unaligned sample must select at least one aligned sample. The soft positive sample of an unaligned sample is defined as follows:

[0151] (16)

[0152] in, It is a soft positive sample of the i-th sample (unaligned sample) in view m. This means taking the largest 'a' values, and argmax means taking the maximum value.

[0153] iv. Select high-confidence samples as false positives based on their similarity to the global cluster centers to reduce the impact of false negatives (treating similar samples as negative samples), thereby improving sample discriminative power:

[0154] (17)

[0155] (18)

[0156] in, The confidence threshold is set to 1.8. This threshold is used to filter out high-confidence samples from all samples as false positives. It is a pseudo-positive sample of the i-th sample (high-confidence sample) in view m. It is the set of high-confidence samples for view m. It is the set of high-confidence samples for view n. This is view m containing all sample categories. It is view n, encompassing all sample categories;

[0157] v. For the i-th sample in view m, its positive and negative sample set is defined as follows:

[0158] (19)

[0159] in, It is the set of positive samples for the i-th sample in view m. It is the set of negative samples of the i-th sample in view m;

[0160] vi. Through cross-view comparison learning, effective learning of views can be achieved. Figure 1 Sexual information, view Figure 1 The consistency loss function is defined as follows:

[0161] (20)

[0162] (twenty one)

[0163] in, It is a temperature parameter;

[0164] Step 8: Improve model performance through joint optimization and cluster center iteration.

[0165] i. Combine all the above loss functions for model training:

[0166] (twenty two)

[0167] in, , , and It is a hyperparameter used to balance the importance of different losses.

[0168] ii. Based on the above loss optimization of the latent features of each view after a fixed number of rounds, the k-means algorithm is used to update the cluster centers of each view, and a preliminary sample correspondence is established based on sample similarity to update the global cluster centers.

[0169] Step 9: By calculating the similarity of samples between views, match the most similar cross-view unaligned sample to the unaligned sample, thereby establishing the sample correspondence; then, the aligned latent features are concatenated, and the k-means algorithm is used to obtain the final clustering result.

[0170] Step 10: Repeat steps 2-9 above until the model converges or the termination condition is met.

[0171] Example 3

[0172] like Figure 2As shown, in multi-view clustering tasks with incomplete alignment, the goal is to complete the clustering task in complex scenarios where only some samples have alignment relationships. However, the performance of multi-view clustering largely depends on sufficient alignment relationships between views to ensure effective transmission and complementary enhancement of cross-view information. However, in practical applications, limitations in data acquisition conditions or transmission errors often make it difficult to guarantee complete alignment relationships between all views. This situation increases the risk of cross-view clustering errors. Figure 1 The difficulty of learning consistent information significantly increases the challenge of clustering tasks.

[0173] To address the aforementioned issues, this invention develops a non-fully aligned multi-view clustering method based on structural consistency contrastive learning to tackle the clustering challenges in scenarios with non-fully aligned samples. Existing methods often over-rely on aligned samples, making it difficult to fully utilize the structural information contained in unaligned samples, thus limiting the accuracy of sample correspondence construction. This invention's method delves into the intrinsic structural relationships between samples, constructing a more discriminative clustering structure from a global perspective, and fully mining the potential information of unaligned samples. Based on this, a robust contrastive loss function is designed to enhance the consistency representation between views by constructing high-quality positive sample pairs, progressively optimizing the correspondence between cross-view samples. Experimental results on multiple real-world datasets demonstrate that the proposed method has significant effectiveness and superiority in non-fully aligned scenarios.

[0174] Table 1. Experimental results under the scenario of 50% sample misalignment.

[0175]

[0176] In scenarios where samples are not perfectly aligned, the method of this invention, along with other partially aligned multi-view clustering methods such as PVC, MvCLN, SURE, and CGDA, demonstrates superior performance compared to traditional multi-view clustering methods on all datasets, with particularly significant improvements on the HandWritten, BDGP, ​​and Fashion datasets. This advantage is mainly attributed to the fact that all the aforementioned methods explicitly consider the sample misalignment problem, effectively mitigating the challenges posed by partial sample alignment between views by accurately learning the correspondences between samples. Experimental results show that the method of this invention outperforms other partially aligned multi-view clustering methods on multiple datasets, further validating its effectiveness and advantages in feature learning and cross-view mapping construction.

[0177] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A non-fully aligned multi-view clustering method based on structural consistency contrastive learning, characterized in that, Includes the following steps: S1: Obtain the original dataset, which is a non-fully aligned multi-view dataset; S2: Use the encoder to extract latent features from the incompletely aligned multi-view dataset, and use the alignment indicator matrix to indicate whether the samples are aligned, with 1 for aligned and 0 for misaligned. S3: The extracted latent features are reconstructed through a decoder that is symmetrical to the encoder, and a reconstruction loss is introduced to reduce the difference between the original features and the reconstructed features, thereby optimizing the latent features; S4: Based on latent features, initialize the cluster centers for each view and the global cluster center using K-means and the Hungarian algorithm; S5: Introduce cluster center contrast loss and semantic consistency loss to optimize global cluster centers; S6: Based on global semantic consistency learning, samples are encouraged to move closer to their global cluster centers in the feature space, thus achieving implicit class alignment of samples; S7: Learn view consistency information based on view consistency loss; S8: Construct a total loss function based on each loss, update the parameters of the backbone network through backpropagation, and iteratively update the cluster centers of each view and the global cluster center; S9: Combine the aligned latent features and use the k-means algorithm to obtain the final clustering result.

2. The incompletely aligned multi-view clustering method based on structural consistency contrastive learning according to claim 1, characterized in that, In step S2, via the encoder Extract to the first The potential characteristics of each view are as follows: ； in, It is the first Potential features of a view It is the first Encoder for each view, It is the first The original features of each view It is the first View encoder parameters.

3. The incompletely aligned multi-view clustering method based on structural consistency contrastive learning according to claim 2, characterized in that, In step S3, based on the first The latent features of each view are reconstructed using a decoder that is symmetrical to the encoder, as follows: ； in, It is the first Reconstruction features of individual views It is the first Decoder for each view It is the first Each view decoder parameter.

4. The incompletely aligned multi-view clustering method based on structural consistency contrastive learning according to claim 3, characterized in that, In step S3, the reconstruction loss The definition is as follows: ； Where V is the total number of views.

5. The incompletely aligned multi-view clustering method based on structural consistency contrastive learning according to claim 4, characterized in that, In step S4, the specific processing procedure is as follows: S41: After obtaining the latent features of the samples, the K-means algorithm is used to initialize the cluster centers of each view: ； in, It is the first The cluster centers of each view, where k is the number of cluster centers; S42: Based on the latent feature matrices of the aligned samples in each view, concatenate them along the feature dimension and use the K-means algorithm to generate the initial global cluster centers: ； in, It is the global cluster center. It is the fusion feature after splicing; S43: Use the Hungarian algorithm to match cluster centers of different views to achieve cross-view cluster center alignment.

6. The incompletely aligned multi-view clustering method based on structural consistency contrastive learning according to claim 5, characterized in that, In step S5, the specific processing procedure is as follows: S51: By comparing the cluster centers of each view with the global cluster centers, the discriminative power of the cluster structure is enhanced and the differences in cluster structure between views are eliminated. The similarity measure between cluster centers is defined as follows: ； in, It is the i-th cluster center of the m-th view. It is the j-th global cluster center; S52: Define the cluster center contrastive loss as follows: ； Where K is the number of categories; S53: The soft assignment probability of a sample is obtained by calculating the similarity between the sample in each view and each global cluster center. , representing the probability that sample i in the v-th view belongs to class j, and the soft assignment probability that sample i in the v-th view is assigned to class j. Defined as: ； in, It's a temperature parameter. It is the latent feature of the i-th sample in the v-th view; S54: Apply consistency constraints to global cluster centers using aligned samples, and define the semantic consistency loss as follows: ； The first term is used for cross-view semantic consistency learning, and the second term is the entropy of cluster assignment probability, which is used for cluster regularization. N is the number of samples. It represents the semantic information of the i-th sample in the m-th view.

7. The incompletely aligned multi-view clustering method based on structural consistency contrastive learning according to claim 6, characterized in that, In step S6, the global semantic consistency loss function is constructed as follows: ； in, For temperature parameters, , , It is the global semantic consistency loss of the v-th view. It is the similarity between the i-th sample in the v-th view and the j-th cluster center in that view. It is the weight of the i-th sample in the v-th view that is closest to the j-th global cluster center.

8. The incompletely aligned multi-view clustering method based on structural consistency contrastive learning according to claim 7, characterized in that, In step S7, the specific processing procedure is as follows: S71: Cosine distance is used to measure the similarity between samples across views, as defined below: ； S72: When constructing positive sample pairs for contrastive learning, for aligned samples, the corresponding samples in different views will be directly regarded as positive samples: ； in, It is the positive sample of the i-th sample in view m; For unaligned samples, select the 'a' samples with the highest similarity based on the similarity between samples, and then use the corresponding samples of the aligned samples among these 'a' samples as the soft positive samples of that sample. Meanwhile, to ensure validity, each unaligned sample must select at least one aligned sample. The soft positive samples of unaligned samples are defined as follows: ； in, It is a soft positive sample of the i-th sample in view m. This means taking the largest 'a' values, and argmax means taking the maximum value. S73: Select high-confidence samples as pseudo-positive samples based on the similarity between the sample and the global cluster center. ； in, The confidence threshold. It is a pseudo-positive sample of the i-th sample in view m. It is the set of high-confidence samples for view m. It is the set of high-confidence samples for view n. This is view m containing all sample categories. It is view n, encompassing all sample categories; S74: For the i-th sample in view m, its positive and negative sample set is defined as follows: in, It is the set of positive samples for the i-th sample in view m. It is the set of negative samples of the i-th sample in view m; S75: View consistency information is learned through cross-view comparison learning, and the view consistency loss function is defined as follows: ； in, .

9. The incompletely aligned multi-view clustering method based on structural consistency contrastive learning according to claim 8, characterized in that, In step S8, the specific processing procedure is as follows: S81: Construct the total loss function for training. The total loss function is as follows: ； in, , , and It's a hyperparameter; S82: Based on the total loss function above, optimize the latent features of each view after a fixed number of rounds, use the k-means algorithm to update the cluster centers of each view, and establish a preliminary sample correspondence based on sample similarity to update the global cluster centers.

10. The incompletely aligned multi-view clustering method based on structural consistency contrastive learning according to claim 9, characterized in that, In step S9, by calculating the similarity between samples in the view, the most similar cross-view misaligned sample is matched for the misaligned sample, thereby establishing a sample correspondence relationship; Subsequently, the aligned latent features are concatenated, and the k-means algorithm is used to obtain the final clustering result.

Images