Incomplete multi-view clustering method based on cross-view interpolation and contrast learning

Abstract

The invention discloses an incomplete multi-view clustering method based on cross-view interpolation and comparative learning, and relates to the technical field of computer vision and pattern recognition, and the method comprises the steps: collecting original multi-view data; original multi-view data is utilized to train a constructed incomplete multi-view clustering model, and the model comprises a plurality of encoders, a multi-view reconstruction decoder, a cross-view missing subspace interpolation module and a cross-view comparison learning module; the encoder encodes the original multi-view data to obtain potential representation; the cross-view missing subspace interpolation module performs propagation and completion according to the potential representation to obtain an overall potential representation; the cross-view comparative learning module carries out comparative learning according to the overall potential representation to obtain a clustering result; a multi-view reconstruction decoder decodes the overall potential representation; and collecting to-be-clustered multi-view data, and inputting the to-be-clustered multi-view data into the trained incomplete view clustering model to obtain a clustering result. According to the method, high-quality clustering of incomplete multiple views is realized.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and pattern recognition technology, and more specifically to an incomplete multi-view clustering method based on cross-view interpolation and contrastive learning. Background Technology

[0002] In recent years, multi-view clustering has received increasing attention. Compared with single-view clustering, multi-view clustering can explore consistency and complementary information about the same object from data from different sources or modalities. This technology is becoming increasingly popular in fields such as computer vision and pattern recognition. Almost all methods rely on data integrity, i.e., all data must be complete. However, due to issues with data transmission or storage, multi-view data inevitably encounters data loss, resulting in incomplete data.

[0003] Therefore, several effective solutions have been proposed to address the problem of incomplete data. Residual Reconstruction Autoencoders (CRAs) minimize the residual between the reconstructed data and the original data by stacking residual autoencoder (AE) structures, thereby recovering the missing view. View Completion Generative Adversarial Networks (VIGANs) combine the network structures of AEs and Generative Adversarial Networks (GANs) to reconstruct the missing view based on the output of the GANs. While these methods can partially solve the problem of incomplete data, they rely on a combination of single modules from multiple datasets. The tight coupling between these modules limits the model's flexibility in adapting to data with different spatial structures and high-level semantic information. Therefore, the model's clustering performance fails to reach optimum levels.

[0004] Therefore, how to improve the performance of multi-view clustering is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, this invention is proposed to provide an incomplete multi-view clustering method based on cross-view interpolation and contrastive learning to overcome or at least partially solve the above problems. It is used to solve the problem of incomplete multi-view data, utilize existing multi-view information to recover missing data, and achieve high-quality clustering by fully activating the system theory.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, embodiments of the present invention provide an incomplete multi-view clustering method based on cross-view interpolation and contrastive learning, comprising the following steps: Step 1: Collect data from multiple samples at different views to construct the original multi-view data; Step 2: Train the constructed incomplete multi-view clustering model using the original multi-view data. The incomplete multi-view clustering model includes several encoders, a multi-view reconstruction decoder, a cross-view missing subspace imputation module, and a cross-view contrastive learning module. The encoder encodes the original multi-view data to obtain a latent representation set. The cross-view missing subspace imputation module constructs neighborhood relationships based on the latent representation set, propagates and completes the latent representations in the missing views, and obtains the overall latent representation set. The cross-view contrastive learning module performs contrastive learning to optimize the encoder and multi-view reconstruction decoder based on the overall latent representation set and obtains the clustering results. The multi-view reconstruction decoder decodes the overall latent representation set. Step 3: Collect the multi-view data to be clustered, input it into the trained incomplete multi-view clustering model, and obtain the clustering results.

[0008] Preferred, original multi-view data Includes complete data and incomplete data Here, v represents the view number, and V represents the number of views. Incomplete data corresponds to views that are missing; complete data is sample data containing data from all views, while incomplete data is sample data containing only partial view data. Because the data from different views is encoded using an independent channel method, it is possible to distinguish whether the data is complete or incomplete after encoding.

[0009] The technical effects of the above solution are as follows: the encoder encodes the data uniformly, mapping the data to a unified dimension; the cross-view missing subspace imputation module completes the latent representation of incomplete data based on the latent representation of complete data; the cross-view contrastive learning module introduces cross-view contrastive learning constraints, bringing closer the representations of the same sample in different views in the complete latent representation in the latent space, and widening the representations between different samples to achieve clustering, and reducing gradients through backpropagation, adjusting the weights of the encoder and decoder, and optimizing the model; the multi-view reconstruction decoder reconstructs the latent representation to make it meaningful.

[0010] Preferably, each encoder processes data from a different view to obtain a set of latent representations for that view. , including the complete set of latent representations and incomplete latent representation set For incomplete data, the encoder uses zero-placeholders to fill in the incomplete latent representation set to match the vector shape; the latent representation expression generated by the encoder is: ; in, This represents the potential representation set of view v. d represents the number of samples in view v, and d represents the dimension of the latent representation; This indicates the encoder corresponding to view v. , Represents a view The number of samples corresponding to the complete data in the dataset. Represents a view The number of samples corresponding to incomplete data.

[0011] Preferably, the specific processing procedure of the cross-view missing subspace interpolation module is as follows: Step 211: Calculate the similarity between any target sample corresponding to the complete data in each view and other samples using Gaussian geometric distance, and sort them from high to low according to the similarity. Step 212: Select the other samples corresponding to the top K similarity values ​​in each view as the neighborhood of the target sample, and construct the K-nearest neighbor graph of the corresponding view using the similarity value of each neighborhood as the weight. , represented as: ; in, Represents the K-nearest neighbor graph of view v; This represents the i-th target sample data in view v. and the jth other sample data The similarity between them; T represents transpose; Indicates the norm; Step 213: Take the sample data of the same sample corresponding to the incomplete data in different views as anchor points, and select the K-nearest neighbor graph of the view where the anchor point of each incomplete data is located in turn; Step 214: Perform a weighted summation based on the weights of the K-nearest neighbor graph and the sample data in the view containing the incomplete data to obtain the latent representation of the completed incomplete data; represented as: ; in, This represents the potential representation of the incomplete data corresponding to the t-th sample position in view v; This represents the K-nearest neighbor graph of view r, where the anchor point corresponding to the position of the t-th sample is located, and the anchor point is the ith sample data; [t,:] represents taking the t-th row, that is, the weight of the t-th sample in view r with respect to all its neighbors in the complete data; This represents the latent representation set of sample data corresponding to the K neighborhoods of the K-nearest neighbor graph in view v.

[0012] Step 215: The complete data latent representation set in the original multi-view data and the incomplete data latent representation set are combined to form the overall latent representation set, realizing the interpolation of incomplete data.

[0013] The technical effect of the above solution is that it uses neighborhood similarity in consistency information to infer neighborhood information of missing data, and uses neighborhood information propagated across views and available data within the view to achieve high-quality missing data recovery by combining similarity information embedded in the neighborhood and its k nearest neighbors.

[0014] Preferably, when there is incomplete data in the sample data corresponding to the K neighborhoods of the K-nearest neighbor graph in step 214, the weights of the neighborhoods corresponding to the complete data are normalized to obtain updated weights, and the updated weights and the sample data corresponding to the complete data are weighted and summed.

[0015] Preferably, the cross-view contrastive learning module uses a contrastive learning mechanism to construct a contrastive learning loss function based on the overall latent representation, uses the contrastive learning loss function to perform backpropagation to update the weight parameters of the encoder and the multi-view reconstruction decoder, and clusters the overall latent representation to generate clustering results.

[0016] Preferably, the contrastive learning loss function is expressed as: ; ; ; ; Among them, L CCL This represents the contrastive learning loss function; This represents the loss function applied to positive sample pairs in contrastive learning; This represents the loss function applied to negative sample pairs; P represents the sample pair type, with a value of 0 indicating a negative sample pair and a value of 1 indicating a positive sample pair. These are hyperparameters; This represents the latent representation of the incomplete data after the i-th sample in view v is completed. Other views The latent representation of the incomplete data after the completion of the i-th sample, i.e., multi-view data of the same object; d represents the latent representation distance; Indicates the number of samples corresponding to incomplete data; Other views The latent representation of other randomly selected samples n is the complement of the positive samples, N = -1; Indicates the number of positive sample pairs; Indicates the number of negative sample pairs; This represents the latent representation of the i-th sample in view v; Other views The latent representation of the i-th sample; m represents the effective distinguishing boundary, which is the sum of the average distance between positive samples and the average distance between negative samples, ensuring that negative sample pairs maintain sufficient distance to be effectively distinguished.

[0017] The technical effect of the above technical solution is that the cross-view contrastive learning module is based on the semantic consistency assumption, uses reconstructed complete data to construct cross-view positive and negative sample pairs, and fully explores the consistency and complementarity information between views by using a contrastive learning loss function that minimizes the distance between positive sample pairs and maximizes the distance between negative sample pairs in the cross-view positive and negative sample pairs, thereby obtaining clustering results.

[0018] Preferably, the latent representations of different views in the overall latent representation set are concatenated into a cross-view complete latent representation, which is then input into a multi-view reconstruction decoder for decoding to obtain the reconstructed latent representation; represented as: ; in, Reconstructing the latent representation; Indicates a multi-view reconstruction decoder; [ ] represents a connection operation; This represents the set of potential representations of view V within the overall potential representation set.

[0019] Preferably, the incomplete multi-view clustering model is optimized using a total loss function during training. The total loss function L is expressed as: ; ; ; ; ; in, This represents the encoder loss function, used to optimize the multi-view reconstruction module; , These are hyperparameters; This represents the interpolation loss function, used to optimize the interpolation module for missing subspaces across views; Indicates true potential characteristics; Indicates the overall potential characteristics; This indicates taking the F-norm; V represents the number of views; a is a hyperparameter, a=m / 3; Indicates the number of samples corresponding to incomplete data; This represents the completion loss for the t-th sample in view v that is incomplete. This represents the potential representation of the incomplete data after the t-th sample in view v is completed; d represents the latent representation of the complete data corresponding to the t-th sample in view v; d represents the distance of the latent representation. This represents the latent representation corresponding to the k-th sample in view v; This represents the latent representation corresponding to the l-th sample in view r; Used to calculate the distance between the potential representations of any sample in any view.

[0020] As can be seen from the above technical solution, compared with the prior art, this invention discloses an incomplete multi-view clustering method based on cross-view interpolation and contrastive learning. Utilizing the concept of a fully driven system, it decouples multiple modules to construct an incomplete multi-view clustering model and grants each module different degrees of freedom to handle data with different spatial structures and semantic information, addressing different problems and thus extracting diverse information more accurately. The incomplete multi-view clustering model includes a multi-view reconstruction module, a cross-view missing subspace interpolation module (CVI), and a cross-view contrastive learning module (CCL). The original multi-view data is first processed by encoders corresponding to their respective viewpoints in the multi-view reconstruction module to extract latent representations, and then reconstructed by decoders to preserve the original structural information. Subsequently, the CVI utilizes cross-view... Figure 1 The method constructs neighborhood relationships to propagate and complete latent features in missing views. Based on this, a cross-view contrastive learning constraint is introduced into the CCL (Concurrent View Collateralization) to bring the representations of the same sample closer together across different views in the latent space, while widening the gaps between representations of different samples, thereby jointly mining consistency and complementarity information from multi-view data. Each module interacts with information through latent features and performs end-to-end optimization under the joint constraint of a unified loss function. The method of this invention has been validated on different datasets, achieving optimal clustering performance on various datasets and yielding competitive results on all datasets. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the incomplete multi-view clustering model structure provided in an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] This invention discloses an incomplete multi-view clustering method based on cross-view interpolation and contrastive learning, comprising the following steps: S1: Collect data from multiple samples at different views to construct the original multi-view data; S2: Train the constructed incomplete multi-view clustering model using the original multi-view data. The incomplete multi-view clustering model includes several encoders, multi-view reconstruction decoders, cross-view missing subspace imputation modules, and cross-view contrastive learning modules, such as... Figure 1 As shown; S3: Collect the multi-view data to be clustered, input it into the trained incomplete multi-view clustering model, and obtain the clustering results.

[0025] Furthermore, the original multi-view data is an incomplete dataset, denoted as... It consists of complete data and incomplete data The data is composed of several parts, where v represents the view number, V represents the number of views, and incomplete data corresponds to views that are missing. Complete data consists of sample data containing data from all views, while incomplete data consists of sample data containing only partial view data. Because the data from different views is encoded using a channel-independent method, it is possible to distinguish whether the data is complete or incomplete after encoding.

[0026] Furthermore, each encoder processes data corresponding to a specific view, reducing the dimensionality of each view's data to a unified latent subspace with the same dimensions for further processing. This reduces dimensionality differences between different viewpoints due to variations in data sources or modalities. Encoders for different viewpoints can differ in network layers, hidden units, and parameter scale to accommodate the differences in dimensionality, distribution, and semantic level of the original data from each viewpoint. Having a dedicated encoder for each viewpoint ensures that the data from different viewpoints are independent and decoupled during the encoding phase, enabling the extraction of more accurate and denoised latent representations tailored to the specific characteristics of each viewpoint's data. The latent representation set obtained for each viewpoint is based on the data from different viewpoints. Represented as: ; in, d represents the set of potential representations of view v, and d represents the dimension of the potential representation. It is a specific encoder for view v, the potential representation Accordingly, it is divided into complete latent representations and incomplete potential representation . , This indicates the number of samples in view v. This indicates the number of samples corresponding to the complete data in view v. This indicates the number of samples corresponding to incomplete data in view v.

[0027] Furthermore, the cross-view missing subspace interpolation module constructs neighborhood relationships based on latent representations, propagating and completing the latent representations in the missing views; high-quality interpolation is achieved using cross-view information through the cross-view interpolation module, specifically as follows: S211: For each individual sample corresponding to the complete data in the view, use Gaussian geometric distance to calculate its distance to other samples in the same view to obtain similarity; S212: For each individual sample in the view, select the top K other samples as its neighborhood, and construct a K-nearest neighbor graph of the view data using the similarity of each neighborhood as a weight. , represented as: ; in, Represents the K-nearest neighbor graph of view v; This represents the i-th target sample data in view v. and the jth other sample data The similarity between them; T represents transpose; Indicates the norm; S213: Take the sample data of the same sample corresponding to the incomplete data in different views as anchor points, and sequentially select the K-nearest neighbor graph of the view where an anchor point of each incomplete data is located. S214: Based on the weights of the K-nearest neighbor graph and the sample data in the view containing the corresponding incomplete data, perform a weighted summation to obtain the potential representation of the completed incomplete data; The potential representation of the completed incomplete data is: ; in, This represents the potential representation of the incomplete data corresponding to the t-th sample position in view v; Let [t,:] represent the K-nearest neighbor graph of view r where the anchor point corresponding to the t-th sample position is located, and the anchor point is the ith sample data; [t,:] represents taking the t-th row, that is, the weight of sample t in view r with respect to all neighborhoods in the complete data; Let V represent the latent representation set of sample data corresponding to the K neighborhoods of the K-nearest neighbor graph in the v-th view.

[0028] S215: The complete data potential representation set in the original multi-view data and the incomplete data potential representation set are combined to form the overall potential representation set, realizing the interpolation of incomplete data.

[0029] Furthermore, by minimizing the distance between the recovered missing data and its corresponding complete data, the consistency and complementary information between these views are learned. To achieve this goal, a contrastive learning method is employed to effectively extract and explore underlying data information. ; Where a=m / 3 is a hyperparameter, and d(,) is calculated using the following formula: ; Extending this to all samples, the optimized imputation loss function for the cross-view missing subspace imputation module is as follows: ; in, This indicates taking the F-norm; V represents the number of views; a is a hyperparameter, a=m / 3; Indicates the number of samples corresponding to incomplete data; This represents the completion loss for the t-th sample in view v that is incomplete. This represents the potential representation of the incomplete data after the t-th sample in view v is completed; d represents the latent representation of the complete data corresponding to the t-th sample in view v; d represents the distance of the latent representation. This represents the latent representation corresponding to the k-th sample in view v; This represents the latent representation corresponding to the l-th sample in view r; Used to calculate the distance between the potential representations of any sample in any view.

[0030] Furthermore, the cross-view contrastive learning module employs a contrastive learning mechanism to construct a contrastive learning loss function based on the overall latent representation. It then uses the contrastive learning loss function to perform backpropagation to update the weight parameters of the encoder and the multi-view reconstruction decoder, and clusters the overall latent representation to generate clustering results.

[0031] According to the semantic consistency hypothesis, multi-perspective data describing the same object should be consistent, while data describing different objects should be distinctive. Therefore, the concept of contrastive learning can be used to promote the mining and learning of cross-perspective information.

[0032] The constructed contrastive learning loss function is expressed as: ; in This represents the loss function applied to positive sample pairs in contrastive learning, while This represents the loss function applied to negative sample pairs. P takes values ​​of 0 and 1, corresponding to negative and positive sample pairs respectively. β is a hyperparameter.

[0033] When selecting positive and negative sample pairs, samples from different views describing the complete data of the same object are used as positive sample pairs. For each sample in the complete dataset, N samples are randomly selected from other views as negative samples. Since the positive samples are selected from the complete dataset, they correspond to the same indexed samples in the multi-view dataset. Therefore, for positive samples, they are brought closer together by minimizing the distance between them.

[0034] ; ; ; in and It is multi-view data describing the same object. This represents the latent representation of the incomplete data after the i-th sample in view v is completed. Other views The latent representation of the incomplete data after the completion of the i-th sample; d represents the distance of the latent representation; Indicates the number of samples corresponding to incomplete data; Other views selected randomly The latent representation of the other samples n is the complement of the positive samples, N = -1; and These represent the number of positive and negative pairs, respectively. This represents the latent representation of the i-th sample in view v; Other views The latent representation of the i-th sample. A positive pair (PositivePair) is the latent representation of the same sample under different views; comparative training requires them to be as similar as possible (the smaller the distance d between the latent vectors of different views, the smaller the loss). For... Related negative sample pairs are designed to increase the distance between them. A negative sample pair refers to the latent representation of different samples under the same or different views; m is a boundary designed to ensure that negative sample pairs maintain sufficient distance to be effectively distinguishable.

[0035] Furthermore, the decoder receives a complete cross-view latent representation, obtained by concatenating latent representations from different views within the overall latent representation set. It then learns and decodes this cross-view information to generate reconstructed latent features. This addresses the problem that, due to data gaps in multi-view datasets, the generated latent representation cannot capture cross-view information when information is extracted independently from each view using only encoded and corresponding decoded data. It effectively utilizes external data to extract consistent and complementary information across different views. The decoder's formula is as follows: ; in, Reconstructing the latent representation; This indicates a multi-view reconstruction decoder; [ ] represents a connection operation, which can be performed using the `cancatenate` function; Let X represent the set of potential representations of view V within the overall potential representation set. Then, the original multi-view data X can be reconstructed into complete multi-view data. .

[0036] Furthermore, during training, the incomplete multi-view clustering model is optimized using a total loss function, which is: ; ; in, This represents the encoder loss function, used to optimize the multi-view reconstruction module; For hyperparameters; This represents the interpolation loss function, used to optimize the interpolation module for missing subspaces across views; The contrastive constraint loss function is used to optimize the cross-view contrastive learning module. Furthermore, the modules of the incomplete multi-view clustering model are decoupled and independent, each with its own independent loss and weights, and are jointly controlled and optimized through the overall loss function. This allows for more full utilization of spatial structure and semantic information from different perspectives, effectively extracting multiple types of information from the data and significantly improving clustering performance.

[0037] On the other hand, in a specific embodiment, temperature data and images corresponding to five samples are obtained using temperature sensors and cameras in five scenes (s1, s2, s3, s4, s5) as sample data to construct the original multi-view data; wherein, the temperature data corresponds to view v1, and the images correspond to view v2. If the data for s2 and s3 in view v1 is missing, and the data for s4 in view v2 is missing, then the sample data corresponding to s1 and s5 in view v1, the sample data for view v2 in s2 and s3, and the sample data for view v1 in s4 are complete data, while the sample data for view v1 in s2 and s3 and the sample data for view v2 in s4 are incomplete data. The clustering process using the incomplete multi-view clustering model is as follows: S1: Encode the complete sample data in view v1 and view v2 using two sets of encoders to obtain the latent representation; S2: Calculate the similarity between sample data of s2 and other sample data in view v2, select the top 3 samples with the highest similarity s1, s3 and s5 as the neighborhood, and construct the K-nearest neighbor graph of view v2 using the corresponding similarity as weights. Similarly, select a target sample to construct the K-nearest neighbor graph of view v1; Incomplete data in views v1 and v2 are interpolated sequentially. When traversing to the sample data of s2 in view v1, the sample data of s2 in view v2 is used as the anchor point. For a neighborhood of s1, s3 and s5 in the K-nearest neighbor graph of view v2, the weights in the K-nearest neighbor graph and the latent representations corresponding to the sample data of s1, s3 and s5 in view v1 are weighted and summed to obtain the latent representation of the sample data of s2 in view v1. However, since the sample data of s3 in view v1 is missing, the weights {0.1, 0.2} corresponding to the neighborhoods s1 and s5 are normalized to obtain the updated weights {0.333, 0.667}. The updated weights and the latent representations corresponding to the sample data of s1 and s5 in view v1 are weighted and summed to obtain the latent representation of the sample data of s2 in view v1. Other incomplete data are interpolated in the same way. S3: A contrastive learning mechanism is used to construct a contrastive learning loss function based on the overall latent representation. The contrastive learning loss function is used for backpropagation to update the weight parameters of the encoder and the multi-view reconstruction decoder. The overall latent representation is clustered to generate clustering results. The latent representations of different views in the overall latent representation are concatenated into a cross-view complete latent representation and input into the multi-view reconstruction decoder for decoding to obtain the reconstructed latent representation.

[0038] On the other hand, in one specific embodiment, comparative experiments were conducted using four widely used datasets to compare the method of the present invention with nine baseline methods. Accuracy (ACC), normalized mutual information (NMI), and adjusted RAND index (ARI) were used as evaluation metrics, and higher values ​​of these metrics indicate better method performance.

[0039] (1) The dataset includes: Scene 15: This dataset contains a total of 4,485 images, covering 15 different categories, including indoor and outdoor scenes. The dataset is represented using two feature perspectives: 20-dimensional GIST features and 59-dimensional PHOG features.

[0040] A subset of samples from a given society contains 18,758 samples, categorized into six distinct classes. English and French are transformed into a 10-dimensional latent space using a standard autoencoder, providing two different feature views for analysis.

[0041] NoisyMNIST: This dataset contains 70,000 samples across 10 classes. To adapt to benchmark models handling large-scale datasets, 30,000 samples were randomly selected for evaluation.

[0042] MNIST-USPS: The MNIST and USPS datasets represent two different perspectives. MNIST images span 10 digit categories and have a dimension of 784, while USPS images have a dimension of 256. Each dataset contains 5,000 randomly selected samples.

[0043] The IFAST algorithm of this invention was compared with nine multi-view clustering baseline methods, including AE2-Nets, PMVC, UEAF, DAIMC, EERIMVC, SURE, COMPLETER, DSIMVC, and DIVIDE. Experimental results show that the model of this invention achieves competitive results on all four datasets. On the Reuters dataset, the method of this invention improves the ACC metric by 8.35% compared to the second-best method. Similarly, on the ARI metric, the method of this invention also improves the ACC metric by 6.08% compared to the second-best method.

[0044] (2) An ablation experiment was conducted on the method of the present invention. An ablation study was conducted on the three modules of the model in this invention: the multi-view reconstruction decoder, the cross-view missing subspace imputation module, and the cross-view contrastive learning module, to evaluate their contributions. Experimental results show that optimal experimental results are obtained only when all three modules are present. The importance of the cross-view reconstruction loss is evident, as it effectively eliminates redundant information in the original data. Similarly, the cross-view missing view imputation method is also significant, significantly improving clustering performance. Furthermore, the cross-view contrastive learning loss enables the model to fully utilize the consistency and complementary information between different views. These three modules work synergistically and reinforce each other, thereby achieving the best clustering results.

[0045] In summary, the incomplete multi-view clustering method of the present invention can flexibly handle datasets from different sources and make full use of the diverse information in incomplete multi-view data.

[0046] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0047] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An incomplete multi-view clustering method based on cross-view interpolation and contrastive learning, characterized in that, Includes the following steps: Step 1: Collect data from multiple samples at different views to construct the original multi-view data; Step 2: Train the constructed incomplete multi-view clustering model using the original multi-view data. The incomplete multi-view clustering model includes several encoders, multi-view reconstruction decoders, cross-view missing subspace imputation modules, and cross-view contrastive learning modules; the encoders encode the original multi-view data to obtain a set of latent representations. The cross-view missing subspace interpolation module constructs a K-nearest neighbor graph based on the latent representation set, and uses the K-nearest neighbor graph to perform weighted summation of sample data to propagate and complete the latent representations in the missing view, thereby obtaining the overall latent representation set. The cross-view contrastive learning module performs contrastive learning to optimize the encoder and multi-view reconstruction decoder based on the overall latent representation set, and obtains clustering results; the multi-view reconstruction decoder decodes the overall latent representation set. Step 3: Collect the multi-view data to be clustered, input it into the trained incomplete multi-view clustering model, and obtain the clustering results.

2. The incomplete multi-view clustering method based on cross-view interpolation and contrastive learning as described in claim 1, characterized in that, Original multi-view data Includes complete data and incomplete data v represents the view number, and V represents the number of different views; incomplete data corresponds to the missing view; complete data is sample data containing all view data, and incomplete data is sample data containing only some view data.

3. The incomplete multi-view clustering method based on cross-view interpolation and contrastive learning as described in claim 2, characterized in that, Each encoder processes data from a different view, obtaining a latent representation set composed of latent representations of different samples from different views. , including the complete set of latent representations and incomplete latent representation set .

4. The incomplete multi-view clustering method based on cross-view interpolation and contrastive learning as described in claim 3, characterized in that, The latent representation expression generated by the encoder is: ； in, This represents the potential representation set of view v. d represents the number of samples in view v, and d represents the dimension of the latent representation; This indicates the encoder corresponding to view v.

5. The incomplete multi-view clustering method based on cross-view interpolation and contrastive learning as described in claim 4, characterized in that, The specific processing procedure of the cross-view missing subspace interpolation module is as follows: Step 211: Calculate the similarity between any target sample corresponding to the complete data in each view and other samples using Gaussian geometric distance, and sort them from high to low according to the similarity. Step 212: Select the other samples corresponding to the top K similarity values ​​in each view as the neighborhood of the target sample, and construct the K-nearest neighbor graph of the corresponding view using the similarity value of each neighborhood as the weight. , represented as: ； in, Represents the K-nearest neighbor graph of view v; This represents the i-th target sample data in view v. and the jth other sample data The similarity between them; T represents transpose; Indicates the norm; Step 213: Take the sample data of the same sample corresponding to the incomplete data in different views as anchor points, and select the K-nearest neighbor graph of the view where the anchor point of each incomplete data is located in turn; Step 214: Perform a weighted summation based on the weights of the K-nearest neighbor graph and the sample data in the view containing the incomplete data to obtain the latent representation of the completed incomplete data; the latent representation of the completed incomplete data is: ； in, This represents the potential representation of the incomplete data corresponding to the t-th sample position in view v; Let [t,:] represent the K-nearest neighbor graph of view r where the anchor point corresponding to the position of the t-th sample is located, and the anchor point is the data of the ith sample; [t,:] represents the weight of the t-th sample in view r with respect to all its neighbors, taking the t-th row and the complete data. This represents the latent representation set of sample data corresponding to the K neighborhoods of the K-nearest neighbor graph in view v; Step 215: The complete data latent representation set and the incomplete data latent representation set in the original multi-view data are combined to form the overall latent representation set.

6. The incomplete multi-view clustering method based on cross-view interpolation and contrastive learning as described in claim 5, characterized in that, When there is incomplete data in the sample data corresponding to the K neighborhoods of the K-nearest neighbor graph in step 214, the weights of the neighborhoods corresponding to the complete data are normalized to obtain updated weights, and the updated weights and the sample data corresponding to the complete data are weighted and summed.

7. The incomplete multi-view clustering method based on cross-view interpolation and contrastive learning as described in claim 1, characterized in that, The cross-view contrastive learning module uses a contrastive learning mechanism to construct a contrastive learning loss function based on the overall latent representation. It then uses the contrastive learning loss function to perform backpropagation to update the weight parameters of the encoder and the multi-view reconstruction decoder, and clusters the overall latent representation to generate clustering results.

8. The incomplete multi-view clustering method based on cross-view interpolation and contrastive learning as described in claim 7, characterized in that, The contrastive learning loss function is expressed as: ； ； ； ； Wherein, LCCL represents the contrastive learning loss function; This represents the loss function applied to positive sample pairs in contrastive learning; This represents the loss function applied to negative sample pairs; P represents the sample pair type, with a value of 0 indicating a negative sample pair and a value of 1 indicating a positive sample pair. These are hyperparameters; This represents the latent representation of the incomplete data after the i-th sample in view v is completed. Represents other views The latent representation of the incomplete data after the completion of the i-th sample; d represents the distance of the latent representation; Indicates the number of samples corresponding to incomplete data; Represents other views The latent representation of other randomly selected samples n is the complement of the positive samples, N= -1, This indicates the number of samples in view v; Indicates the number of positive sample pairs; Indicates the number of negative sample pairs; This represents the latent representation of the i-th sample in view v; Represents other views The latent representation of the i-th sample; m represents the effective distinguishing boundary.

9. The incomplete multi-view clustering method based on cross-view interpolation and contrastive learning as described in claim 3, characterized in that, The latent representations of different views in the overall latent representation set are concatenated into a cross-view complete latent representation, which is then input into the multi-view reconstruction decoder for decoding to obtain the reconstructed latent representation; the expression is: ； in, Reconstructing the latent representation; Indicates a multi-view reconstruction decoder; [ ] represents a connection operation; This represents the set of potential representations of view V within the overall potential representation set.

10. The incomplete multi-view clustering method based on cross-view interpolation and contrastive learning as described in claim 8, characterized in that, During training, the incomplete multi-view clustering model is optimized using a total loss function, L, which is expressed as: ； ； ； ； ； in, Represents the encoder loss function; , These are hyperparameters; Indicates the interpolation loss function; This represents the contrast constraint loss function; Indicates true potential characteristics; Indicates the overall potential characteristics; This indicates taking the 2-norm; 'a' is a hyperparameter. Indicates the number of samples corresponding to incomplete data; This represents the completion loss for the t-th sample in view v that is incomplete. This represents the potential representation of the incomplete data after the t-th sample in view v is completed; d represents the latent representation of the complete data corresponding to the t-th sample in view v; d represents the distance of the latent representation. This represents the latent representation corresponding to the k-th sample in view v; This represents the latent representation corresponding to the l-th sample in view r.

Images