An Incomplete Multimodal Smooth Contrastive Clustering Method Based on Information Imputation

Abstract

This invention discloses an incomplete multimodal smooth contrastive clustering method based on information imputation. First, a view imputation mechanism based on statistical information is used to model missing views as a multivariate Gaussian distribution based on distance-weighted nearest neighbors, and the missing views are imputed accordingly to obtain a complete multi-view dataset. Second, an encoder is used to extract latent features, and an FFN-Attention-FFN structure is used to capture the global structural information of the multi-views, while a graph convolutional network is used to extract the local structural information of the multi-views. Third, a graph decoder is used to reconstruct the views. Finally, an MLP is used to generate clustering assignment results. This invention introduces a view imputation mechanism based on statistical information, which reduces the influence of weakly correlated views on the completion result by introducing distance weighting, thus improving the quality of data recovery. Combining the FFN-Attention-FFN structure with a graph convolutional network effectively captures the global and local structural information of the data. Adding Gaussian noise to expand negative sample pairs makes the sample distribution smoother.

Description

Technical Field

[0001] This invention belongs to the field of multi-view clustering technology, specifically relating to an incomplete multimodal smooth contrastive clustering method based on information interpolation. Background Technology

[0002] With the development of data acquisition technology, information about the same object can often be obtained from multiple different sources, forming multi-view data. Multi-view clustering (MVC), as an unsupervised learning paradigm, groups unlabeled instances into different clusters by utilizing consistent and complementary information among multi-view data. However, in real-world applications, due to factors such as equipment failure and sensor errors, some view information in multi-view data may be missing, resulting in incomplete data. For example, in medical diagnosis, patients may not have undergone systematic examinations for various reasons, leading to missing information. These problems render many existing MVC methods unusable, thus prompting the development of Incomplete Multi-view Clustering (IMVC) technology.

[0003] Deep neural networks have been widely used in fields such as computer vision and pattern recognition due to their powerful representation learning capabilities and scalability. To further improve clustering performance, researchers have begun to combine deep neural networks with the IMVC method, proposing incomplete multi-view clustering methods based on deep learning frameworks, such as methods based on generative adversarial networks (GANs), methods based on Transformers, methods based on graph convolutional networks (GCNs), and methods based on contrastive learning.

[0004] However, existing depth-incomplete multi-view clustering methods still have the following drawbacks: 1. While methods based on Generative Adversarial Networks (GANs) can handle the problem of missing data imputation by utilizing the data generation capabilities of GANs, the data generation process is difficult to interpret, and the generated missing data may lack intuitive interpretability, leading to increased uncertainty in the results. 2. Methods based on Transformer or Graph Convolutional Networks (GCNs): These methods only focus on the global or local structure of the data, making it difficult to efficiently utilize both types of information simultaneously, resulting in the underutilization of the feature information contained in the data; 3. Contrastive learning-based methods: These methods group similar samples (positive sample pairs) together in the latent space and disperse dissimilar samples (negative sample pairs). However, these methods lack effective strategies for selecting and utilizing negative samples, resulting in a scarcity of high-confidence negative samples, which affects the model's discriminative ability and clustering accuracy. Summary of the Invention

[0005] To address the problems existing in the above-mentioned background technology, the purpose of this invention is to provide an incomplete multimodal smooth contrastive clustering method based on information interpolation.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: An incomplete multimodal smooth contrastive clustering method based on information interpolation includes the following steps: S1. Using the view with missing samples in the incomplete multi-view as the target view, firstly, determine the neighbor samples of the missing samples in the available views of the incomplete multi-view, then find the representation samples corresponding to the neighbor samples in the target view, and finally, establish a multivariate Gaussian distribution based on the characteristics of the representation samples, and sample the average of multiple values ​​from the multivariate Gaussian distribution as the missing value to imput the missing samples in the target view, thus obtaining the dataset of the complete multi-view. S2. Use a view-specific encoder to extract latent features from the dataset; S3. The latent features are refined through an attention mechanism based on a dual feedforward network (FFN-Attention-FFN structure) to capture the global structural information of multiple views and obtain high-level features of the views. S4. The high-level features are passed to the graph learning module, and graph convolution operations are performed through the graph convolution network in the graph learning module to extract the local structural information of the multiple views, obtain the filtered features, and reconstruct the views based on the filtered features using the graph decoder, and introduce graph reconstruction loss. S5. In the reconstructed view, a multi-view contrast learning process is performed on the filtered features using a multi-view smoothing contrast loss. Gaussian noise terms are introduced into the information noise contrast estimation loss function and the semantic label-level contrast loss function, respectively. Finally, the MLP generates clustering assignment results.

[0007] Furthermore, in step S1, the imputation process for missing samples in the target view is as follows: First, calculate the pairwise distance between the feature vector of a given sample and the feature vectors of all other available samples in each available view of the incomplete multi-view system. : ; in, Indicates available views medium sample eigenvectors, Indicates available views medium sample eigenvectors, Indicates sample The There are 1 view. Indicates sample The A missing view exists; Then, select according to the following formula. The nearest neighbor pair distance: ; in, Indicates a missing view medium sample The nearest neighbor index, It is selected based on the distance set. Find the nearest neighbor operator; and then obtain the nearest neighbor index set. , Indicates available views The nearest neighbor index for all samples in the dataset; Finally, a multivariate Gaussian distribution describing the missing feature vectors is established. : ; ; in, For weighted average, For covariance, The weighting coefficient is inversely proportional to the distance. Indicates a missing view medium sample The eigenvectors, the parameters of the multivariate Gaussian distribution are derived from the nearest neighbor set. The corresponding features determine; Missing feature vectors are imputed by averaging multiple values ​​sampled from a multivariate Gaussian distribution.

[0008] Furthermore, in step S3, the FFN-Attention-FFN structure includes a first FFN layer, an attention layer, and a second FFN layer. The method for obtaining high-level features of the view using the FFN-Attention-FFN structure is as follows: First, the latent features are input into the first FFN layer for filtering; Then, the features passed through the first FFN layer are input into the attention layer, and mapped to different feature spaces through the weight matrices of query, key, and value to obtain the query matrix, key matrix, and value matrix respectively. Then, the score matrices of the query matrix and key matrix are obtained, and the calculation result is obtained by multiplying the score matrix and the value matrix. Finally, the calculation results are input into the second FFN layer to obtain the high-level features of the view.

[0009] Furthermore, in step S4, the expression for the graph convolution operation is: ; in, These are the filtered features. The convolutional layer order is the number of layers in the graph. These are parameters used to balance the influence of a node's self-information and its neighborhood information. , Indicates the first Normalized affinity matrix of graph convolutional layers, Indicates high-level features, Learnable parameters shared across all views.

[0010] Furthermore, in step S4, the graph decoder reconstructs the view in the following way: ; in, Indicates the first A graph structure reconstructed from individual views. For the first Learnable parameters for each view decoder; The expression for the graph reconstruction loss is as follows: ; in, This represents the graph reconstruction loss. For the number of views, For the first Adjacency matrix of views, index This represents the Frobenius norm.

[0011] Furthermore, in step S5, the method for introducing a Gaussian noise term into the information noise contrast estimation loss function is as follows: multiple Gaussian noise vectors with the same dimension as the feature vector are randomly sampled from the Gaussian cloth, and the multiple Gaussian noise vectors are paired with each sample to form a high-confidence negative sample pair, as shown below: ; ; in, Represents a view and Feature-level multi-view smooth contrast loss, For the sample size, For temperature parameters, This represents the similarity measurement function. , Views and The Middle Features of each sample For view The Middle Features of each sample This is the sum of the similarities of all feature-level negative sample pairs. This represents the number of Gaussian noise vectors involved in the computation during feature-level contrastive learning. For balancing parameters, , Views and The randomly generated number A Gaussian noise vector; Feature-level multi-view smooth contrast loss function for: .

[0012] Furthermore, in step S5, the Gaussian noise term introduced into the semantic label-level contrastive loss function is represented as follows: ; ; in, Represents a view and Semantic label-level multi-view smooth contrast loss, The number of clusters, It is the temperature coefficient. and Views and Cluster assignment vectors for the same semantic cluster, For view The Middle Cluster assignment vectors for each sample The sum of similarities for all semantically labeled negative sample pairs. It is the number of Gaussian noise vectors involved in the computation during semantic label-level contrastive learning; Semantic label-level multi-view smooth contrast loss function for: .

[0013] Compared with the shortcomings and deficiencies of existing technologies, the present invention has the following beneficial effects: 1. The incomplete multimodal smooth contrast clustering method based on information interpolation proposed in this invention introduces a view interpolation mechanism based on statistical information, models the missing view as a multivariate Gaussian distribution based on distance-weighted nearest neighbors, and improves the quality of data recovery by introducing distance weighting to reduce the influence of weakly correlated views on the completion result; 2. This invention proposes a framework that combines the FFN-Attention-FFN structure with a graph convolutional network. The FFN-Attention-FFN structure enhances the model's ability to extract features and effectively captures the global structure of the data. By constructing a graph in the latent feature space and combining it with a graph convolutional network to complete feature enhancement, the inner product decoder and graph reconstruction learning are used to ensure the preservation of structural information in the low-dimensional space, thereby making full use of local structural information. 3. This invention proposes an efficient smoothing strategy to address the problem of insufficient negative samples in contrastive learning. This strategy expands negative sample pairs by adding Gaussian noise, making the sample distribution smoother. Since this method is equivalent to directly adding high-confidence negative samples, it can improve the model's feature learning ability. Attached Figure Description

[0014] Figure 1 This is a flowchart of the incomplete multimodal smooth contrastive clustering method based on information interpolation provided in this embodiment of the invention; Figure 2 This is a schematic diagram of missing view interpolation based on statistical information provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the FFN-Attention-FFN structure provided in an embodiment of the present invention; Figure 4 This is the result of missing data recovery provided by the embodiments of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0016] The flowchart of the incomplete multimodal smooth contrastive clustering method based on information interpolation provided by this invention is as follows: Figure 1 As shown, firstly, in the view reconstruction module, a view interpolation mechanism based on statistical information is used to model the missing view as a multivariate Gaussian distribution based on distance-weighted nearest neighbors to improve the quality of data recovery. Then, an encoder and decoder are used to reconstruct the view and extract latent features. Secondly, in the FFN-Attention-FFN structure (an attention mechanism based on a dual feedforward network), the extracted latent features are refined through the FFN-Attention-FFN structure to capture the global structural information of multiple views and pass it to the graph learning module. Next, local structure extraction is performed in the graph learning module, and the view is reconstructed. Finally, a multilayer perceptron (MLP) is used to generate clustering assignment results.

[0017] The following is combined with Figure 1The present invention provides a detailed description of the incomplete multimodal smooth contrastive clustering method based on information interpolation.

[0018] 1. Missing view imputation based on statistical information When dealing with incomplete multi-view data, this invention imputes missing view data based on data from available views, which helps reduce the impact of incomplete data on clustering results.

[0019] use This represents an incomplete multi-view dataset, where, Indicates the number of samples. Indicates the number of views. Indicates available views The dimension. Furthermore. An index matrix indicating whether data is missing. Indicates the first The first sample If the view exists, then...; if it does not exist, then... .

[0020] For any missing view samples , using the existence of the first The missing information is imputed using data from other samples in the incomplete multi-view set. As shown in Figure 2, the imputation strategy is as follows: The view containing the missing sample in the incomplete multi-view set is used as the target view. First, the neighboring samples of the missing sample are determined in the available views of the incomplete multi-view set. This is done to impute the missing information. Figure 2 For missing data in sample A, the nearest neighbors are determined in the available view for imputation. Specifically, in the view... Figure 1 In the middle, sample B was found to be a neighbor, while in the view Figure 3 In the process, sample D was found to be a neighbor. Then, in the target view (view... Figure 2 Find the representation samples corresponding to these neighbor samples, namely samples F and G. Finally, construct a multivariate Gaussian distribution based on the features of samples F and G, and sample the view from this distribution. Figure 2 A new representation of sample A. It is worth noting that, like the view... Figure 1 Sample C and view Figure 3 Sample E in the image cannot be used in this process because they are in the view. Figure 2 It is also incomplete.

[0021] The formal representation of the above process is as follows: First, calculate the available views. medium sample eigenvectors With each available view All other available samples eigenvectors Pair distances between : ; in, Indicates sample The There are 1 view. Indicates sample The A missing view exists; Then, select according to the following formula. The nearest neighbor pair distance: ; in, Indicates a missing view medium sample The nearest neighbor index, It is selected based on the distance set. Find the nearest neighbor operator; and then obtain the nearest neighbor index set. , Indicates available views The nearest neighbor index for all samples in the dataset; Finally, assuming missing feature vectors Multivariate Gaussian distribution can be used The description, whose parameters are derived from the nearest neighbor set. The corresponding features determine the weight coefficients, which are inversely proportional to the distance. The resulting weighted average Covariance as follows: ; ; in, Indicates a missing view medium sample eigenvectors; The missing feature vectors in the target view are imputed by averaging multiple values ​​sampled from a multivariate Gaussian distribution. This process is repeated for all missing values ​​to obtain a complete multi-view dataset. .

[0022] 2. In-view reconstruction module After obtaining the complete multi-view dataset, a key challenge in the unsupervised IMVC task is how to extract discriminative features from views with significant dimensionality differences and learn a consistent representation for clustering. Given the superior performance of autoencoders in unsupervised feature extraction, this approach introduces a view-specific encoder for multi-view data. Specifically, the view-specific encoder... It is a fully connected network that maps the obtained full multi-view dataset as input data to the latent space and extracts latent features. Then use the decoder Reconstruct the original data based on potential features.

[0023] To better capture information and reduce information loss, a reconstruction loss is introduced, which is the total reconstruction goal of all views. The expression is: ; Through view-specific encoders Latent features can be obtained. However, relying solely on an autoencoder framework has a significant limitation: the process tends to capture only view-specific information, ignoring the global data structure, which is often crucial for downstream tasks. To address this issue, the latent feature extraction stage of this invention is implemented by integrating a view-specific encoder with a subsequent FFN-Attention-FFN structure.

[0024] 3. FFN-Attention-FFN structure To better capture the global structure of the data, an FFN-Attention-FFN structure is used to obtain high-level features. For example... Figure 3 As shown, the FFN-Attention-FFN structure includes a first FFN layer (feedforward layer), an attention layer, and a second FFN layer (feedforward layer). The method for obtaining high-level features of the view using the FFN-Attention-FFN structure is as follows: First, latent features The first FFN layer is used as a filter to extract more important features for subsequent attention layers, laying the groundwork for capturing the global structure. This approach helps the model learn more comprehensive and richer feature representations. Then, the features passed through the first FFN layer are input into the attention layer, and the weight matrix is ​​queried. Key weight matrix Value weight matrix Mapping to different feature spaces yields query matrices respectively. Key matrix Sum matrix : ; Thus, it can be achieved get and The score matrix, where It is the feature dimension input from the attention layer; multiplying the score matrix by the value matrix yields... ; Finally, the calculation results After inputting the second FFN layer, obtain the advanced features of the view. : ; in, This represents a self-attention layer. While the FFN-Attention-FFN structure can effectively capture global structure, linear MLPs alone are insufficient to capture local structural information of features. To address this issue, this invention proposes using graph convolutional networks to fully discover the local structure hidden in multi-view data and achieve better clustering performance.

[0025] 4. Graph Learning Module To fully extract local structural information from multiple views, a GCN encoder is assigned to each view. Specifically, an adjacency matrix is ​​first constructed for each view. To characterize the adjacency relationship between samples, for each sample feature Calculate its Nearest neighbor set And based on this, the affinity matrix is ​​obtained. .like ,but ;otherwise, .make Indicates and The corresponding degree matrix, where express The degree. The normalized affinity matrix is ​​defined as... ,in for An identity matrix of order 1.

[0026] To obtain smooth feature representations that are beneficial for clustering, low-pass filtering is implemented using graph convolution operations. For the ... Advanced features of each view The basic graph convolution method can be written as: ; in, These are the filtered features. Indicates the first Normalized affinity matrix of graph convolutional layers, The convolutional layer order is the number of layers in the graph. These are learnable parameters. However, as the order of the graph convolutional layer increases, Over-smoothing can easily occur. To mitigate this problem, low-pass filtering of the features is implemented using the following method: ; in, Used to balance the influence of a node's own information and its neighborhood information. These are learnable parameters shared across all views. To more effectively capture local correlations between different samples, a graph reconstruction loss is introduced. This is because the representation extracted by the above formula... Including structural information and content features of each view, an inner product-based decoding mechanism was designed to predict the affinity between instances. The graph decoder for each view is defined as follows: ; in, Indicates the first A graph structure reconstructed from individual views. For the first Learnable parameters for each view decoder; To enable the model to better capture the local structure of the data, minimize the graph reconstruction loss for each view: ; in, Indicates graph reconstruction loss, subscript This represents the Frobenius norm.

[0027] 5. Multi-view comparison learning (1) Feature-level contrastive learning In incomplete multi-view clustering tasks, the key objective is to ensure that the similarity between feature representations of the same sample across different views (i.e., positive sample pairs) is significantly higher than the similarity between different samples (i.e., negative sample pairs). In the feature space, contrastive learning is used to further achieve the consistency objective, making... The focus is on learning the common semantics of all views. However, traditional contrastive learning-based methods may mistakenly identify similar samples that should belong to the same cluster as negative sample pairs, resulting in a lack of high-confidence negative samples. To address this issue, this invention introduces a multi-view smoothing contrastive loss to enhance the discriminative power of features in multi-view learning. Specifically, a Gaussian noise term is added to the information-noise contrastive estimation loss function to generate high-confidence negative samples. Specifically, given the following Gaussian distribution: The mean is The variance is Randomly sample from this distribution with the same dimension as the feature vector. We generate Gaussian noise vectors and pair these noise vectors with each sample to form high-confidence negative sample pairs, thereby improving the effect of contrastive learning, as shown in the following formula: ; ; in, Represents a view and Feature-level multi-view smooth contrast loss, For the sample size, For temperature parameters, This represents the similarity metric function, typically a cosine similarity function or a dot product. , Views and The Middle Features of each sample For view The Middle Features of each sample This is the sum of the similarities of all feature-level negative sample pairs. This represents the number of Gaussian noise vectors involved in the computation during feature-level contrastive learning. For balancing parameters, , Views and The randomly generated number A Gaussian noise vector; Feature-level multi-view smooth contrast loss function for: .

[0028] (2) Semantic label-level comparative learning Features in each view A view-specific label MLP is stacked on top of the initial clustering assignments. The last layer of the MLP uses a softmax function to output probabilities. To further increase the diversity among clustering assignments, a target distribution is used. It is considered A good alternative, where each element is calculated as follows: ; in, Each element in Indicates sample To clustering The soft clustering assignment vector. Therefore, The Listing This corresponds to the clustering assignment vector belonging to the same semantic cluster. Instances from different views originating from the same entity should share consistent semantic information, and their corresponding clustering assignment vectors (e.g., ...) should also be consistent. and They are respectively views and The Middle The cluster assignment vectors of each sample should have high similarity. The inner product is used to measure the similarity between any two cluster assignment vectors: ; The goal of contrastive learning is to maximize the similarity of positive sample pairs while minimizing the similarity of negative sample pairs. To avoid the problem of insufficient high-confidence negative samples, a Gaussian noise term is introduced into the semantic label-level contrastive loss. and The loss between them is defined as follows: ; ; in, Represents a view and Semantic label-level multi-view smooth contrast loss, The number of clusters, It is the temperature coefficient. and Views and Cluster assignment vectors for the same semantic cluster, The sum of similarities for all semantically labeled negative sample pairs. This represents the number of Gaussian noise vectors involved in the computation during semantic label-level contrastive learning. It's worth noting that the value of each element in the cluster assignment is [0,1], and the value of the generated Gaussian vector also corresponds to this. Semantic label-level multi-view smooth contrast loss function for: .

[0029] Since multi-view clustering requires learning general semantic labels from the feature representations of multi-view data, this invention will use soft-assignment distribution. Defined as a matrix The Okay. After the model training is complete, for each sample... Final semantic label prediction as follows: .

[0030] in, Represents a soft-assigned distribution The Each element.

[0031] 6. Using the incomplete multimodal smooth contrastive clustering method based on information interpolation proposed in this invention, the following methods are applied: Figure 4 The original image of the missing view shown in the upper middle row is restored, and the restored corresponding view is as follows. Figure 4The views shown in the lower middle row demonstrate that the restored views can capture the essential features of the original objects, such as the shape of a T-shirt or handbag. This indicates that the present invention has the ability to restore missing views with high quality. This high-quality data restoration is crucial for subsequent representation learning and model training, providing a more complete and reliable data foundation.

[0032] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An incomplete multimodal smooth contrastive clustering method based on information interpolation, characterized in that, Includes the following steps: S1. Obtain multi-view image data. Take the view with missing samples in the incomplete multi-view as the target view. First, determine the neighbor samples of the missing samples in the available views of the incomplete multi-view. Then, find the representation samples corresponding to the neighbor samples in the target view. Finally, establish a multivariate Gaussian distribution based on the features of the representation samples, and sample the average of multiple values ​​from the multivariate Gaussian distribution as the missing value to imput the missing samples in the target view, so as to obtain the dataset of the complete multi-view. The imputation process for missing samples in the target view is as follows: First, calculate the pairwise distance between the feature vector of a given sample and the feature vectors of all other available samples in each available view of the incomplete multi-view system. : ； in, Indicates available views medium sample eigenvectors, Indicates available views medium sample eigenvectors, Indicates sample The There are 1 view. Indicates sample The A missing view exists; Then, select according to the following formula. The nearest neighbor pair distance: ； in, Indicates a missing view medium sample The nearest neighbor index, It is selected based on the distance set. Find the nearest neighbor operator; and then obtain the nearest neighbor index set. , Indicates available views The nearest neighbor index for all samples in the dataset; Finally, a multivariate Gaussian distribution describing the missing feature vectors is established. : ； ； in, For weighted average, For covariance, The weighting coefficient is inversely proportional to the distance. Indicates a missing view medium sample The eigenvectors, the parameters of the multivariate Gaussian distribution are derived from the nearest neighbor set. The corresponding features determine; The missing feature vector is imputed by averaging multiple values ​​sampled from a multivariate Gaussian distribution. S2. Use a view-specific encoder to extract latent features from the dataset; S3. The latent features are refined through the FFN-Attention-FFN structure to capture the global structural information of multiple views and obtain the high-level features of the views; S4. The high-level features are passed to the graph learning module, and graph convolution operations are performed through the graph convolution network in the graph learning module to extract the local structural information of the multiple views, obtain the filtered features, and reconstruct the views based on the filtered features using the graph decoder, and introduce graph reconstruction loss. S5. In the reconstructed view, MLP is used to perform multi-view contrast learning on the filtered features, and multi-view smooth contrast loss is introduced. Gaussian noise terms are introduced into the information noise contrast estimation loss function and the semantic label level contrast loss function, respectively. Finally, MLP generates clustering assignment results.

2. The incomplete multimodal smooth contrastive clustering method based on information interpolation as described in claim 1, characterized in that, In step S3, the FFN-Attention-FFN structure includes a first FFN layer, an attention layer, and a second FFN layer. The method for obtaining high-level features of the view using the FFN-Attention-FFN structure is as follows: First, the latent features are input into the first FFN layer for filtering; Then, the features passed through the first FFN layer are input into the attention layer, and mapped to different feature spaces through the weight matrices of query, key, and value to obtain the query matrix, key matrix, and value matrix respectively. Then, the score matrices of the query matrix and key matrix are obtained, and the calculation result is obtained by multiplying the score matrix and the value matrix. Finally, the calculation results are input into the second FFN layer to obtain the high-level features of the view.

3. The incomplete multimodal smooth contrastive clustering method based on information interpolation as described in claim 1, characterized in that, In step S4, the expression for the graph convolution operation is: ； in, These are the filtered features. The convolutional layer order is the number of layers in the graph. These are parameters used to balance the influence of a node's self-information and its neighborhood information. , Indicates the first Normalized affinity matrix of graph convolutional layers, Indicates high-level features, Learnable parameters shared across all views.

4. The incomplete multimodal smooth contrastive clustering method based on information interpolation as described in claim 3, characterized in that, In step S4, the graph decoder reconstructs the view in the following way: ； in, Indicates the first A graph structure reconstructed from individual views. For the first Learnable parameters for each view decoder; The expression for the graph reconstruction loss is as follows: ； in, This represents the graph reconstruction loss. For the number of views, For the first Adjacency matrix of views, index This represents the Frobenius norm.

5. The incomplete multimodal smooth contrastive clustering method based on information interpolation as described in claim 1, characterized in that, In step S5, the method for introducing a Gaussian noise term into the information noise contrast estimation loss function is as follows: multiple Gaussian noise vectors with the same dimension as the feature vector are randomly sampled from the Gaussian cloth, and these multiple Gaussian noise vectors are paired with each sample to form a high-confidence negative sample pair, as shown below: ； ； in, Represents a view and Feature-level multi-view smooth contrast loss, For the sample size, For temperature parameters, This represents the similarity measurement function. , Views and The Middle Features of each sample For view The Middle Features of each sample This is the sum of the similarities of all feature-level negative sample pairs. This represents the number of Gaussian noise vectors involved in the computation during feature-level contrastive learning. For balancing parameters, , Views and The randomly generated number A Gaussian noise vector; Feature-level multi-view smooth contrast loss function for: 。 6. The incomplete multimodal smooth contrastive clustering method based on information interpolation as described in claim 1, characterized in that, In step S5, the Gaussian noise term introduced into the semantic label-level contrastive loss function is represented as follows: ； ； in, Represents a view and Semantic label-level multi-view smooth contrast loss, The number of clusters, It is the temperature coefficient. This represents the similarity measurement function. and Views and Cluster assignment vectors for the same semantic cluster, For view The Middle Cluster assignment vectors for each sample The sum of similarities for all semantically labeled negative sample pairs. It is the number of Gaussian noise vectors involved in the computation during semantic label-level contrastive learning. For balancing parameters, , Views and The randomly generated number A Gaussian noise vector; Semantic label-level multi-view smooth contrast loss function for: 。

Images