Incomplete multi-mode smooth contrast clustering method based on information interpolation

Abstract

The invention discloses an incomplete multi-mode smooth contrast clustering method based on information interpolation, which comprises the following steps of: firstly, modeling a missing view into multivariate Gaussian distribution based on distance weighted nearest neighbor by utilizing a view interpolation mechanism based on statistical information, and carrying out interpolation on the missing view to obtain a complete multi-view data set; secondly, using an encoder to extract potential features, capturing global structure information of multiple views through an FFN-Attention-FFN structure, and extracting local structure information of the multiple views through a graph convolutional network; thirdly, reconstructing a view by using a graph decoder; and finally, generating a clustering distribution result by using the MLP. According to the method, a view interpolation mechanism based on statistical information is introduced, the influence of a weak correlation view on a completion result is reduced by introducing distance weighting, and the data recovery quality is improved; the FFN-Attention-FFN structure is combined with the graph convolutional network, so that the global and local structure information of the data can be effectively captured; and negative sample pairs are expanded by adding Gaussian noise, so that sample distribution is smoother.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of multi-view clustering, and particularly relates to an incomplete multi-modal smooth contrast clustering method based on information interpolation. BACKGROUND

[0002] With the development of data acquisition technology, information of the same object can usually be obtained from multiple different sources to form multi-view data. Multi-view clustering (MVC) as an unsupervised learning paradigm, groups unlabeled instances into different clusters by utilizing consistent information and complementary information between multi-view data. However, in actual application scenarios, due to factors such as device failure and sensor error, some view information in multi-view data may be missing, resulting in incomplete data. For example, in medical diagnosis, patients may not undergo systematic examination due to various reasons, resulting in missing information. These problems make many existing MVC methods unable to function properly, and therefore, incomplete multi-view clustering (IMVC) technology has developed.

[0003] Deep neural networks are widely used in computer vision and pattern recognition due to their strong representation learning ability and scalability. In order to further improve clustering performance, researchers have begun to combine deep neural networks with IMVC methods and proposed incomplete multi-view clustering methods based on deep learning frameworks, such as methods based on generative adversarial networks (GANs), methods based on Transformers, methods based on graph convolution networks (GCNs), and methods based on contrast learning.

[0004] However, existing deep incomplete multi-view clustering methods still have the following defects: 1. Methods based on generative adversarial networks (GANs) can handle missing data filling problems by utilizing the data generation ability of GANs, but the data generation process is difficult to explain, and the generated missing data may lack intuitive interpretability, increasing the uncertainty of the results; 2. Methods based on Transformers or graph convolution networks (GCNs) only focus on global or local structures of data, making it difficult to efficiently utilize both types of information, resulting in insufficient utilization of feature information contained in the data; 3. Methods based on contrast learning: similar samples (positive sample pairs) are clustered together in the latent space, and dissimilar samples (negative sample pairs) are dispersed, but this method lacks an effective strategy for selecting and utilizing negative samples, resulting in a scarcity of high-confidence negative samples, affecting the discriminability of the model and the clustering accuracy. SUMMARY

[0005] In view of the problems in the background art, the purpose of the present application is to provide an incomplete multi-modal smooth contrast clustering method based on information interpolation.

[0006] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions: An incomplete multi-modal smooth contrast clustering method based on information interpolation comprises the following steps: S1, taking the missing sample view in the incomplete multi-view as the target view, first determining the neighbor sample of the missing sample in the available view of the incomplete multi-view, then finding the representation sample corresponding to the neighbor sample in the target view, and finally establishing a multivariate Gaussian distribution according to the features of the representation sample, and sampling the average value of multiple values from the multivariate Gaussian distribution as the missing value to interpolate the missing sample in the target view, obtaining a complete multi-view dataset; S2, using a view-specific encoder to extract the latent features in the dataset; S3, the latent features are refined by an attention mechanism based on a double feedforward network (FFN-Attention-FFN structure) to capture global structural information of the multi-view, obtaining high-level features of the view; S4, the high-level features are passed to a graph learning module, and graph convolution operation processing is performed through a graph convolution network in the graph learning module to extract local structural information of the multi-view, obtaining filtered features, and using a graph decoder to reconstruct the view according to the filtered features and introducing a graph reconstruction loss; S5, using a multi-layer perceptron (MLP) in the reconstructed view to perform multi-view contrast learning on the filtered features, and introducing a multi-view smooth contrast loss, and introducing a Gaussian noise term in the information noise contrast estimation loss function and the semantic label level contrast loss function respectively; finally, the MLP generates a clustering assignment result.

[0007] Further, in step S1, the interpolation process of the missing sample in the target view is as follows: First, calculate the pairwise distance between the feature vector of a sample and the feature vectors of all other available samples in each available view of the incomplete multi-view : ; Wherein, represents the feature vector of the sample in the available view , represents the feature vector of the sample in the available view , represents the existence of the th view of the sample , represents the feature vector of the sample , represents the feature vector of the sample The A missing view exists; Then, select according to the following formula. The nearest neighbor pair distance: ; in, Indicates a missing view medium sample The nearest neighbor index, It is selected based on the distance set. Find the nearest neighbor operator; and then obtain the nearest neighbor index set. , Indicates available views The nearest neighbor index for all samples in the dataset; Finally, a multivariate Gaussian distribution describing the missing feature vectors is established. : ; ; in, For weighted average, For covariance, The weighting coefficient is inversely proportional to the distance. Indicates a missing view medium sample The eigenvectors, the parameters of the multivariate Gaussian distribution are derived from the nearest neighbor set. The corresponding features determine; Missing feature vectors are imputed by averaging multiple values ​​sampled from a multivariate Gaussian distribution.

[0008] Furthermore, in step S3, the FFN-Attention-FFN structure includes a first FFN layer, an attention layer, and a second FFN layer. The method for obtaining high-level features of the view using the FFN-Attention-FFN structure is as follows: First, the latent features are input into the first FFN layer for filtering; Then, the features passed through the first FFN layer are input into the attention layer, and mapped to different feature spaces through the weight matrices of query, key, and value to obtain the query matrix, key matrix, and value matrix respectively. Then, the score matrices of the query matrix and key matrix are obtained, and the calculation result is obtained by multiplying the score matrix and the value matrix. Finally, the calculation results are input into the second FFN layer to obtain the high-level features of the view.

[0009] Furthermore, in step S4, the expression for the graph convolution operation is: ; in, is the filtered feature, is the number of graph convolutional layers, is the parameter for balancing the influence of node self-information and its neighborhood information, , denotes the normalized affinity matrix of the th graph convolutional layer, denotes the high-level feature, is the shared learnable parameter across views.

[0010] Further, in step S4, the way the graph decoder reconstructs the views is: ; wherein, denotes the graph structure of the th view reconstruction, is the learnable parameter of the th view decoder; The expression of the graph reconstruction loss is as follows: ; wherein, denotes the graph reconstruction loss, is the number of views, is the adjacency matrix of the th view, and the subscript denotes the Frobenius norm.

[0011] Further, in step S5, the method for introducing a Gaussian noise term in the information noise contrast estimation loss function is: randomly sampling multiple Gaussian noise vectors consistent with the feature vector dimension from a Gaussian distribution, and pairing the multiple Gaussian noise vectors with each sample to form a high-confidence negative sample pair, denoted as follows: ; ; wherein, denotes the feature-level multi-view smooth contrast loss of views and , is the number of samples, is the temperature parameter, denotes the similarity measure function, , are the features of the th sample in views and , is the feature of the th sample in view , a sum of similarities of all feature-level negative sample pairs, a number of Gaussian noise vectors participating in calculation in feature-level contrast learning, a balance parameter, , are respectively view and the first Gaussian noise vector randomly generated; feature-level multi-view smooth contrast loss function is: .

[0012] Further, in step S5, the Gaussian noise term introduced in the semantic label-level contrast loss function is represented as follows: ; ; Wherein, indicates the semantic label-level multi-view smooth contrast loss of view and , is the number of clusters, is a temperature coefficient, and are respectively the cluster assignment vector of the same semantic cluster in view and , is the cluster assignment vector of the first sample in view , a sum of similarities of all semantic label-level negative sample pairs, is the number of Gaussian noise vectors participating in calculation in semantic label-level contrast learning; semantic label-level multi-view smooth contrast loss function is: .

[0013] Compared with the disadvantages and deficiencies of the prior art, the present application has the following beneficial effects: 1. The incomplete multi-modal smooth contrast clustering method based on information interpolation proposed in the present application introduces a view interpolation mechanism based on statistical information, models the missing view as a multivariate Gaussian distribution based on distance weighted nearest neighbor, reduces the influence of weakly related views on the completion result by introducing distance weighting, and improves the quality of data recovery. 2.The application proposes a framework combining FFN-Attention-FFN structure and graph convolution network, enhances the ability of the model to extract features through FFN-Attention-FFN structure, and effectively captures the global structure of the data; the graph is constructed in the latent feature space and combined with the graph convolution network to complete feature enhancement, the inner product decoder and graph reconstruction learning are used to ensure the reservation of structure information in the low-dimensional space, so as to fully utilize the local structure information; 3.The application proposes an efficient smoothing strategy for the problem of insufficient negative samples in contrast learning, which expands the negative sample pair by adding Gaussian noise to make the sample distribution more smooth. Since this method is equivalent to directly adding negative samples with high confidence, it can improve the feature learning ability of the model. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is a flowchart of an incomplete multi-modal smooth contrast clustering method based on information interpolation provided by an embodiment of the application; Figure 2 is a missing view interpolation schematic diagram based on statistical information provided by an embodiment of the application; Figure 3 is a schematic diagram of FFN-Attention-FFN structure provided by an embodiment of the application; Figure 4 is a missing data recovery result provided by an embodiment of the application. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below in combination with specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application.

[0016] The flowchart of the incomplete multi-modal smooth contrast clustering method based on information interpolation provided by the application is shown in Figure 1 Firstly, in the view reconstruction module, the view interpolation mechanism based on statistical information is used to model the missing view as a multivariate Gaussian distribution based on distance weighted nearest neighbor to improve the quality of data recovery; then the encoder and decoder are used to reconstruct the view and extract the latent feature. Secondly, in the FFN-Attention-FFN structure (attention mechanism based on double feedforward network), the extracted latent feature is refined through the FFN-Attention-FFN structure to capture the global structure information of the multi-view and is transmitted to the graph learning module. Thirdly, the local structure is extracted in the graph learning module, and the view is reconstructed. Finally, the multi-layer perception (MLP) is used to generate the clustering assignment result.

[0017] The following will be described in combination with Figure 1The present invention provides a detailed description of the incomplete multimodal smooth contrastive clustering method based on information interpolation.

[0018] 1. Missing view imputation based on statistical information When dealing with incomplete multi-view data, this invention imputes missing view data based on data from available views, which helps reduce the impact of incomplete data on clustering results.

[0019] use This represents an incomplete multi-view dataset, where, Indicates the number of samples. Indicates the number of views. Indicates available views The dimension. Furthermore. An index matrix indicating whether data is missing. Indicates the first The first sample If the view exists, then...; if it does not exist, then... .

[0020] For any missing view samples , using the existence of the first The missing information is imputed using data from other samples in the incomplete multi-view set. As shown in Figure 2, the imputation strategy is as follows: The view containing the missing sample in the incomplete multi-view set is used as the target view. First, the neighboring samples of the missing sample are determined in the available views of the incomplete multi-view set. This is done to impute the missing information. Figure 2 For missing data in sample A, the nearest neighbors are determined in the available view for imputation. Specifically, in the view... Figure 1 In the middle, sample B was found to be a neighbor, while in the view Figure 3 In the process, sample D was found to be a neighbor. Then, in the target view (view... Figure 2 Find the representation samples corresponding to these neighbor samples, namely samples F and G. Finally, construct a multivariate Gaussian distribution based on the characteristics of samples F and G, and sample the view from this distribution. Figure 2 A new representation of sample A. It is worth noting that, like the view... Figure 1 Sample C and view Figure 3 Sample E in the image cannot be used in this process because they are in the view. Figure 2 It is also incomplete.

[0021] The formal representation of the above process is as follows: First, calculate the available views. medium sample eigenvectors With each available view All other available samples eigenvectors Pair distances between : ; in, Indicates sample The There are 1 view. Indicates sample The A missing view exists; Then, select according to the following formula. The nearest neighbor pair distance: ; in, Indicates a missing view medium sample The nearest neighbor index, It is selected based on the distance set. Find the nearest neighbor operator; and then obtain the nearest neighbor index set. , Indicates available views The nearest neighbor index for all samples in the dataset; Finally, assuming missing feature vectors Multivariate Gaussian distribution can be used The description, whose parameters are derived from the nearest neighbor set. The corresponding features determine the weight coefficients, which are inversely proportional to the distance. The resulting weighted average Covariance as follows: ; ; in, Indicates a missing view medium sample eigenvectors; The missing feature vectors in the target view are imputed by averaging multiple values ​​sampled from a multivariate Gaussian distribution. This process is repeated for all missing values ​​to obtain a complete multi-view dataset. .

[0022] 2. In-view reconstruction module After obtaining the complete multi-view dataset, a key challenge in the unsupervised IMVC task is how to extract discriminative features from views with significant dimensionality differences and learn a consistent representation for clustering. Given the superior performance of autoencoders in unsupervised feature extraction, this approach introduces a view-specific encoder for multi-view data. Specifically, the view-specific encoder... It is a fully connected network that maps the obtained full multi-view dataset as input data to the latent space and extracts latent features. Then use the decoder Reconstruct the original data based on potential features.

[0023] To better capture information and reduce information loss, a reconstruction loss is introduced, which is the total reconstruction goal of all views. The expression is: ; Through view-specific encoders Latent features can be obtained. However, relying solely on an autoencoder framework has a significant limitation: the process tends to capture only view-specific information, ignoring the global data structure, which is often crucial for downstream tasks. To address this issue, the latent feature extraction stage of this invention is implemented by integrating a view-specific encoder with a subsequent FFN-Attention-FFN structure.

[0024] 3. FFN-Attention-FFN structure To better capture the global structure of the data, an FFN-Attention-FFN structure is used to obtain high-level features. For example... Figure 3 As shown, the FFN-Attention-FFN structure includes a first FFN layer (feedforward layer), an attention layer, and a second FFN layer (feedforward layer). The method for obtaining high-level features of the view using the FFN-Attention-FFN structure is as follows: First, latent features The first FFN layer is used as a filter to extract more important features for subsequent attention layers, laying the groundwork for capturing the global structure. This approach helps the model learn more comprehensive and richer feature representations. Then, the features passed through the first FFN layer are input into the attention layer, and the weight matrix is ​​queried. Key weight matrix Value weight matrix Mapping to different feature spaces yields query matrices respectively. Key matrix Sum matrix : ; Thus, it can be achieved get and The score matrix, where It is the feature dimension input from the attention layer; multiplying the score matrix by the value matrix yields... ; Finally, the calculation result Input the second FFN layer to obtain the high-level features of the view : ; wherein, represents a self-attention layer. Although the FFN-Attention-FFN structure can effectively capture the global structure, using only the linear MLP is insufficient to capture the local structure information of the features. In order to solve this problem, the present application proposes to use a graph convolution network to fully discover the local structure hidden in the multi-view data and obtain better clustering performance.

[0025] 4, graph learning module In order to fully excavate the local structure information in the multi-view, a GCN encoder is assigned to each view. Specifically, first, an adjacency matrix is constructed for each view to represent the adjacency relationship between samples, for each sample feature , its neighborhood set is calculated, and the affinity matrix is obtained accordingly. If , ; otherwise, . Let denote the degree matrix corresponding to , where denotes the degree of . The normalized affinity matrix is defined as , where is an order unit matrix.

[0026] In order to obtain a smooth feature representation that is conducive to clustering, a low-pass filter is implemented using a graph convolution operation. For the high-level feature of the i-th view , the basic graph convolution method can be written as: ; wherein, is the filtered feature, denotes the normalized affinity matrix of the i-th graph convolution layer, is the number of graph convolution layers, is a learnable parameter. However, with the increase of the number of graph convolution layers, over-smoothing is prone to occur. In order to alleviate this problem, the following method is used to implement low-pass filtering of the features: ; wherein,​ to balance the influence of node self-information and its neighborhood information, shared learnable parameters for each view. To capture the local correlation between different samples more effectively, a graph reconstruction loss is introduced. Since the representation contains the structure information and content features of each view, a decoding mechanism based on inner product is designed to predict the affinity between instances. The graph decoder of the i-th view is defined as: ; where, denotes the reconstructed graph structure of the i-th view, is the learnable parameter of the i-th view decoder; To make the model better capture the local structure of the data, minimize the graph reconstruction loss of each view: ; where, denotes the graph reconstruction loss, and the subscript i represents the i-th view. denotes the Frobenius norm. 5. Multi-view contrastive learning (1) Feature-level contrastive learning

[0027] In the incomplete multi-view clustering task, the key goal is to make the similarity between the feature representations of the same sample in different views (i.e., positive sample pairs) significantly higher than that between different samples (i.e., negative sample pairs). In the feature space, the consistency goal is further achieved through contrastive learning, so that focuses on learning the common semantics of all views. However, traditional contrastive learning-based methods may mistakenly consider similar samples that should belong to the same cluster as negative sample pairs, resulting in a lack of high-confidence negative samples. To solve this problem, the present invention introduces a multi-view smooth contrastive loss to enhance the discriminative ability of features in multi-view learning, i.e., a Gaussian noise term is added to the information noise contrast estimation loss function to generate high-confidence negative samples. Specifically, given the following Gaussian distribution: with mean and variance . Randomly sample Gaussian noise vectors consistent with the dimension of the feature vector from this distribution, and pair these noise vectors with each sample to form high-confidence negative sample pairs to improve the effect of contrastive learning, as shown in the following formula: ; ; ; where, denotes the view​ and feature-level multi-view smooth contrastive loss, is the number of samples, is the temperature parameter, denotes the similarity measure function, usually cosine similarity function or dot product, , are the features of the th sample in view and , respectively. is the feature of the th sample in view , is the sum of similarities of all feature-level negative sample pairs, is the number of Gaussian noise vectors involved in the calculation of feature-level contrastive learning, is the balance parameter, , are the th randomly generated Gaussian noise vectors in view and , respectively. Feature-level multi-view smooth contrastive loss function is: .

[0028] (2) Semantic label-level contrastive learning Stack a view-specific label MLP on the feature of each view, and the last layer of the MLP uses the softmax function to output the probability to obtain the initial clustering assignment result. In order to further increase the diversity between clustering assignments, the target distribution is considered a good alternative to , where each element is calculated as follows: ; where, each element in denotes the soft clustering assignment vector of sample to cluster . Therefore, the th column of is recorded as , which corresponds to the clustering assignment vector belonging to the same semantic cluster. The instances under different views derived from the same entity should share consistent semantic information, and the corresponding clustering assignment vectors (e.g. and , which are the features of the th sample in view and , respectively.The cluster assignment vectors of different samples should have high similarity. The inner product is used to measure the similarity between any two cluster assignment vectors: ; The goal of contrastive learning is to maximize the similarity of positive sample pairs while minimizing the similarity of negative sample pairs. To avoid the problem of insufficient high-confidence negative samples, a Gaussian noise term is introduced in the semantic label-level contrastive loss. And The loss between ; ; Wherein, Indicates the semantic label-level multi-view smooth contrastive loss of view And , Is the number of clusters, Is the temperature coefficient, And The cluster assignment vectors of the same semantic cluster in view And , The sum of the similarities of all semantic label-level negative sample pairs, Is the number of Gaussian noise vectors involved in the calculation in semantic label-level contrastive learning. It is worth noting that the value of each element in the cluster assignment is [0, 1], and the value of the generated Gaussian vector is also consistent with it; The semantic label-level multi-view smooth contrastive loss function Is: .

[0029] Since multi-view clustering needs to learn general semantic labels from the feature representation of multi-view data, the soft assignment distribution Is defined as the Row of matrix After the model training is completed, the final semantic label prediction Of each sample Is as follows: .

[0030] Wherein, Indicates the Element of the soft assignment distribution .

[0031] 6、The incomplete multi-modal smooth contrastive clustering method based on information interpolation proposed in the application is used to restore the original image of the missing view shown in the upper row of Figure 4 The corresponding view after restoration is as shown in Figure 4The middle and lower rows of views show that the recovered views can capture the essential features of the original objects, such as the shape of the T-shirt or the handbag, indicating that the application has the ability to recover missing views with high quality, which is crucial for subsequent feature learning and model training, providing a more perfect and reliable data basis.

[0032] The above merely describes preferred embodiments of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An incomplete multi-modal smooth contrast clustering method based on information imputation, characterized in that, The method comprises the following steps: S1, taking the view with missing samples in the incomplete multi-view as a target view, first determining the neighbor samples of the missing samples in the available views of the incomplete multi-view, then finding the corresponding representation samples of the neighbor samples in the target view, and finally establishing a multivariate Gaussian distribution according to the characteristics of the representation samples, sampling the average value of multiple values from the multivariate Gaussian distribution as the missing value to interpolate the missing samples in the target view, and obtaining the data set of the complete multi-view; S2, extracting the latent features in the data set using a view-specific encoder; S3, refining the latent features through an FFN-Attention-FFN structure to capture the global structural information of the multi-view and obtain high-level features of the view; S4, passing the high-level features to a graph learning module, performing graph convolution operation processing through the graph convolution network in the graph learning module, extracting the local structural information of the multi-view, obtaining filtered features, reconstructing the view according to the filtered features using a graph decoder, and introducing a graph reconstruction loss; S5, performing multi-view contrast learning on the filtered features using an MLP in the reconstructed view, introducing a multi-view smooth contrast loss, introducing a Gaussian noise term in an information noise contrast estimation loss function and a semantic label level contrast loss function, and finally generating a clustering assignment result by the MLP.

2. The incomplete multi-modal smooth contrast clustering method based on information imputation as claimed in claim 1, wherein, In step S1, the interpolation process of the missing samples in the target view is as follows: First, the pair-wise distances between the feature vector of a sample and the feature vectors of all other available samples in each available view in the incomplete multi-view are computed : ； wherein, represents a view available in the sample , represents a view available in the sample , represents that the th view of the sample exists, represents that the th missing view of the sample exists; Then, the nearest neighbor pair distance is chosen according to the following formula D = min (d1, d2, d3, d4) ； wherein, denotes the missing view nearest neighbor indicator for a sample in is an operator that selects nearest neighbors from the distance set; and thereby obtains a set of nearest neighbor indicators , denotes the available views nearest neighbor indicators for all samples in​ Finally, a multivariate Gaussian distribution describing the missing feature vector is established : ； ； wherein, is a weighted average, is a covariance, is a weight coefficient inversely proportional to the distance, denotes a missing view a sample a feature vector, the parameters of the multivariate Gaussian distribution being determined by the corresponding features of the set of neighbors . The missing feature vector is interpolated by the average value of multiple values sampled from the multivariate Gaussian distribution.

3. The incomplete multi-modal smooth contrast clustering method based on information imputation as claimed in claim 1, wherein, In step S3, the FFN-Attention-FFN structure comprises a first FFN layer, an attention layer and a second FFN layer, and the method for obtaining view high-level features by the FFN-Attention-FFN structure is as follows: First, input the latent features into the first FFN layer for filtering processing; Then, input the features after the first FFN layer into the attention layer, map them to different feature spaces through the weight matrix of query, key and value, respectively obtain the query matrix, key matrix and value matrix, and then obtain the score matrix of the query matrix and the key matrix, and obtain the calculation result by multiplying the score matrix with the value matrix; Finally, input the calculation result into the second FFN layer to obtain the high-level features of the view.

4. The incomplete multi-modal smooth contrast clustering method based on information imputation as claimed in claim 1, wherein, In step S4, the expression of the graph convolution operation is: ； wherein, is the filtered feature, is the number of graph convolutional layers, is a parameter for balancing the influence of node self-information and its neighborhood information, , denotes the normalized affinity matrix of the th graph convolutional layer, denotes the high-level feature, is the shared learnable parameter across views.

5. The incomplete multi-modal smooth contrast clustering method based on information imputation as claimed in claim 4, wherein, In step S4, the way of the graph decoder to reconstruct the view is: ； wherein, represents a graph structure of the reconstruction of the view, are learnable parameters of the view decoder; The expression of the graph reconstruction loss is as follows: ； wherein, denotes the view reconstruction loss, is the number of views, is the adjacency matrix of the th view, the index denotes the Frobenius norm.

6. The incomplete multi-modal smooth contrast clustering method based on information imputation as claimed in claim 1, wherein, In step S5, the method for introducing a Gaussian noise term in the information noise contrast estimation loss function is: randomly sampling multiple Gaussian noise vectors consistent with the dimension of the feature vector from a Gaussian distribution, and pairing the multiple Gaussian noise vectors with each sample to form a high-confidence negative sample pair, which is represented as follows: ； ； in, Represents a view and Feature-level multi-view smooth contrast loss, For the sample size, For temperature parameters, This represents the similarity measurement function. , Views and The Middle Features of each sample For view The Middle Features of each sample This is the sum of the similarities of all feature-level negative sample pairs. This represents the number of Gaussian noise vectors involved in the computation during feature-level contrastive learning. For balancing parameters, , Views and The randomly generated number A Gaussian noise vector; Feature-level multi-view smooth contrastive loss function is: 。 7. The incomplete multi-modal smooth contrast clustering method based on information imputation as claimed in claim 1, wherein, In step S5, the representation of introducing a Gaussian noise term in the semantic label level contrast loss function is as follows: ； ； in, Represents a view and Semantic label-level multi-view smooth contrast loss, The number of clusters, It is the temperature coefficient. This represents the similarity measurement function. and Views and Cluster assignment vectors for the same semantic cluster, For view The Middle Cluster assignment vectors for each sample The sum of similarities for all semantically labeled negative sample pairs. It is the number of Gaussian noise vectors involved in the computation during semantic label-level contrastive learning. For balancing parameters, , Views and The randomly generated number A Gaussian noise vector; Semantic label level multi-view smooth contrastive loss function For: 。

Images