An incomplete multi-view clustering method and system based on deep learning

Abstract

The present invention relates to the field of data mining technology, and in particular to an incomplete multi-view clustering method and system based on deep learning. An incomplete multi-view clustering method based on deep learning includes S1. obtaining a cross-view training set, including a complete data set and an incomplete data set; S2. building a clustering model, including an encoder network and a decoder network; S3. using the clustering model to extract and reconstruct features of the cross-view training set, and calculating the reconstruction loss, and optimizing the reconstruction loss; S4. based on the reconstruction loss, respectively using the complete data set and the incomplete data set to train the clustering model; S5. using the trained clustering model to make predictions. The present invention solves the problem of improving clustering accuracy through effective double-layer operations on complete and incomplete data.

Description

Technical Field

[0001] The present invention relates to the technical field of data mining, and in particular to an incomplete multi-view clustering method and system based on deep learning. Background Art

[0002] IMVC is the abbreviation of Incomplete Multi-view Clustering. It is a clustering method mainly used to deal with the situation where there is missing data in multi-view data.

[0003] Current IMVC methods face some challenges in practical applications. Since the data collected in real life are often missing, this leads to the problem of incomplete multi-view clustering. When some views are missing, the natural alignment property of the same instance between multiple views will be destroyed, thus affecting the exploration of complementary and consistent information. In addition, if the missing data after repair is inconsistent with the semantics of the missing data, it will lead to a decline in clustering performance.

[0004] Existing IMVC mainly includes encoder-based methods, Gaussian-based methods, GCNs (graph convolutional neural network)-based methods and contrast-based methods. Encoder-based methods can obtain consistent feature representations to fill missing data by extracting feature representations from data, thereby achieving good clustering performance. Gaussian-based methods use generative adversarial networks to mine and fill missing parts in multi-view data, and continuously improve the ability of the generator through competition between the generator and the discriminator. GCNs-based methods explore the common representation of multiple views by learning structural information in multiple views. Contrast-based methods complete missing data by comparing multiple views and explore consistent representations.

[0005] Although these methods have made some progress, many IMVC methods still focus on learning the complete part and ignore the impact of low-quality input of the missing part on clustering. Therefore, how to better handle missing data and improve clustering performance is still an important issue that needs to be solved in the IMVC field. Summary of the invention

[0006] In order to solve the above-mentioned problems, the present invention provides an incomplete multi-view clustering method and system based on deep learning.

[0007] In a first aspect, the present invention provides an incomplete multi-view clustering method based on deep learning, which adopts the following technical solution:

[0008] An incomplete multi-view clustering method based on deep learning, comprising:

[0009] S1. Obtain cross-view training sets, including complete data sets and incomplete data sets;

[0010] S2. Build a clustering model, including an encoder network and a decoder network;

[0011] S3. Use the clustering model to extract and reconstruct features of the cross-view training set, calculate the reconstruction loss, and optimize the reconstruction loss;

[0012] S4. Based on the reconstruction loss, the clustering model is trained using the complete dataset and the incomplete dataset respectively;

[0013] S5. Use the trained clustering model to make predictions.

[0014] Furthermore, the extracting and reconstructing features of the cross-view training set using the clustering model includes:

[0015] A1. inputting the multi-view training set into the encoder network to obtain high-level semantic features of the multi-view training set;

[0016] A2. Input the high-level semantic features of the multi-view training set into the decoder network to reconstruct the high-level features of the multi-view training set and calculate the reconstruction loss;

[0017] A3. Utilize the optimal transfer to calculate the cross-view alignment loss and optimize the reconstruction loss and the cross-view alignment loss.

[0018] Further, the calculating cross-view alignment loss by using optimal transmission includes:

[0019] B1. Based on the cross-view training set, obtain the prototype corresponding to each view, calculate the distance from each feature in the view to each prototype to obtain the total distance, minimize the total distance, and determine the unique prototype set in the feature space ;

[0020] B2. Using the Optimal Transport Pair Prototype Set Align the prototypes of each view in the image to make the examples in the same prototype consistent in different views, and calculate the alignment loss .

[0021] Furthermore, the use of the optimal transport pair prototype set The prototype of each view in is aligned, including:

[0022] B21. Calculate the similarity relationship between the prototypes, including constructing a cost matrix based on the similarity relationship and calculating the total cost of matching each prototype of the view with each prototype of another view;

[0023] B22. Use the optimal transport to obtain a matching solution under the cost matrix with the minimum total cost, and use the matching solution to align the prototypes of different views.

[0024] Furthermore, the clustering model is trained using a complete data set and an incomplete data set based on the reconstruction loss, respectively, including:

[0025] C1. Repair the incomplete data set by averaging to obtain the repaired incomplete data set;

[0026] C2. Select the complete data set and the incomplete data set to construct the original data space, project the original data space into the low-dimensional feature space, and obtain the high-level semantic features of the complete data set. and high-level semantic features of incomplete datasets ;

[0027] C3. and Using spectral contrast learning, we get feature contrast loss ; Process high-level semantic features through a multi-layer perceptron network, and obtain data label contrast loss based on soft clustering assignment , in label contrast loss Add regularization terms to jointly optimize the regularization terms and calculate the regularization loss ;

[0028] C4. Construct and optimize the weighted function G, and calculate the loss for the complete part respectively and the loss of the incomplete part is ,

[0029] in , , Indicates the parameters, is the weighting function representing the weight, is a parameter;

[0030] C5. Back-propagate the loss of clustering model training, optimize the loss function, and update the similarity matrix , get new repair data, train the clustering model multiple times, make the loss function converge, and complete the training of the clustering model.

[0031] Further, the repairing the incomplete data set by averaging to obtain the repaired incomplete data set includes:

[0032] C11 calculates the similarity matrix between samples of the same view at each iteration , where t represents the number of update iterations, represents a view, i and j represent samples;

[0033] C12. Using the similarity matrix of different views Identify similar samples of the currently missing view, including of the views Incomplete samples, at the tth iteration, construct the A set of similar samples similar to incomplete samples ;

[0034] C13. By similar sample sets The samples in are averaged to recover the missing sample values.

[0035] Furthermore, the constructing and optimizing the weighting function G includes:

[0036] C41. Define a two-layer optimization, the two-layer optimization includes an upper layer and a lower layer, and the loss function of the two-layer optimization is expressed as:

[0037] ,in represents the set of parameters that minimizes the loss value, represents the upper layer optimization parameters, represents the lower layer optimization parameter, G is the weighting function;

[0038] C42. The lower layer performs feature learning and clustering assignment learning on the complete data and the repaired incomplete data, and determines the use of gradient descent optimization. Minimum parameters ;

[0039] C43. Minimum parameters for upper layer utilization update Optimizing the weighted function , multiple optimization iterations of the upper and lower layers are performed to complete the optimization of the weighted function.

[0040] In the second aspect, an incomplete multi-view clustering system based on deep learning includes:

[0041] The data acquisition module is configured to acquire a cross-view training set, including a complete data set and an incomplete data set;

[0042] The model building module is configured to build a clustering model, including an encoder network and a decoder network;

[0043] The feature extraction module is configured to extract and reconstruct features of the cross-view training set using a clustering model, calculate a reconstruction loss, and optimize the reconstruction loss;

[0044] The model training module is configured to train the clustering model using the complete data set and the incomplete data set respectively based on the reconstruction loss;

[0045] The model prediction module is configured to make predictions using the trained clustering model.

[0046] In a third aspect, the present invention provides a computer-readable storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor of a terminal device and executing the incomplete multi-view clustering method based on deep learning.

[0047] In a fourth aspect, the present invention provides a terminal device comprising a processor and a computer-readable storage medium, the processor being used to implement various instructions; the computer-readable storage medium being used to store multiple instructions, the instructions being suitable for being loaded by the processor and executing the incomplete multi-view clustering method based on deep learning.

[0048] In summary, the present invention has the following beneficial technical effects:

[0049] 1. The high-quality incomplete multi-view clustering method and system proposed in this invention based on deep learning can obtain high-quality real view repair data. The invention can more accurately calculate the dynamic structural relationship between high semantic features in each view through the missing view repair strategy of cross-view dynamic structure learning. These repaired data have higher integrity and accuracy, greatly reducing the information bias that may be caused by missing data. This provides an extremely reliable foundation for subsequent clustering analysis, making the clustering results more accurate and meaningful.

[0050] 2. The high-quality incomplete multi-view clustering method and system based on deep learning proposed in the present invention, including a triple cross-view alignment method of prototype, feature and cluster assignment, is an innovative solution that can effectively deal with the position deviation problem obtained from different views. Through the designed alignment mechanism, it can ensure that the prototype, feature and cluster assignment are highly consistent between different views. This consistency makes the representation between different views more coordinated and unified, thereby significantly improving the accuracy and reliability of clustering.

[0051] 3. The high-quality incomplete multi-view clustering method and system based on deep learning proposed in the present invention fully utilizes incomplete view data to improve clustering performance. Through the dual optimization process of alternately iteratively designing complete view features and repair features, the potential information in incomplete view data can be fully mined. This method can handle missing data, so that the model can better adapt to the characteristics of incomplete data, thereby effectively improving clustering performance.

[0052] 4. The high-quality incomplete multi-view clustering method and system proposed by the present invention based on deep learning improves the generalization ability of the model. The method performs well in dealing with missing situations in multi-view data and can effectively improve the adaptability of the model to different data distributions. By learning and adapting to various complex data situations, the model can develop stronger generalization capabilities, that is, it can make accurate predictions and analyses when faced with new and unseen data. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is an overall implementation block diagram of an incomplete multi-view clustering method based on deep learning in Example 1 of the present invention.

[0054] Figure 2 Schematic diagrams of several commonly used incomplete multi-view inpainting methods in the prior art; among them, (a) represents a neighbor-based inpainting strategy diagram, (b) represents a cross-view prediction inpainting strategy diagram, and (c) represents an adversarial inpainting strategy diagram.

[0055] Figure 3 It is a repair process diagram of the repair strategy of the present invention.

[0056] Figure 4 It is a schematic diagram of the problem of prototype transfer in the prior art.

[0057] Figure 5 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0058] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0059] Example 1

[0060] Reference Figure 1 In this embodiment, an incomplete multi-view clustering method based on deep learning includes a pre-training stage and a training stage, wherein the pre-training stage includes:

[0061] S1. Obtaining cross-view training sets , including the full dataset and incomplete datasets ;

[0062] The cross-view training set is: ,in, is the number of views in the sample, N is the number of samples, represents the dimension, R represents the set of real numbers, Indicates The training set data of each view for matrix;

[0063] The complete dataset is: ,in, Indicates The number of complete samples on views, represents the dimension, Indicates The complete data set for each view;

[0064] The incomplete data set is: ,in, Indicates The number of incomplete samples on views, Indicates An incomplete data set of views, where ;

[0065] Introducing the indicator matrix in the incomplete multi-view training set , used to indicate missing data, where represents the nth sample Views exist, =0 means the nth sample The view does not exist.

[0066] S2. Use the deep learning model to build a clustering model, including an encoder network and a decoder network, wherein the encoder network and the decoder network are both composed of four linear layers, and the input dimensions of the encoder network and the decoder network are the same. The linear layer dimensions of the encoder network are: input dimension, 500, 500, 2000, 256, and the linear layer dimensions of the decoder network are 256, 500, 500, 2000, input dimension.

[0067] The deep learning model has a multi-layer structure, and these layers are all linear network layers. The function of the model is to perform a series of processing on the input vector and finally output a vector of a specific dimension. For example, a 1×1677 vector is converted to 1×512 after passing through a linear layer, and then after passing through multiple similar linear layers, a 1×256 vector is finally output. Each linear layer contains some RELU functions, which are a commonly used activation function. Its function is to introduce nonlinear characteristics into the neural network. For the input numerical value, the RELU function will change the value less than 0 to 0, and the value greater than 0 remains unchanged. This can increase the expressive power of the model and enable it to learn more complex patterns.

[0068] S3. Use the clustering model to extract and reconstruct features of the cross-view training set, calculate the reconstruction loss, and optimize the reconstruction loss;

[0069] The method of extracting and reconstructing features of a cross-view training set using a clustering model includes:

[0070] A1. The multi-view training set Input the encoder network to obtain high-level semantic features of the multi-view training set ;

[0071] A2. In order to obtain better feature representation in the next iteration, the high-level semantic features of the multi-view training set are The high-level features of the multi-view training set are reconstructed in the input decoder network to obtain the reconstructed representation , using the complete multi-view training set and the reconstructed representation Calculate the reconstruction loss , the formula of the reconstruction loss is: = - ;

[0072] The high-level semantic features: ,in, The dimension of the high-level feature is equal to 256, N is the number of samples,

[0073] The reconstructed representation is: ,in, represents the dimension, N is the number of samples, is the number of views in the sample;

[0074] A3. Reference Figure 4 Due to the unsupervised nature of clustering, directly aligning features will lead to the prototype shift problem. In addition, most deep IMVC methods only focus on learning from complete data and ignore learning from incomplete data. Using optimal transfer to calculate cross-view alignment loss ,in Represents the prototype, optimizes the reconstruction loss and the cross-view alignment loss, and uses the optimal transmission to calculate the cross-view alignment loss ,include:

[0075] B1. Based on the cross-view training set, obtain the prototype corresponding to each view, calculate the distance from each feature in the view to each prototype to obtain the total distance, minimize the total distance, and determine the unique prototype set in the feature space , the prototype set is: , where K represents clustering, Represents the dimension, is the number of views in the sample;

[0076] B2. Using the Optimal Transport Pair Prototype Set Align the prototypes of each view in the image to make the examples in the same prototype consistent in different views, and calculate the alignment loss .

[0077] The optimal transport pair prototype set The prototype of each view in is aligned, including:

[0078] B21. Calculate the similarity relationship between the prototypes, including constructing a cost matrix based on the similarity relationship and calculating the total cost of matching each prototype of the view with each prototype of another view;

[0079] B22. Use the optimal transport to obtain a matching solution under the cost matrix with the minimum total cost, use the matching solution to align the prototypes of different views, and adjust the prototypes so that the examples in the same prototype are represented consistently in different views, so as to convert the incomplete multi-view training set into a complete multi-view training set. The difference between the p-th prototype in view m and the q-th prototype in view n is recorded as , in order to match representations with the same number of prototypes in different views to achieve the highest degree of alignment, and constrain All elements on the diagonal are 1, and the constraint is applied to each pair of prototypes of each pair of views, resulting in an alignment loss .

[0080] Clear the parameters in the optimizer to 0. and Backward propagation, the optimizer updates the model parameters so that the reconstruction loss and the loss of the cross-view prototype alignment module Minimize, and after multiple epochs, the pre-training phase is completed. In machine learning, epoch refers to the number of times the entire dataset is traversed during the training process. Usually, when training a model, we divide the dataset into several small batches, and then input the model batch by batch for training. Each time the entire dataset is traversed once, it is called an epoch.

[0081] Reference Figure 1

[0082] S4. Based on the reconstruction loss, we use the complete dataset and incomplete datasets Train the clustering model to get the complete part loss and loss of incomplete parts , utilizing the complete portion of the loss and loss of incomplete parts Further optimizing and updating the encoder network, where the encoder network is a pre-trained network, calling the pre-trained network to learn and optimize the data includes the following steps:

[0083] C1. Repair the incomplete data set by averaging to obtain the repaired incomplete data set.

[0084] Reference Figure 4 In the existing technology, based on high-level semantic features, directly aligning features will lead to the prototype shift problem due to the unsupervised nature of clustering. In addition, most deep IMVC methods only focus on learning from complete data and ignore learning from incomplete data. Therefore, during training, we first use the optimal transmission to construct cross-view prototype alignment, and simultaneously align the features and cluster assignments of the full view features and the repaired features through a dual optimization process, and calculate the alignment loss ,in, The prototype represents the complete data. The alignment here is performed on the complete data, and the incomplete part does not need to be aligned;

[0085] Reference Figure 3 , including the following steps:

[0086] C11. Selecting incomplete multi-view dataset Construct the original data space. The training process first projects the original data space into a low-dimensional feature space to obtain high-level semantic features. , which reduces view-private information and facilitates the identification of more reliable similar samples, we propose a missing view inpainting strategy with cross-view dynamic structure learning. Figure 2 Common repair strategies are shown. Among them, (a) represents the neighbor-based repair strategy, (b) represents the cross-view prediction repair strategy, and (c) represents the adversarial repair strategy. Since the repair strategy is based on dynamic structural relationships, it is necessary to calculate the similarity matrix between samples in each iteration within the same view. , where t represents the number of update iterations, represents a view, i and j represent samples, and the similarity matrix is ​​set. If a sample is missing in the similarity matrix, the elements of the row and column corresponding to the missing sample in the similarity matrix are set to 0;

[0087] C12. Using the similarity matrix of different views Identify similar samples of the currently missing view, including of the views Incomplete samples, at the tth iteration, construct the A set of similar samples similar to incomplete samples ; Specifically include: defining In the view The position index of the sample with the highest structural relationship is ,pass Find the The similar samples with the highest structural relationship among the samples are classified into the similar sample set ;

[0088] C13. By similar sample sets The samples in are averaged to recover the This approach allows the inpainted sample to contain mutual information from other samples, thus improving its consistency with the original view.

[0089] C2. Select the complete data set separately and incomplete datasets Construct the original data space, project the original data space into the low-dimensional feature space, and obtain the high-level semantic features of the complete data set and high-level semantic features of incomplete datasets ;

[0090] C3. and Using spectral contrast learning, we get feature contrast loss ; Since multiple views contain the same semantics, the consistency of cluster assignment probabilities between views is achieved through cluster assignment contrast learning. The high-level semantic features are processed through a multi-layer perceptron network, and the data label contrast loss is obtained according to the soft cluster assignment. , in label contrast loss Add regularization terms to jointly optimize the regularization terms and calculate the regularization loss , soft clustering assignment ;

[0091] C4. Construct and optimize the weighted function G, and calculate the loss for the complete part respectively and the loss of the incomplete part is ,

[0092] in , , Indicates the parameters, is the weighting function representing the weight, is a parameter;

[0093] To mitigate the negative impact of low-quality repair data, a The IMVC model of the weighted function G includes multiple MLP layers, and the model is optimized using the small batch gradient descent algorithm Adam. The construction and optimization of the weighted function G includes:

[0094] C41. Define a two-layer optimization, the two-layer optimization includes an upper layer and a lower layer, and the loss function of the two-layer optimization is expressed as:

[0095] ,in represents the set of parameters that minimizes the loss value, represents the upper layer optimization parameters, represents the lower layer optimization parameter, G is the weighting function;

[0096] C42. The lower layer performs feature learning and clustering assignment learning on the complete data and the repaired incomplete data, and determines the use of gradient descent optimization. Minimum parameters ;

[0097] C43. Minimum parameters for upper layer utilization update Optimizing the weighted function , multiple optimization iterations of the upper and lower layers are performed to complete the optimization of the weighted function.

[0098] C5. The loss of training the clustering model and Back propagation, optimize loss function, update similarity matrix , get new repair data , train the clustering model multiple times to make the loss function converge and complete the training of the clustering model.

[0099] In order to further verify the feasibility and effectiveness of the method of the present invention, experiments were conducted.

[0100] For all datasets, the missing rate of each view is denoted by p, with a step size of 0.2 and ranging from 0.1 to 0.7. This experiment is based on PyTorch implementation and executed on NVIDIA TITAN Xp GPU.

[0101] Table 1 shows the experimental results of the method on six datasets.

[0102] Table 1 Evaluation results of the method of the present invention on the test set

[0103] 0.1 0.3 0.5 0.7 ACC NMI PUR ACC NMI PUR ACC NMI PUR ACC NMI PUR BDGP 98.48±0.08 94.99±0.29 98.48±0.08 97.42±0.27 91.57±0.79 97.42±0.27 95.78±0.27 87.28±0.59 95.78±0.27 93.37±0.55 81.69±0.84 93.37±0.55 MNIST-USPS 98.94±0.11 97.07±0.31 98.94±0.11 98.13±0.07 95.00±0.20 98.13±0.07 97.11±0.14 92.61±0.33 97.11±0.14 95.34±0.4 89.06±0.75 95.34±0.4 Synthetic3d 96.67±0.41 86.24±1.42 96.67±0.41 95.37±0.39 82.03±1.27 95.37±0.39 92.93±0.89 75.01±2.2 92.93±0.89 91.70±0.83 72.4±2.07 91.70±0.83 NGs 69.28±3.32 51.63±1.82 69.28±3.32 67.8±5.58 49.05±4.27 67.8±5.58 55.36±1.47 32.38±2.15 55.84±1.36 52.72±4.74 28.57±3.65 53.08±4.40 Cora 50.36±2.97 37.53±1.22 57.5±2.95 46.56±1.6 31.85±1.72 53.85±3.18 42.22±3.35 25.94±2.53 47.9±2.4 35.81±3.38 18.13±2.83 43.54±2.06 CCV 28.32±1.35 29.23±0.59 33.23±0.57 27.18±0.3 27.89±0.7 31.74±0.56 25.67±1.11 25.55±0.62 29.76±0.79 22.62±0.49 22.23±0.41 27.00±0.75 In order to further illustrate the feasibility and effectiveness of the method of this embodiment, an ablation experiment was conducted to remove the prototype alignment module, leaving only the DSR module and the dual optimization module. Table 2 shows that the lack of the prototype alignment module (TriAlign-P) leads to poor clustering performance compared to the proposed inventive method. Specifically, in terms of accuracy ACC, the results without TriAlign-P are lower than those of the inventive method. This difference stems from the divergence of clustering prototypes between different views, which hinders multi-view clustering. In contrast, this module can not only align clustering prototypes from multiple views, but also promote multi-view clustering by exploiting the structural relationship between samples and prototypes. Figure 1 These results show that our TriAlign module effectively solves the prototype shift problem caused by missing views.

[0104] Table 2 Ablation experiment of the method of the present invention

[0105] CCV 0.1 0.5 Prototype Alignment Dynamic interpolation Two-layer optimization ACC NMI PUR ACC NMI PUR √ × √ 26.04 ± 0.90 27.65 ± 0.64 30.12 ± 1.07 22.94 ± 0.84 23.58 ± 0.47 27.07 ± 0.74 × √ √ 27.44 ± 1.02 28.87 ± 0.45 32.81 ± 0.21 25.32 ± 0.84 25.03 ± 0.38 30.23 ± 0.45 √ √ × 24.66 ± 1.26 25.08 ± 0.74 29.29 ± 0.84 23.03 ± 0.72 23.73 ± 0.51 27.28 ± 0.63 √ √ √ 28.32 ± 1.35 29.23 ± 0.59 33.23 ± 0.57 25.67 ± 1.11 25.55 ± 0.62 29.76 ± 0.79 Synthetic3d 0.1 0.5 Prototype Alignment Dynamic interpolation Two-layer optimization ACC NMI PUR ACC NMI PUR √ × √ 91.33 ± 0.86 71.92 ± 1.69 91.33 ± 0.86 86.40 ± 1.34 60.91 ± 2.46 86.40 ± 1.34 × √ √ 95.63 ± 0.52 84.11 ± 1.66 95.63 ± 0.52 90.83 ± 0.57 71.15 ± 1.38 90.83 ± 0.57 √ √ × 89.80 ± 1.99 68.99 ± 3.77 89.80 ± 1.99 86.20 ± 0.96 61.07 ± 2.19 86.20 ± 0.96 √ √ √ 96.67 ± 0.41 86.24 ± 1.42 96.67 ± 0.41 92.93 ± 0.89 75.01 ± 2.20 92.93 ± 0.89

[0106] This ablation experiment removes the dynamic remediation module (DSR) and replaces the repair module with a k-nearest neighbor based repair method. Table 2 shows that the clustering performance of the k-nearest neighbor repair method is worse than that of the proposed invention method. For ACC, the clustering results of the k-nearest neighbor repair method are lower than those of the invention method. The reason for this difference may be the false positive problem, which leads to the degradation of the quality of the repaired view. In contrast, the DSR module repairs the missing view by calculating the similarity relationship with other samples within the view, which helps to avoid false positives of cluster edge samples and improve the quality of the repaired view. The results show that the missing view repair strategy based on cross-view dynamic structure learning improves the consistency of the input view with the original view.

[0107] The ablation experiment removes the dual-optimization module (DLO) and retains the DSR module and the TriAlign-P module, that is, the total loss in the complete data is used instead of the dual-layer optimization loss. Table 2 shows that the lack of DLO leads to worse clustering performance. Specifically, in terms of ACC accuracy, the results without DLO are lower than those of the DITA-IMVC method. The dual-layer optimization makes the model learned on complete and incomplete data better than the model learned only on complete data, learns an effective incomplete multi-view clustering model, and improves the clustering performance of the model. These results show that our dual-layer optimization effectively addresses the risk of degraded clustering performance.

[0108] Example 2

[0109] This embodiment provides an incomplete multi-view clustering system based on deep learning, including:

[0110] The data acquisition module is configured to acquire a cross-view training set, including a complete data set and an incomplete data set;

[0111] The model building module is configured to build a clustering model, including an encoder network and a decoder network;

[0112] The feature extraction module is configured to extract and reconstruct features of the cross-view training set using a clustering model, calculate a reconstruction loss, and optimize the reconstruction loss;

[0113] The model training module is configured to train the clustering model using the complete data set and the incomplete data set respectively based on the reconstruction loss;

[0114] The model prediction module is configured to make predictions using the trained clustering model.

[0115] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor of a terminal device and executing the incomplete multi-view clustering method based on deep learning.

[0116] A terminal device includes a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the incomplete multi-view clustering method based on deep learning.

[0117] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An incomplete multi-view clustering method based on deep learning, characterized in that: include: S1. Obtain a cross-view training set, including a complete data set and an incomplete data set; wherein the cross-view training set is an image training set; The cross-view training set is: ,in, is the number of views in the sample, N is the number of samples, represents the dimension, R represents the set of real numbers, Indicates The training set data of each view for matrix; The complete dataset is: ,in, Indicates The number of complete samples on views, represents the dimension, Indicates The complete data set for each view; The incomplete data set is: ,in, Indicates The number of incomplete samples on views, Indicates An incomplete data set of views, where ; S2. Build a clustering model, including an encoder network and a decoder network; S3. Use the clustering model to extract and reconstruct the cross-view training set, calculate the reconstruction loss, and optimize the reconstruction loss, including: A1. inputting the cross-view training set into the encoder network to obtain high-level semantic features of the cross-view training set; A2. Input the high-level semantic features of the cross-view training set into the decoder network to reconstruct the high-level features of the cross-view training set and calculate the reconstruction loss; A3. Calculate the cross-view alignment loss using the optimal transmission and optimize the reconstruction loss and the cross-view alignment loss. S4. Based on the reconstruction loss, the clustering model is trained using the complete dataset and the incomplete dataset respectively, including: C1. Repair the incomplete data set by averaging to obtain the repaired incomplete data set, including: C11 calculates the similarity matrix between samples of the same view at each iteration , where t represents the number of update iterations, represents a view, i and j represent samples; C12. Using the similarity matrix of different views Identify similar samples of the currently missing view, including of the views Incomplete samples, at the tth iteration, construct the A set of similar samples similar to incomplete samples ; C13. By similar sample sets The samples in are averaged to recover the missing sample values; C2. Select the complete data set and the incomplete data set to construct the original data space, project the original data space into the low-dimensional feature space, and obtain the high-level semantic features of the complete data set. and high-level semantic features of incomplete datasets ; C3. and Using spectral contrast learning, we get feature contrast loss ; Process high-level semantic features through a multi-layer perceptron network, and obtain data label contrast loss based on soft clustering assignment , in label contrast loss Add regularization terms to jointly optimize the regularization terms and calculate the regularization loss ; C4. Construct and optimize the weighted function G, and calculate the loss for the complete part respectively and the loss of the incomplete part is ,in + + , = + + , Indicates the parameters, is the weighting function representing the weight, is a parameter; The construction and optimization of the weighting function G includes: C41. Define a two-layer optimization, the two-layer optimization includes an upper layer and a lower layer, and the loss function of the two-layer optimization is expressed as: ,in represents the set of parameters that minimizes the loss value, represents the upper layer optimization parameters, represents the lower layer optimization parameter, G is the weighting function; C42. The lower layer performs feature learning and clustering assignment learning on the complete data and the repaired incomplete data, and determines the use of gradient descent optimization. Minimum parameters ; C43. The upper layer uses the updated minimum parameters Optimizing the weighted function , multiple optimization iterations of the upper and lower layers are performed to complete the optimization of the weighted function; C5. Back-propagate the loss of clustering model training, optimize the loss function, and update the similarity matrix , get new repair data, train the clustering model multiple times, make the loss function converge, and complete the training of the clustering model; S5. Use the trained clustering model to make predictions.

2. The incomplete multi-view clustering method based on deep learning according to claim 1, characterized in that: The method of calculating the cross-view alignment loss using the optimal transmission includes: B1. Based on the cross-view training set, obtain the prototype corresponding to each view, calculate the distance from each feature in the view to each prototype to obtain the total distance, minimize the total distance, and determine the unique prototype set in the feature space ; B2. Using the Optimal Transport Pair Prototype Set ; Align the prototypes of each view in the image to make the examples in the same prototype consistent in different views, and calculate the alignment loss .

3. The incomplete multi-view clustering method based on deep learning according to claim 2, characterized in that: The optimal transport pair prototype set The prototype of each view in is aligned, including: B21. Calculate the similarity relationship between the prototypes, including constructing a cost matrix based on the similarity relationship and calculating the total cost of matching each prototype of the view with each prototype of another view; B22. Use the optimal transport to obtain a matching solution under the cost matrix with the minimum total cost, and use the matching solution to align the prototypes of different views.

4. An incomplete multi-view clustering system based on deep learning, executing the method according to any one of claims 1 to 3, characterized in that: include: The data acquisition module is configured to acquire a cross-view training set, including a complete data set and an incomplete data set; The model building module is configured to build a clustering model, including an encoder network and a decoder network; The feature extraction module is configured to extract and reconstruct features of the cross-view training set using a clustering model, calculate a reconstruction loss, and optimize the reconstruction loss; The model training module is configured to train the clustering model using the complete data set and the incomplete data set respectively based on the reconstruction loss; The model prediction module is configured to make predictions using the trained clustering model.

5. A computer-readable storage medium storing a plurality of instructions, characterized in that: The instructions are suitable for being loaded by a processor of a terminal device and executing the incomplete multi-view clustering method based on deep learning as claimed in claim 1.

6. A terminal device, comprising a processor and a computer-readable storage medium, wherein the processor is used to implement each instruction; and the computer-readable storage medium is used to store multiple instructions, characterized in that: The instructions are suitable for being loaded by a processor and executing the incomplete multi-view clustering method based on deep learning as claimed in claim 1.

Images