A multimodal clustering method and system based on feature fusion and label alignment

Abstract

The present invention relates to the field of deep learning technology, and in particular to a multimodal clustering method and system based on feature fusion and label alignment. The method comprises obtaining multi-view sample data and true clustering labels; constructing a multi-view contrastive learning neural network model; training the multi-view contrastive learning neural network model using the multi-view sample data to obtain a trained multi-view contrastive learning neural network model; extracting features from the multi-view sample data using the trained multi-view contrastive learning neural network model, optimizing the label quality of the model output by calculating the loss, and obtaining a multi-view clustering model; and effectively embedding feature learning and label learning tasks into a single framework through a self-supervised method. This method is simpler and faster than the two-stage deep multi-view clustering method, and it also effectively utilizes view-specific information to assist in the generation of view consensus semantics.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a multimodal clustering method and system based on feature fusion and label alignment. Background Art

[0002] With the increase of data dimensions and the diversification of information sources, multi-view clustering methods have gradually become a key tool for processing complex data. It can effectively fuse heterogeneous data from different perspectives, thus showing excellent clustering effects in the fields of image processing, natural language processing, bioinformatics, and recommendation systems. By integrating multiple feature representations, multi-view clustering not only improves the accuracy of data analysis, but also provides strong support for deep mining of potential patterns in data. Multi-view clustering represents a clustering method that integrates data on multiple views, thereby solving the constraints of single-view clustering methods. Multi-view clustering methods can be divided into two categories: traditional methods and deep learning methods. Traditional methods can be mainly divided into four categories: non-negative matrix factorization (NMF), multi-kernel clustering (MKC), graph-based clustering, and subspace clustering methods. Specifically, the NMF method uses matrix decomposition to perform representation learning of multi-view data. The MKC algorithm uses a multi-kernel learning framework to extract kernel representations from a predefined kernel matrix. Graph-based clustering methods use a unified graph structure to merge multi-view data. In addition, subspace clustering methods are designed to obtain consistent subspace representations. However, traditional multi-view clustering methods have difficulties in dealing with view heterogeneity, kernel parameter selection, and sensitivity to noise. These limitations have driven the extensive exploration and application of deep learning methods on multi-view datasets.

[0003] In recent years, with the rise of deep learning, there has been an increasing number of deep multi-view clustering methods that integrate traditional methods, including methods that use deep neural networks to identify high-level semantic features. These deep multi-view clustering methods have strong data structure extraction capabilities and have achieved high performance on multiple benchmark datasets. There are two types of methods based on deep learning multi-view clustering. The first is a two-stage multi-view clustering method. The two-stage method first extracts features through methods such as multi-view fusion. The goal of multi-view fusion is to extract shared representations from specific views, which are valuable for subsequent tasks such as clustering or classification. Existing technologies use the attention mechanism to fuse multi-view features and utilize the complementarity of similar samples to obtain fused multi-view features. Finally, the two-stage method uses the K-means clustering method to use the features to obtain clustering results. The second is a single-stage clustering method, which integrates feature learning and clustering tasks into a unified framework through a deep multi-view clustering method, thereby achieving end-to-end clustering by learning the clustering label matrix. Compared with the two-stage clustering method, the single-stage clustering method has the advantages of low complexity and fast execution speed in performing clustering tasks.

[0004] Most existing multi-view clustering methods directly use the reconstructed features obtained by the autoencoder for feature processing and feature fusion. These reconstructed features still contain a lot of view-specific information. This irrelevant view-specific information will mislead the extraction of fusion features and the generation of semantic labels, thus affecting the quality of clustering. Summary of the invention

[0005] In order to solve the above-mentioned problems, the present invention provides a multimodal clustering method and system based on feature fusion and label alignment, which has high clustering efficiency and high clustering accuracy.

[0006] In a first aspect, the present invention provides a multimodal clustering method based on feature fusion and label alignment, which adopts the following technical solution:

[0007] A multimodal clustering method based on feature fusion and label alignment, comprising:

[0008] Obtain multi-view sample data and true clustering labels;

[0009] Build a multi-view contrastive learning neural network model;

[0010] The multi-view comparative learning neural network model is trained by using the multi-view sample data to obtain a trained multi-view comparative learning neural network model;

[0011] The trained multi-view contrastive learning neural network model is used to extract features from multi-view sample data, and the label quality of the model output is optimized by calculating the loss to obtain a multi-view clustering model.

[0012] Test the multi-view clustering model and get the cluster labels predicted by the model.

[0013] Furthermore, the acquisition of multi-view sample data and true cluster labels includes selecting The transformed multi-view sample data and the corresponding real cluster labels of the samples are used as the training set. The sample The view is represented as The first The true cluster label to which the sample belongs is expressed as is a positive integer, take .

[0014] Furthermore, the construction of the multi-view contrast learning neural network model includes constructing an encoder, a decoder, a feature contrast module, a multi-view mapping module, a feature fusion module, and a label contrast module, wherein each view corresponds to an encoder and a decoder, the encoder and the decoder serve as input layers, the feature fusion module contains a specified number of superimposed hidden layers, the label contrast module is the output layer, and the label result finally generated by the fusion of high-level features of the multi-view is the output of the model.

[0015] Furthermore, the multi-view contrast learning neural network model is trained using the multi-view sample data, including Multi-view sample data Input into the encoder and decoder for training to obtain reconstructed sample data , and the original data and the reconstructed sample data The overall loss of all samples is calculated by the mean square error loss function MSEloss, and the reconstruction ability of the encoder and decoder is optimized through the loss.

[0016] Furthermore, the method uses the trained multi-view contrast learning neural network model to extract features from the multi-view sample data, including extracting Multi-view sample data is used to obtain multi-view features through the trained encoder Contrastive learning is used to constrain the feature extraction between different views to maximize the mutual information between different features, and then the multi-view features are Obtain multi-view high-level features through feature comparison module .

[0017] Furthermore, the label quality of the model output is optimized by calculating the loss to obtain a multi-view clustering model, including calculating different high-level features The feature contrast loss is calculated using the cross entropy CrossEntropyLoss, and the feature fusion module is used to convert the The views are fused into a single high-level fused feature.

[0018] Furthermore, the method optimizes the label quality of the model output by calculating the loss to obtain a multi-view clustering model, and also includes using a label comparison module to compare the features of the multi-views. The label alignment loss is calculated by generating labels and labels generated by high-level fusion features, where the label alignment loss is given by Divergence calculation, the feature reconstruction loss is calculated synchronously during model training. The label quality of the model output is optimized through feature contrast loss, label alignment loss, label contrast loss and feature reconstruction loss to obtain a trained multi-view clustering model.

[0019] In the second aspect, a multimodal clustering system based on feature fusion and label alignment includes:

[0020] The data acquisition module is configured to acquire multi-view sample data and true clustering labels;

[0021] The model building module is configured to build a multi-view contrastive learning neural network model;

[0022] The model training module is configured to train the multi-view comparative learning neural network model using the multi-view sample data to obtain a trained multi-view comparative learning neural network model;

[0023] The model optimization module is configured to extract features from multi-view sample data using the trained multi-view contrastive learning neural network model, optimize the label quality of the model output by calculating the loss, and obtain a multi-view clustering model;

[0024] The testing module is configured to test the multi-view clustering model and obtain the clustering labels predicted by the model.

[0025] In a third aspect, the present invention provides a computer-readable storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor of a terminal device and executing the multimodal clustering method based on feature fusion and label alignment.

[0026] In a fourth aspect, the present invention provides a terminal device comprising a processor and a computer-readable storage medium, the processor being used to implement various instructions; the computer-readable storage medium being used to store multiple instructions, the instructions being suitable for being loaded by the processor and executing the multimodal clustering method based on feature fusion and label alignment.

[0027] In summary, the present invention has the following beneficial technical effects:

[0028] 1) This paper proposes a one-stage deep multi-view clustering method that effectively embeds feature learning and label learning tasks into a single framework through a self-supervised approach. This method is simpler and faster than the two-stage deep multi-view clustering method, and it also effectively utilizes view-specific information to assist in the generation of view consensus semantics.

[0029] 2) The proposed method introduces a consensus high-level feature learning module, which enhances the consistency of view-specific high-level features by maximizing the mutual information between features from different views, and fuses multi-view features through the proposed U-shaped enhancement network and multi-layer perceptron. This method not only reduces the redundant information in low-level features, but also refines the task-related information while retaining the initial fused feature information.

[0030] 3) Our method proposes a self-supervised label learning module. In this module, we use two branches to learn consistent view-specific semantic labels through contrastive learning, and view-consensus semantic labels from consensus high-level features. We align the view-consensus semantic labels with the view-specific semantic labels through KL divergence. This alignment is promoted by high-confidence consistent cluster assignments, addressing the limitations of clustering performance caused by low-quality views during feature fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a schematic diagram of the method of Example 1 of the present invention;

[0032] Figure 2 When the NGs data set is processed in Embodiment 1 of the present invention, the left, middle and right of the figure are schematic diagrams of cluster separation in the early, middle and late stages of training, respectively;

[0033] Figure 3 When the BDGP data set is processed in Embodiment 1 of the present invention, the left, middle and right of the figure are schematic diagrams of cluster separation in the early, middle and late stages of training respectively;

[0034] Figure 4 When the Synthetic3d data set is processed in Embodiment 1 of the present invention, the left, middle and right sides of the figure are schematic diagrams of cluster separation in the early, middle and late stages of training, respectively. DETAILED DESCRIPTION

[0035] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0036] Example 1

[0037] Reference Figure 1 In this embodiment, a multi-view clustering method based on maximizing mutual information for feature fusion and aligning the distribution of semantic labels is provided. The overall implementation block diagram is as follows: Figure 1 As shown, it includes two processes: training phase and testing phase.

[0038] 1. The specific steps of the training phase are:

[0039] Step 1_1: Select The transformed multi-view sample data and the real clustering labels of the corresponding samples are used to form a training set. The path Path, dimension Dim, sample number Data_size, cluster number Class_num, and view number View of the corresponding data set are read to load the data set. The sample The view is represented as The first The true cluster label to which the sample belongs is expressed as is a positive integer, take .

[0040] Step 1_2: Construction of multi-view contrast learning neural network: The multi-view contrast learning neural network consists of multiple modules, including encoder, decoder, feature contrast module, multi-view mapping module, feature fusion module, label contrast module, where the encoder and decoder are determined by the number of views of the input data, and each view corresponds to an encoder and decoder; the feature fusion module contains a specified number of superimposed hidden layers. The encoder and decoder act as the input layer to process the multi-view data input to the model; the label contrast module is the output layer, and the label result generated by the fusion of high-level features of multiple views is the output of the model.

[0041] For the encoder, according to the view of different data sets, different views have dedicated dimensions (input dimension is input_dim, output dimension is feature_dim) encoder, which is composed of the first fully connected layer (FullyConnected Layer), the first activation layer (Activation, Act), the second fully connected layer, the second activation layer, the third fully connected layer, the third activation layer, and the fourth fully connected layer set in sequence; the encoder accepts the original input in the data set tensors with input dimension input_dim; among them, the input dimension of the first fully connected layer is input_dim, and the output dimension of the first fully connected layer is 500; the activation mode of the first activation layer, the second activation layer, and the third activation layer are all "Relu"; the input dimension of the second fully connected layer is 500, and the output dimension of the second fully connected layer is 500; the input dimension of the third fully connected layer is 500, and the output dimension of the third fully connected layer is 2000; the input dimension of the fourth fully connected layer is 2000, and the output dimension of the fourth fully connected layer is feature_dim.

[0042] For the decoder, according to the view of different data sets, different views have dedicated dimensions (input dimension is feature_dim, output dimension is input_dim) of the decoder, which is composed of the first fully connected layer, the first activation layer, the second fully connected layer, the second activation layer, the third fully connected layer, the third activation layer, and the fourth fully connected layer set in sequence; the encoder accepts the original input in the data set tensors with an input dimension of feature_dim; the input dimension of the first fully connected layer is feature_dim, and the output dimension of the first fully connected layer is 2000; the activation methods of the first activation layer, the second activation layer, and the third activation layer are all "Relu"; the input dimension of the second fully connected layer is 2000, and the output dimension of the second fully connected layer is 500; the input dimension of the third fully connected layer is 500, and the output dimension of the third fully connected layer is 500; the input dimension of the fourth fully connected layer is 500, and the output dimension of the fourth fully connected layer is input_dim.

[0043] For the feature comparison module, it consists of only one fully connected layer set up; the input dimension of the fully connected layer is feature_dim, and the output dimension of the third fully connected layer is high_feature_dim; where feature_dim represents the feature_dim feature dimension generated by the encoder-decoder, and high_feature_dim represents the high-level feature dimension obtained through our fully connected layer.

[0044] For the multi-view mapping module, it consists of a fully connected layer, a batch normalization layer (BatchNormalization Layer, BN), and an activation layer set in sequence; among them, the input dimension of the fully connected layer is view*high_feature_dim, the output dimension is high_feature_dim, the bias is set to True, the feature dimension size of the batch normalization layer BN is high_feature_dim, the activation mode of the activation layer is "Relu", and the parameter inplace is set to True.

[0045] For the feature fusion module, it consists of a first fully connected layer, a batch normalization layer, an activation layer, a superimposed linear layer (Linear Layer), and a second fully connected layer, which are set in sequence; the input dimension of the first fully connected layer is high_feature_dim, and the output dimension is high_feature_dim / 2, wherein the bias is set to Fasle, the input dimension of the second fully connected layer is high_feature_dim / 2, and the output dimension is high_feature_dim, wherein the bias is set to Fasle, the parameter inplace is set to True, the feature dimension size of the batch normalization layer is high_feature_dim / 2, and the activation mode of the activation layer is "Relu"; the superimposed linear layer (Linear Layer) can set the number of its hidden layers by the number of num_layers, and each hidden layer contains a fully connected layer and an activation layer, wherein the input dimension of the fully connected layer is high_feature_dim / 2, the output dimension is high_feature_dim / 2, the bias is set to Fasle, the activation mode of the activation layer is "Relu", and the parameter inplace is set to True.

[0046] For the label comparison module, it is composed of a first fully connected layer, a second fully connected layer, and an activation layer, which are arranged in sequence; wherein, the input dimension of the first fully connected layer is high_feature_dim, and the output dimension of the first fully connected layer is feature_dim,

[0047] The input dimension of the second fully connected layer is feature_dim, the output dimension of the second fully connected layer is class_num, the activation method of the activation layer is "Softmax", and its output dimension dim is 1.

[0048] Step 1_3: Put the training set Multi-view sample data Input into the encoder and decoder modules for training, through the encoder Get the corresponding low-level features :

[0049]

[0050] in For the Encoder Network parameter, Responsible for learning low-dimensional features and then reconstructing features The reconstructed sample data is obtained after the decoder :

[0051]

[0052] And the original data and the reconstructed sample data The overall loss of all samples is calculated by the mean square error loss function (MSEloss):

[0053]

[0054] We optimize the reconstruction capabilities of our encoder and decoder by calculating the loss and repeat the above steps In this example, , and obtain the pre-trained encoder and decoder.

[0055] Step 1_4: Put the training set Multi-view sample data is passed through the pre-trained encoder to obtain multi-view features Then use the multi-layer perceptron to reduce the feature dimension and extract high-level features :

[0056]

[0057] in, and Respectively represent The goal of this embodiment is to extract task-relevant information that is shared between representations of different views (consistency) while eliminating irrelevant details. To enhance the consistency of these high-level features, we maximize the mutual information between features from different views:

[0058]

[0059] in Representative high-level features of each view. Directly solving mutual information is very challenging due to the difficulty of double integration of high-dimensional continuous variables and the fact that the data distribution is often unknown. But we do not care about the exact value of mutual information. Instead, our goal is to enhance the model's learning of useful information by imposing mutual information constraints. This allows us to indirectly optimize the model by searching for mutual information bounds. Our goal is to maximize mutual information by minimizing the lower bound of mutual information through contrastive loss. That is:

[0060]

[0061] In the formula The temperature parameter is used to reconstruct each view, which avoids model collapse and preserves both the private information of the view and the The common semantics of Represents the cosine distance. The feature contrast loss is calculated using cross entropy CrossEntropyLoss.

[0062] To obtain consensus feature representation from multiple views, we use the high-level features obtained from all views Concatenated into a consensus feature This helps mitigate the negative impact that ambiguous cluster structures in certain views can have on overall performance. The process is described below:

[0063]

[0064] We then construct the fusion block using a linear layer and ReLU activation , mapping the fused connection vector to the dimension of a single high-level view feature to obtain the fused feature S.

[0065] In addition, we use a multi-layer perceptron MLP to project the fused high-level features into a low-dimensional space and then map them back. We add the pre-scaling and post-scaling features to get the final fused feature F:

[0066]

[0067] in and Used to project the fused features into a low-dimensional space. and Used to map the fused features back to the original space. Our fusion block scales the high-level features and adds the pre-scaled and post-scaled fused features together. Another advantage of our fusion method is that it is able to map any complex data into a compatible representation space.

[0068] Step 1_5: To assist high-level features in generating semantic labels, we use low-level features Get the corresponding label and optimize the low-level features through label contrast loss Generated Tags :

[0069]

[0070] In the formula is the temperature parameter, The probability that the n views representing the sample belong to the jth class; the loss calculation method set in the implementation example of the present invention is sum, and the label comparison loss is calculated using cross entropy.

[0071] At the same time, we use label MLP to obtain the corresponding labels for the fused high-level features. , use the label comparison module to compare the features of multiple views The label alignment loss is calculated by generating labels and labels generated by high-level fusion features, where the label contrast loss is given by Divergence calculation:

[0072]

[0073] Step 1_6: Calculate the feature reconstruction loss while training the model. We calculate the feature contrast loss, label alignment loss, label contrast loss, and feature reconstruction loss to optimize the label quality of the model output:

[0074]

[0075] in Reconstruct loss for low-level features; and They represent feature contrast loss and label contrast loss respectively; represents the label alignment loss. In addition, to prevent the consistent cluster assignment from overly affecting the generation of common semantic labels, we introduce a balancing parameter in the alignment loss In this example, .

[0076] Repeat the above steps In this example, , and we get our trained multi-view clustering model.

[0077] 2. The specific steps of the testing phase are:

[0078] The test set Multi-view sample data The input is into the trained model, and the obtained fused multi-view high-level features generate semantic labels. The generated semantic labels are aligned with the multi-view semantic labels generated by the reconstructed features to obtain the clustering labels predicted by our model.

[0079] In order to further verify the feasibility and effectiveness of the method of the present invention, experiments were conducted.

[0080] The performance is evaluated using the evaluation method of the scikit-learn deep learning library based on Python. The multi-view datasets Prokaryotic, Synthetic3d, BDGP, ​​NGs, Hdigit, Cifar10, and Caltech-5V are used to analyze the clustering effect of the method of the present invention on multi-view image data. Here, three commonly used objective parameters for evaluating clustering methods are used as evaluation indicators, namely accuracy (Acurracy), normalized mutual information (NMI), and purity (PUR) to evaluate the clustering performance of predicting multi-view images.

[0081]

[0082]

[0083] in yes and The mutual information between and They are and The entropy of .

[0084]

[0085] in is the total number of samples; It is The set of samples in the cluster; Is the real category A sample set of It is Clusters and true categories The intersection of the number of samples.

[0086] The method of the present invention is used to predict all data in the multi-view datasets Prokaryotic, Synthetic3d, BDGP, ​​NGs, Hdigit, Cifar10, and Caltech-5V, and different test data are divided into corresponding clusters. The accuracy ACC, normalized mutual information NMI, and purity PUR of the clustering effect of the method of the present invention in different datasets are listed in Table 1. From the data listed in Table 1, it can be seen that the multi-view image clustering result obtained by the method of the present invention is good, indicating that it is feasible and effective to use the method of the present invention to perform multi-view image clustering.

[0087] Table 1 Evaluation results of the method of the present invention on the test set

[0088]

[0089] Figure 2 From left to right, the cluster separation of samples in the NGs data set in the initialization case, the case of 10 rounds of training, and the case of 120 rounds of training in Example 1 of the present invention when the NGs data set is processed; Figure 3 From left to right, the cluster separation of samples in the BDGP dataset in the case of initialization, 5 rounds of training, and 10 rounds of training in Example 1 of the present invention when the BDGP dataset is processed; Figure 4 From left to right, the cluster separation of samples in the Synthetic3d dataset in the initialization case, the case of 10 rounds of training, and the case of 40 rounds of training in Example 1 of the present invention are shown; Figure 2 Left, center, right, contrast Figure 3 Left, center, right, contrast Figure 4 From the left, middle and right, it can be seen that the ability to separate different clusters is significantly improved after a certain number of training rounds using the method of the present invention, and the accuracy of multi-view classification tasks is relatively high.

[0090] Example 2

[0091] This embodiment provides a multimodal clustering system based on feature fusion and label alignment, including:

[0092] The data acquisition module is configured to acquire multi-view sample data and true clustering labels;

[0093] The model building module is configured to build a multi-view contrastive learning neural network model;

[0094] The model training module is configured to train the multi-view comparative learning neural network model using the multi-view sample data to obtain a trained multi-view comparative learning neural network model;

[0095] The model optimization module is configured to extract features from multi-view sample data using the trained multi-view contrastive learning neural network model, optimize the label quality of the model output by calculating the loss, and obtain a multi-view clustering model;

[0096] The testing module is configured to test the multi-view clustering model and obtain the clustering labels predicted by the model.

[0097] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor of a terminal device and executing the multimodal clustering method based on feature fusion and label alignment.

[0098] A terminal device includes a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the multimodal clustering method based on feature fusion and label alignment.

[0099] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A multimodal clustering method based on feature fusion and label alignment, characterized in that: include: Obtain multi-view sample data and true clustering labels; Build a multi-view contrastive learning neural network model; The multi-view comparative learning neural network model is trained by using the multi-view sample data to obtain a trained multi-view comparative learning neural network model; The trained multi-view contrastive learning neural network model is used to extract features from multi-view sample data, and the label quality of the model output is optimized by calculating the loss to obtain a multi-view clustering model. Test the multi-view clustering model and obtain the cluster labels predicted by the model; The method is used for multi-view image clustering; The acquisition of multi-view sample data and true clustering labels includes selecting The transformed multi-view sample data and the corresponding real cluster labels of the samples are used as the training set. The sample The view is represented as The first The true cluster label to which the sample belongs is expressed as is a positive integer, take ; The multi-view contrast learning neural network model is constructed, including constructing an encoder, a decoder, a feature contrast module, a multi-view mapping module, a feature fusion module, and a label contrast module, wherein each view corresponds to an encoder and a decoder, the encoder and the decoder are used as input layers, the feature fusion module includes a specified number of superimposed hidden layers, the label contrast module is the output layer, and the label result finally generated by the multi-view fusion high-level features is the output of the model; The multi-view contrast learning neural network model is trained by using the multi-view sample data, including Multi-view sample data Input into the encoder and decoder for training to obtain reconstructed sample data , and the original data and the reconstructed sample data The overall loss of all samples is calculated through the mean square error loss function MSEloss, and the reconstruction ability of the encoder and decoder is optimized through the loss; The multi-view sample data is extracted using the trained multi-view contrast learning neural network model, including extracting features from the training set Multi-view sample data is used to obtain multi-view features through the trained encoder Contrastive learning is used to constrain the feature extraction between different views to maximize the mutual information between different features, and then the multi-view features are Obtain multi-view high-level features through feature comparison module ; The loss is calculated to optimize the label quality of the model output, and a multi-view clustering model is obtained, including the calculation of different high-level features. The feature contrast loss is calculated using the cross entropy CrossEntropyLoss, and the feature fusion module is used to convert the The views are fused into a single high-level fused feature; The method optimizes the label quality of the model output by calculating the loss to obtain a multi-view clustering model, and also includes using a label comparison module to compare the features of multiple views. The label alignment loss is calculated by generating labels and labels generated by high-level fusion features, where the label alignment loss is given by Divergence calculation, the feature reconstruction loss is calculated synchronously during model training. The label quality of the model output is optimized through feature contrast loss, label alignment loss, label contrast loss and feature reconstruction loss to obtain a trained multi-view clustering model.

2. A multimodal clustering system based on feature fusion and label alignment, executing the method as claimed in claim 1, characterized in that: include: The data acquisition module is configured to acquire multi-view sample data and true clustering labels; The model building module is configured to build a multi-view contrastive learning neural network model; The model training module is configured to train the multi-view comparative learning neural network model using the multi-view sample data to obtain a trained multi-view comparative learning neural network model; The model optimization module is configured to extract features from multi-view sample data using the trained multi-view contrastive learning neural network model, optimize the label quality of the model output by calculating the loss, and obtain a multi-view clustering model; The testing module is configured to test the multi-view clustering model and obtain the clustering labels predicted by the model.

3. A computer-readable storage medium storing a plurality of instructions, characterized in that: The instructions are suitable for being loaded by a processor of a terminal device and executing a multimodal clustering method based on feature fusion and label alignment as described in claim 1.

4. A terminal device, comprising a processor and a computer-readable storage medium, wherein the processor is used to implement each instruction; and the computer-readable storage medium is used to store multiple instructions, characterized in that: The instructions are suitable for being loaded by a processor and executing a multimodal clustering method based on feature fusion and label alignment as claimed in claim 1.

Images