Multi-view representation learning method, clustering method, device and storage medium

Abstract

The application relates to the technical field of artificial intelligence, in particular to a multi-view representation learning method, a clustering method, equipment and a storage medium. The multi-view representation learning method comprises the following steps: inputting sample multi-view data set into a pre-constructed multi-variational autoencoder model to obtain shared variable information and unique variable information of each view; using a vMF distribution algorithm to model and sample the unique variable to obtain unique features of each view; fusing the shared variable information of all views and using a Gumbel-Softmax distribution algorithm to model and sample the shared variable to obtain shared features; and fusing the unique features and the shared features of each view to obtain latent representation of each view; and reconstructing and iteratively optimizing the latent representation of each view to obtain an optimized multi-variational autoencoder model. The method can effectively solve the limitation of Gaussian distribution and obtain better clustering results.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a multi-view representation learning method, a clustering method, a device and a storage medium. BACKGROUND

[0002] In recent years, multi-view clustering (MVC) methods have received extensive attention due to their ability to integrate complementary information from multiple sources or perspectives, providing a more comprehensive approach to mining target features and connections between different perspectives to understand complex data.

[0003] However, although MVC methods perform well in integrating multi-source information, existing methods have obvious deficiencies when dealing with complex data. Specifically, most methods rely on the assumption of Gaussian distribution in the latent space, which may cause problems such as collapse of the latent space when dealing with high-dimensional complex data or non-Gaussian distribution data, thereby hindering data alignment and information fusion and impairing similarity measurement between multi-views. For example, in MVC methods based on the Variational Autoencoder (VAE) framework, excessive reliance on the KL (Kullback-Leibler) divergence regularization term may cause the latent representation to be too concentrated, and sometimes even cause the latent space to collapse. This phenomenon occurs when the latent features of different perspectives become highly similar or overlapping, thereby weakening the effectiveness and reliability of the clustering results of the VAE-based MVC method. SUMMARY

[0004] Embodiments of the present application aim to provide a multi-view representation learning method, a clustering method, a device and a storage medium to solve the problem that the MVC method in the prior art may cause the latent space to collapse when dealing with high-dimensional complex data or non-Gaussian distribution data.

[0005] To solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0006] According to a first aspect of the present application, a multi-view representation learning method is provided, the method comprising:

[0007] obtaining a sample multi-view data set, the sample multi-view data set comprising a plurality of sample data, each sample data comprising a plurality of views;

[0008] A multivariate autoencoder model is constructed. The encoder of the multivariate autoencoder model performs latent space mapping and decoupling learning on each view in the sample data to obtain the shared variable information and unique variable information of the sample data in each view. Based on the unique variable information of the sample data in each view, the vMF distribution algorithm is used to model and sample the unique variables to obtain the unique features of the sample data in each view. The shared variable information of all views in the sample data is fused, and the Gumbel-Softmax distribution algorithm is used to model and sample the shared variables to obtain the shared features of the sample data. The unique features of the sample data in each view and the shared features of the sample data are concatenated and fused to obtain the latent representation of the sample data in each view.

[0009] The decoder of the multivariate autoencoder model reconstructs the latent representation of the sample data in each view, and the multivariate autoencoder model is jointly optimized based on the reconstruction loss and KL divergence to obtain the optimized multivariate autoencoder model.

[0010] Optionally, the multivariate autoencoder model includes multiple variational autoencoders, each of which corresponds to a view of the sample data.

[0011] Optionally, the step of performing latent space mapping and decoupling learning on each view of the sample data through the encoder of the multivariate autoencoder model to obtain shared variable information and unique variable information of the sample data in each view includes:

[0012] Based on each view in the sample data, latent space mapping is performed through a variational autoencoder in the multivariate autoencoder model to obtain the hidden variables of the sample data in each view.

[0013] The hidden variables of the sample data in each view are input into the shared learning branch and the specific learning branch for decoupled learning, so as to obtain the shared variable information and specific variable information of the sample data in each view respectively.

[0014] Optionally, the unique variable information includes a unique mean vector and a unique concentration parameter. The step of inputting the hidden variables of the sample data in each view into a unique learning branch for decoupling learning to obtain the unique variable information of the sample data in each view includes:

[0015] The hidden variables of the sample data in each view are input into the neural network of the unique learning branch to obtain the unique hidden vector. The unique hidden vector is normalized by L2 norm to obtain the unique mean vector. The unique hidden vector is activated by an activation function to obtain the unique concentration parameter.

[0016] Optionally, the formula for modeling the specific variables using the vMF distribution algorithm is as follows:

[0017]

[0018] in, It is the first Variables specific to each view It is the first The unique mean vector of each view It is the first The unique concentration parameter of each view. It is a dimension of unique variables. It is the first The normalization constant of each view.

[0019] Optionally, the shared variable information is a shared hidden vector. The process of fusing the shared variable information of all views in the sample data and using the Gumbel-Softmax distribution algorithm to model and sample the shared variables to obtain the shared features of the sample data includes:

[0020] After concatenating and fusing the shared hidden vectors of all views in the sample data, the probability vector of the shared variable is obtained.

[0021] Based on the probability vector, the Gumbel-Softmax distribution algorithm is used to model the shared variables, and an approximately one-hot vector is sampled as the shared feature of the sample data.

[0022] Optionally, the KL divergence includes a first KL divergence corresponding to the specific variables and a second KL divergence corresponding to the shared variables. The prior distribution of the first KL divergence is a hypersphere uniform distribution, and the posterior distribution is a vMF distribution. The prior distribution of the second KL divergence is a classification uniform distribution, and the posterior distribution is a Softmax probability distribution.

[0023] According to a second aspect of this application, a multi-view image clustering method is provided, the method comprising:

[0024] Based on any of the above-mentioned multi-view representation learning methods, an optimized multivariate autoencoder model is obtained;

[0025] Obtain a multi-view dataset to be clustered, wherein the multi-view dataset to be clustered includes multiple datasets to be clustered, and each dataset to be clustered includes multiple views;

[0026] The multi-view dataset to be clustered is input into the optimized multivariate autoencoder model to obtain the latent representation of the data to be clustered in each view.

[0027] The latent representations of the data to be clustered in each view are fused to obtain the fused latent representation of the data to be clustered.

[0028] Clustering is performed based on the fusion latent representation of each data in the multi-view dataset to be clustered, and the clustering result of the multi-view dataset to be clustered is obtained.

[0029] According to a third aspect of this application, an electronic device is provided, including at least one processor and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0030] According to a fourth aspect of this application, a computer storage medium is provided that stores instructions or programs that, when executed by at least one processor, cause the at least one processor to perform the method described above.

[0031] The beneficial effects of this application embodiment are as follows: Unlike existing technologies, this application embodiment provides a multi-view representation learning method. It constructs a multivariate autoencoder (MUE) model. The encoder of the MUE model first performs latent space mapping and decoupling learning on each view in the sample data to obtain shared and unique variable information for each view. Then, based on the unique variable information of each view, the vMF distribution algorithm is used to model and sample the unique variables to obtain the unique features of the sample data in each view. The shared variable information of all views in the sample data is then fused, and the Gumbel-Softmax distribution algorithm is used to model and sample the shared variables to obtain the shared features of the sample data. Next, the unique features and shared features of the sample data in each view are concatenated and fused to obtain the latent representation of the sample data in each view. Finally, the decoder of the MUE model reconstructs the latent representation of the sample data in each view, and the MUE model is jointly optimized based on the reconstruction loss and KL divergence to obtain the optimized MUE model. The method in this application fits continuous specific variables to a hypersphere using a vMF distribution, effectively overcoming the limitations of the Gaussian distribution. As a result, it has an advantage over methods based on the Gaussian assumption when dealing with complex data representation in multi-view clustering tasks, thus obtaining better clustering results. Attached Figure Description

[0032] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0033] Figure 1 This is an architecture diagram of the multi-VAE model provided in the embodiments of this application;

[0034] Figure 2 This is a flowchart of a multi-view representation learning method provided in an embodiment of this application;

[0035] Figure 3 This is a flowchart of a multi-view image clustering method provided in an embodiment of this application;

[0036] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0039] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0040] The proprietary technical terms used in this application are explained as follows:

[0041] 1. A Variational Autoencoder (VAE) is a generative model that combines deep learning and probabilistic graphical models. It can learn low-dimensional latent variable representations of data and generate new data samples. Its core idea is to approximate the true posterior distribution through variational inference, thereby achieving efficient data modeling and generation.

[0042] 2. Multi-view Clustering (MVC) is a method that uses multiple feature sets (views) of a data object for cluster analysis. It is suitable for data of the same object from different dimensions or sources (such as the visual features and textual descriptions of an image). Its core idea is to integrate multi-view information to improve clustering performance.

[0043] 3. Multi-view Representation Learning aims to extract unified or complementary feature representations from multiple data sources (views) to improve the performance of downstream tasks (such as clustering and classification). Its core idea is to leverage the consistency and complementarity between different viewpoints to learn more robust and discriminative low-dimensional representations.

[0044] 4. Multi-view datasets refer to datasets that contain data from multiple different perspectives, features, or modalities of the same object or scene.

[0045] 5. KL (Kullback-Leibler) divergence, also known as relative entropy, is an asymmetric measure of the difference between two probability distributions P and Q.

[0046] 6. The von Mises-Fisher (vMF) distribution is a probability distribution defined on a unit hypersphere and is suitable for describing directional data.

[0047] 7. The Gumbel-Softmax distribution (also known as the Concrete distribution) is a continuous probability distribution used for differentiable sampling of discrete distributions.

[0048] 8. The Softplus activation function is a smooth, monotonically increasing activation function. It is a smooth approximation of ReLU and is differentiable everywhere.

[0049] 9. A hypersphere is a sphere in high-dimensional space, defined as the set of all points equidistant from the center point.

[0050] 10. A one-hot vector is a vector containing discrete variables of K categories, with only one element being 1 (corresponding to the category position) and the rest being 0.

[0051] This application addresses the limitations of traditional multi-view clustering methods when handling high-dimensional or non-Gaussian distributed data. It proposes an innovative multi-view representation learning and clustering method that efficiently decouples multi-view image data using the vMF distribution and embeds its unique features into a hypersphere space to capture complex information that traditional Gaussian-based methods may overlook. The method employs a multi-VAE architecture, inputting each view from a multi-view output into a separate VAE. After L2 norm normalization, the distribution of the latent space is extracted. Then, the vMF distribution is used to model the extracted unique variables, arranging data points on the hypersphere. Finally, the reconstruction loss of the VAE and KL divergence are minimized to promote a tighter distribution of data within the same class, thereby achieving better clustering performance.

[0052] Please refer to Figure 1 , Figure 1 This is an architecture diagram of a multi-VAE model provided in an embodiment of this application. The multi-VAE model includes multiple VAEs, each corresponding to a view of multi-view data. In one embodiment, each VAE includes the same encoder and decoder. Figure 1 As shown, the multi-view dataset Medium sample data Different views are input into each VAE of the multi-VAE model, and the encoder of each VAE obtains the unique features of each view. and shared features And extract the shared features from all views Merge and stitch them together to form a complete view sharing feature Then, based on each view, share features between views. With the unique features of this view The potential representation of the view is obtained by stitching the images together. ], based on the potential representation of this view [ The reconstructed image is generated through the VAE decoder.

[0053] Please refer to Figure 2 , Figure 2 This is a flowchart of a multi-view representation learning method provided in an embodiment of this application. The method specifically includes the following steps:

[0054] Step S201: Obtain a sample multi-view dataset, which includes multiple sample data, and each sample data includes multiple views.

[0055] A multi-view dataset refers to a dataset containing data on the same object or scene from multiple different perspectives, with different features, or different modalities. For example, in computer vision, a multi-view dataset might consist of images of several objects at different rotation angles. In face recognition, it could be facial images of several humans under different poses, lighting conditions, and expressions. In autonomous driving, a multi-view dataset can also be data (images, point clouds, geolocation) collected by different sensors (vehicle cameras, LiDAR, GPS, etc.). Each piece of multi-view data includes multiple views.

[0056] In this embodiment, the multi-view dataset is image data captured from multiple different perspectives based on objects of different categories. For example, the multi-view dataset is image data captured from V perspectives of objects of K categories.

[0057] In some embodiments, the multi-view dataset is further used to clean and preprocess the multi-view dataset according to its characteristics, including steps such as deduplication, handling missing values, and standardization, so as to adjust all views to the same size range to eliminate size inconsistencies caused by differences in shooting angle, distance, or resolution.

[0058] Step S202: Construct a multivariate autoencoder model. The encoder of the multivariate autoencoder model performs latent space mapping and decoupling learning on each view in the sample data to obtain the shared variable information and unique variable information of the sample data in each view. Based on the unique variable information of the sample data in each view, the vMF distribution algorithm is used to model and sample the unique variables to obtain the unique features of the sample data in each view. After fusing the shared variable information of all views in the sample data, the Gumbel-Softmax distribution algorithm is used to model and sample the shared variables to obtain the shared features of the sample data. The unique features of the sample data in each view and the shared features of the sample data are concatenated and fused to obtain the latent representation of the sample data in each view.

[0059] In one embodiment, the architecture of the multi-VAE model is as follows: Figure 1As shown, this multi-VAE model comprises multiple VAEs, each with the same encoder and decoder. Each VAE processes one view of the sample data. Specifically, each view of the sample data in the multi-view dataset is input into the encoder of one VAE in the multi-VAE model for latent space mapping, obtaining the latent variables of the sample data in each view. Each view image of each category contains both shared information and specific information of objects in that category. To obtain this information, a shared learning branch and a specific learning branch are set in the encoder. The latent variables of the sample data in each view are input into the shared learning branch and the specific learning branch respectively for decoupled learning, obtaining the shared variable information and specific variable information of the sample data in each view.

[0060] In this application, shared variables are discrete categorical information, while unique variables are continuous, unique feature information. For unique variables, the vMF distribution algorithm is used for modeling and sampling; for shared variables, the Gumbel-Softmax distribution algorithm is used for modeling and sampling.

[0061] To model the idiosyncratic variables using the vMF distribution algorithm, the encoder outputs the vMF distribution parameters of the idiosyncratic variables, including the idiosyncratic mean vector and the idiosyncratic concentration parameter. Specifically, the hidden variables of the sample data in each view are input into the neural network of the idiosyncratic learning branch to obtain the idiosyncratic hidden vector. The idiosyncratic hidden vector is then normalized using the L2 norm to obtain the idiosyncratic mean vector, and the idiosyncratic concentration parameter is obtained by activating the idiosyncratic hidden vector using an activation function (e.g., the softplus activation function). Further, the idiosyncratic variables are modeled using the following formula based on the vMF distribution parameters output by the encoder:

[0062]

[0063] in, It is the first Variables specific to each view It is the first The unique mean vector of each view It is the first The unique concentration parameter of each view. It is a dimension of unique variables. It is the first The normalization constant of each view.

[0064] To model shared variables using the Gumbel-Softmax distribution algorithm, the encoders of all views first concatenate and fuse the shared variable information (i.e., shared hidden vectors) of all views in the sample data, and output Gumbel-Softmax distribution parameters, i.e., the probability vectors of the shared variables, based on the fused view shared vectors. Then, based on the probability vectors of the shared variables, the Gumbel-Softmax distribution algorithm is used to model the shared variables, and an approximately one-hot vector is sampled as the shared feature of the sample data. The core approach of using the Gumbel-Softmax distribution algorithm to model shared variables is to assign weights to each category in the probability vector. Add Gumbel noise get:

[0065]

[0066] in, , This represents the temperature coefficient. Then, a softmax reparameterization layer is applied to obtain a continuous shared vector. :

[0067]

[0068] Among them, temperature coefficient The lower, The closer it is to the true one-hot encoding, the better. (Input probability vector) It is generated jointly by the encoders of all views. The output is a continuous one-hot approximation. , Where N is the number of samples and K is the number of categories. This is the category number.

[0069] The vMF distribution is used to model and sample the idiomatic variables to obtain the idiomatic features of the sample data. The Gumbel-Softmax distribution algorithm is used to model and sample shared variables to obtain the shared features of the sample data. The unique features of the sample data in each view Shared features of sample data By stitching and merging the data, we can obtain the latent representation of the sample data in each view. ].

[0070] Step S203: The latent representation of the sample data in each view is reconstructed by the decoder of the multivariate autoencoder model, and the multivariate autoencoder model is jointly optimized based on the reconstruction loss and KL divergence to obtain the optimized multivariate autoencoder model.

[0071] In this application, two decoupled representations learned from the latent space are ultimately merged into a single latent representation. Specifically, the unique and shared variables of each view are combined into a new variable. This new variable is decoded and reconstructed to obtain reconstructed data, based on which the reconstruction loss can be calculated. In one embodiment, the reconstruction loss can be expressed as:

[0072]

[0073] Since this application decouples multi-view data into two variables—shared variables and unique variables—it is necessary to consider the KL divergence of these two variables separately. The formula for calculating the first KL divergence corresponding to the unique variable is as follows:

[0074]

[0075] In one embodiment, the prior distribution of the first KL divergence For a hypersphere uniform distribution, the posterior distribution is... If the distribution is vMF, then the formula for calculating the first KL divergence can be expressed as:

[0076]

[0077] in, It is the gamma function. It is a modified Bessel function of the first kind. It is the first The unique concentration parameter of each view. It is a dimension of unique variables. It is the first The normalization constant of each view.

[0078] The formula for calculating the second KL divergence corresponding to the shared variable is:

[0079]

[0080] In one embodiment, the prior distribution of the second KL divergence For uniform distribution of classification, the posterior distribution If the probability distribution is Softmax, then the formula for calculating the second KL divergence can be expressed as follows:

[0081]

[0082] in, It is the number of categories of the shared variable.

[0083] Based on this, the ELBO (Evidence Lower Bound) optimization objective for the multivariate autoencoder model is:

[0084] = - -

[0085] in, and These are the weighting coefficients. By minimizing... To minimize the reconstruction error and the KL divergence, the multivariate autoencoder (MAE) model is jointly optimized to obtain the optimized MAE model. In one embodiment, the optimization of the MAE model is stopped when the maximum number of iterations is reached, resulting in the optimized MAE model.

[0086] Please refer to Figure 3 , Figure 3 This is a flowchart of a multi-view image clustering method provided in an embodiment of this application. The method specifically includes the following steps:

[0087] Step S301, using Figure 2 The multi-view representation learning method in the model yields an optimized multivariate autoencoder model.

[0088] Step S302: Obtain the multi-view dataset to be clustered, wherein the multi-view dataset to be clustered includes multiple data to be clustered, and each data to be clustered includes multiple views.

[0089] The multi-view dataset to be clustered consists of image data captured from V perspectives of objects in one or more of the aforementioned K categories. In one embodiment, the multi-view dataset to be clustered can be either the aforementioned sample multi-view dataset or a new multi-view dataset.

[0090] Step S303: Input the multi-view dataset to be clustered into the optimized multivariate autoencoder model to obtain the latent representation of the data to be clustered in each view.

[0091] Step S304: Fuse the latent representations of the data to be clustered in each view to obtain the fused latent representation of the data to be clustered.

[0092] Step S305: Clustering is performed based on the fusion latent representation of each data to be clustered in the multi-view dataset to be clustered, and the clustering result of the multi-view dataset to be clustered is obtained.

[0093] In one embodiment, the fusion latent representation of each data to be clustered in the multi-view dataset to be clustered is input into the von Mises-Fisher Mixture Model (vMFMM) to obtain the clustering results of the multi-view dataset to be clustered.

[0094] The applicant compared the proposed multi-view clustering method based on VAE architecture and using vMF distribution with the classic clustering algorithm K-means and the multi-view clustering method based on Gaussian assumption (referred to as "Multi-VAE") based on NMI (Normalized Mutual Information) values, as shown in Table 1 below. Each of the three methods was tested 10 times, and the average value was used as the comparison index.

[0095] Table 1

[0096]

[0097] As can be seen from the comparison results in Table 1, the method of this application can obtain better multi-view clustering results (higher NMI value) compared with the classic clustering algorithm K-means and the multi-view clustering method based on Gaussian assumption.

[0098] The multi-view representation learning method provided in this application constructs a multivariate autoencoder (MAE) model. The MAE's encoder first performs latent space mapping and decoupling learning on each view in the sample data to obtain shared and unique variable information for each view. Then, based on the unique variable information for each view, the vMF distribution algorithm is used to model and sample the unique variables, obtaining the unique features of the sample data for each view. The shared variable information from all views in the sample data is then fused, and the Gumbel-Softmax distribution algorithm is used to model and sample the shared variables, obtaining the shared features of the sample data. Next, the unique features and shared features of the sample data for each view are concatenated and fused to obtain the latent representation of the sample data for each view. Finally, the MAE's decoder reconstructs the latent representation of the sample data for each view, and the MAE model is jointly optimized based on reconstruction loss and KL divergence to obtain the optimized MAE model. The method in this application fits continuous specific variables to a hypersphere using a vMF distribution, effectively overcoming the limitations of the Gaussian distribution. As a result, it has an advantage over methods based on the Gaussian assumption when dealing with complex data representation in multi-view clustering tasks, thus obtaining better clustering results.

[0099] According to an embodiment of this application, an electronic device is provided, such as... Figure 4This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 100 may include a processor 10, a communication interface 30, a memory 20, and a communication bus. The processor 10, the communication interface 30, and the memory 20 communicate with each other through the communication bus. The processor 10 can call logical instructions in the memory 20 to execute the aforementioned multi-view representation learning method or multi-view image clustering method.

[0100] Furthermore, the logical instructions in the aforementioned memory 20 can be implemented as software functional units and sold or used as independent products, and can be stored in several computer-readable storage media. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the aforementioned multi-view representation learning method or multi-view image clustering method of this application. The aforementioned storage media include: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, and other media capable of storing program code.

[0101] According to an embodiment of this application, a computer-readable storage medium of the type described above is provided. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor performs the steps of the above-described multi-view representation learning method or multi-view image clustering method.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general-purpose hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments.

[0103] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of variations or substitutions within the technical scope disclosed in this application. Therefore, any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A multi-view representation learning method, characterized in that, The method comprises: acquiring a sample multi-view data set, the sample multi-view data set comprising a plurality of sample data, each sample data comprising a plurality of views; constructing a multi-variational autoencoder model, performing latent space mapping and decoupling learning on each view in the sample data by an encoder of the multi-variational autoencoder model to obtain shared variable information and unique variable information of each view of the sample data, modeling and sampling the unique variable by using a vMF distribution algorithm based on the unique variable information of each view of the sample data to obtain unique features of each view of the sample data, fusing the shared variable information of all views in the sample data, modeling and sampling the shared variable by using a Gumbel-Softmax distribution algorithm to obtain shared features of the sample data, and splicing and fusing the unique features of each view of the sample data and the shared features of the sample data to obtain latent representations of each view of the sample data; reconstructing the latent representations of each view of the sample data by a decoder of the multi-variational autoencoder model, and jointly optimizing the multi-variational autoencoder model based on a reconstruction loss and a KL divergence to obtain an optimized multi-variational autoencoder model; the method comprises: performing latent space mapping by one variational autoencoder in the multi-variational autoencoder model based on each view in the sample data to obtain hidden variables of each view of the sample data; inputting the hidden variables of each view of the sample data into a shared learning branch and a unique learning branch for decoupling learning to obtain shared variable information and unique variable information of each view of the sample data, respectively; the shared variable information is a shared hidden vector, and the method comprises: splicing and fusing the shared hidden vectors of all views in the sample data to obtain a probability vector of the shared variable; modeling the shared variable by using a Gumbel-Softmax distribution algorithm based on the probability vector, and sampling an approximately one-hot vector as the shared features of the sample data.

2. The method of claim 1, wherein, The multi-variational autoencoder model comprises a plurality of variational autoencoders, each variational autoencoder corresponding to one view of the sample data.

3. The method of claim 1, wherein, The unique variable information comprises a unique mean vector and a unique concentration parameter, and the method comprises: inputting the hidden variables of each view of the sample data into a neural network of the unique learning branch to obtain a unique hidden vector, performing L2 norm normalization on the unique hidden vector to obtain the unique mean vector, and activating the unique hidden vector by an activation function to obtain the unique concentration parameter.

4. The method of claim 3, wherein, The formula for modeling the unique variable by using the vMF distribution algorithm is: wherein, is a view-specific variable, is a view-specific mean vector, is a view-specific concentration parameter, is a view-specific concentration parameter, is a view-specific concentration parameter, is a view-specific concentration parameter, is a dimension of the view-specific variable, is a normalization constant for the view, is a normalization constant for the view.

5. The method according to any one of claims 1 to 4, characterized in that, The KL divergence includes a first KL divergence corresponding to the unique variable and a second KL divergence corresponding to the shared variable, the prior distribution of the first KL divergence is a hyperspherical uniform distribution, and the posterior distribution is a vMF distribution, the prior distribution of the second KL divergence is a categorical uniform distribution, and the posterior distribution is a Softmax probability distribution.

6. A multi-view image clustering method, characterized by, The method comprises: The method comprises: Based on the multi-view representation learning method according to any one of claims 1 to 5, an optimized multi-variate autoencoder model is obtained; Obtaining a multi-view data set to be clustered, the multi-view data set to be clustered comprising a plurality of data to be clustered, each data to be clustered comprising a plurality of views; Inputting the multi-view data set to be clustered into the optimized multi-variate autoencoder model to obtain a latent representation of each view of the data to be clustered; Fusing the latent representation of each view of the data to be clustered to obtain a fused latent representation of the data to be clustered; 7. An electronic device, comprising: Clustering based on the fused latent representation of each data to be clustered in the multi-view data set to be clustered to obtain a clustering result of the multi-view data set to be clustered.

8. A computer storage medium, characterized in that The computer storage medium stores instructions or programs, when the instructions or programs are executed by at least one processor, the at least one processor executes the method according to any one of claims 1 to 6. The computer storage medium stores instructions or programs, when the instructions or programs are executed by at least one processor, the at least one processor executes the method according to any one of claims 1 to 6.

Images