An incremental deep multi-view clustering approach

Abstract

The present application discloses an incremental depth multi-view clustering method, including: extracting latent variables of multi-view data using a variational autoencoder, obtaining reconstructed data based on the latent variables, and calculating the first loss value of the loss function of the variational autoencoder based on the reconstructed data; if the first loss value is less than a first preset threshold, updating the clustering parameters of the DPMM model using the current latent variables to obtain the clustering labels output by the updated DPMM model; adding the updated clustering labels to the loss function of the variational autoencoder to obtain the total loss function, and if the second loss value of the total loss function is less than a second preset threshold, outputting the clustering labels as the clustering results of the multi-view data. The present application can better capture the complex distribution of data of different views, so that the number of clusters of the DPMM model can remain stable. At the same time, it can effectively use existing information for a new learning process without retraining, and is suitable for a variety of practical application scenarios.

Description

Technical Field

[0001] The present application relates to the technical field of multi-view clustering, and in particular to a multi-view clustering method with incremental depth. Background Art

[0002] With the popularity of social media platforms and various photography tools, the output rate of images has significantly increased. As images can convey a large amount of information intuitively and vividly, they have become an extremely valuable resource in the era of big data. Image clustering can play an important role in many application fields such as image retrieval and medical image analysis. Traditional clustering algorithms have performed quite well in processing static and unchanging data sets, but in practical applications, dynamically changing data streams are often encountered, and these data often arrive in blocks or batches. For example, in fields such as social media analysis, incremental clustering algorithms are particularly important in order to avoid the need to start clustering from scratch every time new data arrives.

[0003] In existing technologies, incremental clustering usually combines the concepts of online learning and streaming data processing, and uses memory mechanisms to retain important information, thereby keeping the model updated and adaptable in a changing data environment. In addition, the rapid development of deep learning technology in recent years has provided image clustering with more powerful feature extraction capabilities, especially when processing high-dimensional and complex data. Deep generative models such as variational autoencoders (VAE) and generative adversarial networks (GAN) have opened up new perspectives for clustering tasks, making image clustering more flexible and efficient in multi-view learning and dynamic data streams.

[0004] However, when dealing with dynamically changing data, the number of clusters is often not fixed, which makes traditional clustering methods face challenges. In addition, with the introduction of multi-view data, the heterogeneity and information redundancy between different views are further increased, making the clustering task more complicated. At the same time, the model may forget the knowledge of old data when learning new data, resulting in information loss. Summary of the invention

[0005] The purpose of this application is to provide an incremental depth multi-view clustering method to solve the technical problems of the existing clustering model in the prior art, such as the non-fixed number of clusters and poor feature sharing performance. The preferred technical solutions among the many technical solutions provided in this application can produce many technical effects as described below.

[0006] To achieve the above objectives, this application provides the following technical solutions:

[0007] The first aspect of the present application provides an incremental depth multi-view clustering method, comprising: extracting latent variables of multi-view data using a variational autoencoder, obtaining reconstructed data based on the latent variables, and calculating a first loss value of the loss function of the variational autoencoder based on the reconstructed data; if the first loss value is less than a first preset threshold, updating the clustering parameters of the DPMM model using the current latent variables output by the variational autoencoder to obtain a clustering label output by the updated DPMM model; adding the updated clustering label to the loss function of the variational autoencoder to obtain a total loss function, and if the second loss value of the total loss function is less than a second preset threshold, outputting the clustering label as the clustering result of the multi-view data.

[0008] In some embodiments, the method further includes: initializing the variational autoencoder, constructing an independent encoder and an exclusive decoder for each view in the multi-view data; and constructing a shared encoder and a shared decoder for the multi-view data.

[0009] In some embodiments, the use of a variational autoencoder to extract latent variables of multi-view data includes: encoding each view using a corresponding independent encoder to obtain a specific latent variable; encoding all views using the shared encoder to generate a shared latent variable; decoding the specific latent variable using the exclusive decoder, and decoding the shared latent variable using the shared decoder to obtain the reconstructed data.

[0010] In some embodiments, the first loss value of the loss function of the variational autoencoder is calculated based on the reconstructed data, including: calculating the reconstruction loss and KL divergence loss of each view; combining the reconstruction losses of all views and the KL divergence losses to obtain the first loss value.

[0011] In some embodiments, the method further includes: if the first loss value is greater than or equal to the first preset threshold, updating the parameters of the variational autoencoder using a back propagation algorithm.

[0012] In some embodiments, the method of updating the clustering parameters of the DPMM model using the current latent variables output by the variational autoencoder includes: recalculating the cluster mean, covariance and mixing weight of each cluster according to each of the cluster labels, and creating new clusters and / or combining clusters according to the calculation results.

[0013] In some embodiments, adding the updated cluster labels to the loss function of the variational autoencoder to obtain a total loss function includes: using the label information of the cluster labels to add cluster consistency loss to the loss function of the variational autoencoder to obtain the total loss function.

[0014] In some embodiments, the method further includes: if the second loss value of the total loss function is greater than or equal to the second preset threshold, using the second loss value to update the loss function of the variational autoencoder.

[0015] A second aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed, the multi-view clustering method with incremental depth as described above is implemented.

[0016] The third aspect of the present application provides a processing device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors are used to execute the one or more computer programs stored in the memory, so that the one or more processors perform the incremental depth multi-view clustering method as described above.

[0017] Implementing one of the above technical solutions of the present application has the following advantages or beneficial effects: In the present application, by using a variational autoencoder to extract latent variables of multi-view data, using the latent variables to update the clustering parameters of the DPMM model (Dirichlet Process Mixture Model), and finally using the clustering labels output by the DPMM model to update the loss function of the variational autoencoder, the variational autoencoder and the DPMM model are trained simultaneously to improve model performance. In this case, the complex distribution of data from different views can be better captured, so that the number of clusters of the DPMM model can remain stable. At the same time, by further training the model through updated cluster labels, the existing information can be effectively used for a new learning process without retraining, which is suitable for a variety of practical application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. It is obvious that the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0019] Figure 1 is a flowchart of an incremental depth multi-view clustering method according to an embodiment of the present application;

[0020] Figure 2 It is a structural block diagram of the processing device of an embodiment of the present application.

[0021] In the figure: 1. processing device; 10. memory; 11. processor. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present application clearer, the various exemplary embodiments to be described below will refer to the corresponding drawings, which constitute a part of the exemplary embodiments, wherein various exemplary embodiments that may be used to implement the present application are described. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present disclosure. It should be understood that they are only examples of processes, methods, and devices that are consistent with some aspects disclosed in the present application as detailed in the attached claims, and other embodiments may also be used, or the embodiments listed herein may be modified in structure and function without departing from the scope and essence of the present application.

[0023] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", etc. indicate the orientation or positional relationship based on the drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the elements referred to must have a specific orientation, be constructed and operated in a specific orientation. The terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. The term "multiple" means two or more. The terms "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, an integral connection, a mechanical connection, an electrical connection, a communication connection, a direct connection, an indirect connection through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0024] In order to illustrate the technical solution described in the present application, a specific embodiment is provided below, and only the parts related to the embodiment of the present application are shown.

[0025] like Figure 1 As shown, the present application provides an incremental depth multi-view clustering method, including (step S1 to step S3):

[0026] S1. Use a variational autoencoder to extract latent variables of multi-view data, obtain reconstructed data based on the latent variables, and calculate a first loss value of a loss function of the variational autoencoder based on the reconstructed data.

[0027] In some embodiments, the incremental depth multi-view clustering method may further include: obtaining multi-view data from a multi-view dataset; normalizing each view in the multi-view data respectively, and then inputting the normalized multi-view data into a variational autoencoder.

[0028] In some embodiments, acquiring the multi-view data from the multi-view dataset may include: randomly sampling a small batch of multi-view data from the multi-view dataset.

[0029] In some embodiments, a small batch of multi-view data may be represented by the following formula:

[0030] ,

[0031] in, Can be batch size.

[0032] In some embodiments, the normalization process may refer to normalizing the pixel value of each view to [0, 1], thereby ensuring that data distributions between different views are similar.

[0033] In some embodiments, the incremental depth multi-view clustering method may further include: initializing a variational autoencoder, constructing an independent encoder and an exclusive decoder for each view in the multi-view data; and constructing a shared encoder and a shared decoder for the multi-view data.

[0034] In some embodiments, the incremental deep multi-view clustering method may further include: defining a dimension of a latent space of the variational autoencoder.

[0035] In some embodiments, the encoder network and the decoder network can be implemented by deep neural networks.

[0036] In some embodiments, based on the dimension of the latent space, independent encoders may be used to map view data of corresponding views to a specific latent space of the view, and a shared encoder may be used to map view data of all views to a shared latent space.

[0037] In some embodiments, a variational autoencoder is used to extract latent variables of multi-view data, and reconstructed data is obtained based on the latent variables, which may include: encoding each view using a corresponding independent encoder to obtain a specific latent variable; encoding all views using a shared encoder to generate a shared latent variable; decoding a specific latent variable using an exclusive decoder, and decoding the shared latent variable using a shared decoder to obtain reconstructed data.

[0038] In some embodiments, encoding all views using a shared encoder to generate shared latent variables may include: inputting multi-view data into a neural network to extract common features, and outputting the mean and variance of the shared latent variables, and generating shared latent variables using a reparameterization technique.

[0039] In some embodiments, the reconstructed data may include specific reconstructed data and shared reconstructed data, and the decoder may be a structure that combines a shared decoder and multiple exclusive decoders. Specifically, the exclusive decoder may be used to map specific latent variables back to the original data space to obtain specific reconstructed data; correspondingly, the shared latent variables may be mapped back to the original data space to obtain shared reconstructed data.

[0040] In some embodiments, calculating a first loss value of a loss function of a variational autoencoder based on reconstructed data may include: calculating a reconstruction loss and a KL divergence loss for each view; and combining the reconstruction losses and KL divergence losses of all views to obtain a first loss value.

[0041] In some embodiments, the reconstruction loss can be used to measure the difference between the generated reconstructed data and the original data, that is, the view data, and the reconstruction loss can use at least one of the mean squared error (MSE) and the cross-entropy loss (Cross-Entropy Loss). The KL divergence loss (Kullback-Leibler Divergence Loss) can be used to constrain the distribution of latent variables so that the distribution of latent variables is close to the prior distribution.

[0042] In some embodiments, the KL divergence loss may include a specific KL divergence loss and a shared KL divergence loss, wherein the specific KL divergence loss may refer to the KL divergence loss corresponding to each view, and the shared KL divergence loss may refer to the KL divergence loss of the shared latent space.

[0043] In some embodiments, the reconstruction loss of all views, the specific KL divergence loss, and the shared KL divergence loss may be combined to obtain a first loss value.

[0044] In some embodiments, the incremental depth multi-view clustering method may further include: if the first loss value is greater than or equal to a first preset threshold, updating the parameters of the variational autoencoder using a back propagation algorithm.

[0045] In some embodiments, the back-propagation algorithm may update the weights and biases of the variational autoencoder by calculating the gradient of the loss function of the variational autoencoder with respect to the network parameters.

[0046] S2. If the first loss value is less than a first preset threshold, the clustering parameters of the DPMM model (Dirichlet Process Mixture Model) are updated using the current latent variable output by the variational autoencoder to obtain the clustering label output by the updated DPMM model.

[0047] In some embodiments, the incremental depth multi-view clustering method may further include: initializing the DPMM model and initializing the number of clusters to 1.

[0048] In some embodiments, the mean of each cluster of the DPMM model can be sampled from a prior distribution, and the probability of each cluster can be sampled from Sampling from a distribution, the cluster assignment of each data point can be sampled from a discrete distribution. This allows the sparsity of the clusters to be controlled.

[0049] In some embodiments, from Sampling from a distribution can be expressed as:

[0050] ,

[0051] in Can be a concentration parameter, It can represent a sequence of clustering probabilities.

[0052] In some embodiments, sampling from a discrete distribution can be expressed as:

[0053] ,

[0054] in, Can represent cluster labels.

[0055] In some embodiments, the incremental depth multi-view clustering method may further include: using the DPMM model to calculate the clustering probability of each data point in the latent variable and assigning a clustering label. In some embodiments, a variational Bayesian method may be used, assuming that the posterior distribution of each latent variable is a variational distribution , and by minimizing the KL divergence To approximate the true posterior distribution. The DPMM model can calculate each data point based on the distribution of the latent variables. The probability distribution of belonging to each cluster and assigning cluster labels.

[0056] In some embodiments, the latent variables may be used as inputs of the DPMM model, and the DPMM model may perform clustering using the latent variables to obtain clustering results including clustering labels.

[0057] In some embodiments, the similarity between data points can be determined by the latent variables corresponding to the data points and the corresponding cluster means.

[0058] In some embodiments, the probability distribution may be expressed as:

[0059] ,

[0060] in, can be expressed as cluster labels, can be represented as a latent variable and k can represent a cluster.

[0061] In some embodiments, updating the clustering parameters of the DPMM model using the current latent variables output by the variational autoencoder may include: recalculating the cluster mean, covariance, and mixing weight of each cluster according to each cluster label, and creating new clusters and / or combining clusters according to the calculation results. Specifically, if the data point corresponding to the cluster label differs greatly from the cluster mean of the existing cluster, the DPMM model can create a new cluster as the clustering result. The DPMM model can determine whether to merge based on the similarity of the means and variances of the existing clusters. For existing clusters with high similarity, the DPMM model can combine some or all of the existing clusters to optimize the clustering structure.

[0062] In some embodiments, the clustering parameters of the DPMM model can be updated by online variational inference. In this case, the memory of the previous view data can be maintained by online variational inference, and the existing information can be effectively used to determine the learning process of the DPMM model.

[0063] In some embodiments, the clustering label of each data point in the current iteration may be recorded, thereby facilitating tracking of clustering changes of data points in subsequent steps.

[0064] In some embodiments, if the clustering labels of the updated DPMM model change, the variational autoencoder can readjust the latent space according to the latest label feedback, thereby making the same type of view data more clustered in the latent space.

[0065] S3. Add the updated clustering label to the loss function of the variational autoencoder to obtain a total loss function. If the second loss value of the total loss function is less than a second preset threshold, the clustering label is output as a clustering result of the multi-view data.

[0066] In some embodiments, adding the updated cluster labels to the loss function of the variational autoencoder to obtain the total loss function may include: using the label information of the cluster labels to add the cluster consistency loss to the loss function of the variational autoencoder to obtain the total loss function.

[0067] In some embodiments, the cluster consistency loss can be expressed by the following formula:

[0068] ,

[0069] in, It can refer to the first All pairs of data points in a cluster, and can be a shared latent variable for two data points belonging to the same cluster.

[0070] The cluster consistency loss can encourage data points in the same cluster to be closer in the shared latent space, while data points in different clusters are better separated in the latent space.

[0071] In some embodiments, the total loss function can be expressed by the following formula:

[0072] ,

[0073] in, can refer to the reconstruction loss, Can refer to a specific KL divergence loss, can refer to the shared KL divergence loss, Can refer to cluster consistency loss.

[0074] In some embodiments, the multi-view clustering method with incremental depth may further include: if the second loss value of the total loss function is greater than or equal to a second preset threshold, updating the loss function of the variational autoencoder using the second loss value. For example, the weight of the loss function of the variational autoencoder may be adjusted by the second loss value.

[0075] In some embodiments, when the loss function of the variational autoencoder remains stable and the clustering labels of the DPMM model remain unchanged, the multi-view data may be replaced for re-clustering to obtain a clustering result of the replaced multi-view data.

[0076] In this application, the variational autoencoder is used to extract the latent variables of multi-view data, the latent variables are used to update the clustering parameters of the DPMM model, and finally the cluster labels output by the DPMM model are used to update the loss function of the variational autoencoder, so as to simultaneously train the variational autoencoder and the DPMM model to improve the model performance. In this case, the complex distribution of different view data can be better captured, so that the number of clusters of the DPMM model can remain stable. At the same time, by further training the model through updated cluster labels, the existing information can be effectively used for a new learning process without retraining, which is suitable for a variety of practical application scenarios.

[0077] Those skilled in the art can understand that all or part of the features / steps of the above-mentioned method embodiments can be implemented by methods, data processing systems or computer programs, and these features can be implemented without hardware, entirely by software, or by a combination of hardware and software. The aforementioned computer program can be stored in one or more computer-readable storage media, and the storage medium stores the computer program. When the computer program is executed (such as by a processor), the steps of the above-mentioned multi-view clustering method embodiment including the incremental depth are executed.

[0078] The aforementioned storage media that can store program codes include: static hard disks, solid-state hard disks, random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), optical storage devices, magnetic storage devices, flash memory, magnetic disks or optical disks and / or combinations of the above devices, that is, they can be implemented by any type of volatile or non-volatile storage device or a combination thereof.

[0079] like Figure 2 As shown, the present application also provides an embodiment of a processing device 1, comprising one or more processors 11 and a memory 10; wherein the memory 10 is used to store one or more computer programs, and the one or more processors 11 are used to execute one or more computer programs stored in the memory 10, so that the processor 11 performs the features / steps of the above-mentioned incremental depth multi-view clustering method embodiment.

[0080] The above is only the preferred embodiment of the present application. Those skilled in the art will appreciate that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present application. In addition, under the guidance of the present application, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present application. Therefore, the present application is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the present application belong to the protection scope of the present application.

Claims

1. An incremental depth multi-view clustering method, characterized in that include: Extracting latent variables of multi-view data using a variational autoencoder, obtaining reconstructed data based on the latent variables, and calculating a first loss value of a loss function of the variational autoencoder based on the reconstructed data, wherein the first loss value is a combination of reconstruction losses of all views, a specific KL divergence loss, and a shared KL divergence loss; If the first loss value is less than a first preset threshold, the clustering parameters of the DPMM model are updated using the current latent variables output by the variational autoencoder to obtain the clustering labels output by the updated DPMM model; Adding the updated clustering label to the loss function of the variational autoencoder to obtain a total loss function, and if a second loss value of the total loss function is greater than or equal to a second preset threshold, updating the loss function of the variational autoencoder using the second loss value, and retraining the variational autoencoder after the update; If the second loss value of the total loss function is less than the second preset threshold, the cluster label is output as the clustering result of the multi-view data; the adding the updated cluster label to the loss function of the variational autoencoder to obtain the total loss function includes: using the label information of the cluster label, adding the cluster consistency loss to the loss function of the variational autoencoder to obtain the total loss function; the total loss function is expressed by the following formula: ， in, refers to the reconstruction loss, refers to the specific KL divergence loss, refers to the shared KL divergence loss, refers to the clustering consistency loss.

2. The multi-view clustering method with incremental depth according to claim 1, characterized in that: The method also includes: initializing the variational autoencoder, constructing an independent encoder and an exclusive decoder for each view in the multi-view data; and constructing a shared encoder and a shared decoder for the multi-view data.

3. The multi-view clustering method with incremental depth according to claim 2, characterized in that: The method of extracting latent variables of multi-view data using a variational autoencoder includes: encoding each view using a corresponding independent encoder to obtain a specific latent variable; encoding all views using the shared encoder to generate a shared latent variable; decoding the specific latent variable using the exclusive decoder, and decoding the shared latent variable using the shared decoder to obtain the reconstructed data.

4. The multi-view clustering method with incremental depth according to claim 1, characterized in that: The method also includes: if the first loss value is greater than or equal to the first preset threshold, using a back propagation algorithm to update the parameters of the variational autoencoder.

5. The multi-view clustering method with incremental depth according to claim 1, characterized in that: The method of updating the clustering parameters of the DPMM model using the current latent variables output by the variational autoencoder includes: recalculating the cluster mean, covariance and mixing weight of each cluster according to each cluster label, and creating new clusters and / or combining clusters according to the calculation results.

6. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which, when executed, implements the incremental depth multi-view clustering method according to any one of claims 1 to 5.

7. A processing device, characterized in that: include: one or more processors; A memory for storing one or more computer programs, and one or more processors for executing the one or more computer programs stored in the memory, so that the one or more processors perform the incremental depth multi-view clustering method according to any one of claims 1 to 5.

Images