Multi-view clustering method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of multi-view clustering, in particular to a multi-view clustering method and device, electronic equipment and a storage medium, wherein the method comprises the following steps: obtaining a plurality of views to be clustered; extracting at least one image feature of each view to obtain a plurality of image features; inputting the plurality of image features into a clustering model pre-trained to extract complementary information and structural information between different views by using a preset common hidden space and a preset structured hidden space of the clustering model; matching the actual category of each view by using the complementary information and the structural information; clustering the plurality of views based on the actual category to output a clustering result of the plurality of views. Thus, the problems that the related art constructs a graph based on an original data matrix or a shallow feature, and fuses a plurality of graphs by using different fusion strategies, are solved, the calculation amount is large, the clustering effect is seriously dependent on the quality of original graphing, and the effect of multi-view clustering is limited.

Description

Technical Field

[0001] This application relates to the field of multi-view clustering technology, and in particular to a multi-view clustering method, apparatus, electronic device and storage medium. Background Technology

[0002] Clustering is an unsupervised learning task. The basic idea of ​​clustering algorithms is to divide a set of data objects according to certain criteria, grouping similar objects into the same class and different objects into different classes. In the information age, the amount of data is growing exponentially, making it even more necessary to divide and organize similar objects to help researchers more easily distinguish their internal logic and reduce the cost of data annotation. Currently, clustering algorithms are widely used in computer vision, data mining, pattern recognition, and other fields.

[0003] With the rapid development of information technology, data collection methods have become increasingly diversified, resulting in massive amounts of multi-view data. For example, news can be reported in different languages ​​and in the form of text, images, and videos; an image can be represented by different features, such as LBP (Local Binary Patterns, 100-dimensional) and HOG (Histogram of Oriented Gradients, 256-dimensional). Although the final representation of the data may differ, they all represent the same thing. Generally speaking, multi-view data contains richer information than a single view, and rationally integrating this information from different perspectives can lead to a more comprehensive analysis of the problem. However, most traditional methods focus primarily on a single view and cannot be directly applied to multi-view data. Therefore, in recent years, numerous multi-view learning methods have been proposed for various tasks, such as classification and clustering.

[0004] In the field of multi-view learning, the goal of multi-view clustering (MVC) is to rationally integrate information from different views, adaptively classifying data into their respective categories without any label information. By considering the consistency and complementarity between different views, multi-view clustering achieves better clustering results than single-view clustering. Because multi-view clustering reduces the time and manpower costs of multi-view data annotation, it has attracted increasing attention.

[0005] With the development of multimedia, more and more data is presented in multiple modalities or features; this type of data is called multi-view data. Multi-view clustering has attracted increasing attention because it can reduce the annotation costs of massive amounts of data. Most related technologies consider leveraging the consistency and complementarity of views to fuse information from multiple views. Typically, this involves constructing graphs based on the original data matrix or shallow features, and then using different fusion strategies to combine the multiple graphs. However, graph learning-based multi-view clustering methods are not only computationally intensive, but their clustering results also heavily depend on the quality of the original graph construction, thus limiting the effectiveness of multi-view clustering. Summary of the Invention

[0006] This application provides a multi-view clustering method, apparatus, electronic device, and storage medium to solve the problems of related technologies that construct graphs based on original data matrices or shallow features, which involve large computational loads and whose clustering effect is heavily dependent on the quality of the original graph, thus limiting the effectiveness of multi-view clustering.

[0007] The first aspect of this application provides a multi-view clustering method, comprising the following steps: acquiring multiple views to be clustered; extracting at least one image feature from each view to obtain multiple image features; inputting the multiple image features into a pre-trained clustering model; extracting complementary information and structural information between different views using the preset common latent space and preset structured latent space of the clustering model, matching the actual category of each view using the complementary information and the structural information, clustering the multiple views based on the actual category, and outputting the clustering result of the multiple views.

[0008] Based on the aforementioned technical means, embodiments of this application can extract image features from multiple views to be clustered, input them into a pre-trained clustering model, flexibly mine complementary information of different views through a common latent space representation, and utilize the structured latent space to obtain structural information between views, thereby achieving classification of each view category and further obtaining clustering results for multiple views. Simultaneously considering the complementary information between views and the structural relationships between samples, it can not only flexibly fuse information from multiple views but also fully utilize the structural information between samples, thus making the clustering results more accurate.

[0009] Optionally, in one embodiment of this application, the clustering model is trained based on training data carrying labels of real clustering results, including: acquiring training data carrying labels of real clustering results, wherein the training data includes multiple view samples; inputting the multiple view samples into the preset common latent space and the preset structured latent space respectively, and outputting complementary information and structural information between the view samples; calculating the actual similarity between the sample points of each view sample and the preset cluster center according to the complementary information and the structural information, determining the actual category of each view sample according to the actual similarity, clustering the multiple view samples based on the actual category to obtain a training clustering result; calculating a training loss value according to the training clustering result and the real clustering result; when the training loss value is greater than a convergence threshold, optimizing the cluster center from the current confidence allocation using the target distribution determined by the actual similarity, and updating the preset structured latent space, until the training loss value is less than or equal to the convergence threshold, stopping iterative training, and obtaining a trained clustering model.

[0010] Based on the aforementioned technical means, the embodiments of this application can simultaneously optimize latent space representation learning and clustering, thereby making the learned structured latent space representation more suitable for clustering tasks. First, the soft assignment between each sample point in the latent space representation and the cluster centroid is calculated. Second, the structured latent space representation is updated, and the cluster centroids are refined by learning from the current high-confidence assignments using an auxiliary target distribution. This process is repeated until convergence is achieved, making the feature representation of each sample closer to the cluster center, thereby improving cluster cohesion.

[0011] Optionally, in one embodiment of this application, the step of calculating the training loss value based on the training clustering result and the real clustering result includes: calculating the reconstruction loss and the clustering loss based on the training clustering result and the real clustering result; obtaining the weight parameters of the clustering loss; and calculating the training loss value based on the weight parameters, the clustering loss, and the reconstruction loss.

[0012] Based on the above technical means, the embodiments of this application can calculate the training loss value based on the weight parameters of the reconstruction loss, clustering loss and clustering loss to obtain the training clustering result and the real clustering result, so that the learned latent space representation has a better cluster structure.

[0013] Optionally, in one embodiment of this application, before inputting the plurality of view samples into the preset common latent space and the preset structured latent space respectively, and outputting the complementary information and structural information between the view samples, the method includes: obtaining the latent space representation of any view in the preset samples; calculating the mapping relationship between the latent space representation and the arbitrary view with the minimum reconstruction loss of the latent space representation as the optimization objective; and constructing the preset common latent space based on the mapping relationship and the preset multilayer perceptron.

[0014] Based on the above technical means, the embodiments of this application can reconstruct each view based on the latent space representation of the view in the preset samples, so as to realize the mapping between the common latent space representation and the original image features of the original data, thereby more comprehensively describing the image data itself and making the latent space representation more accurate and robust.

[0015] Optionally, in one embodiment of this application, before inputting the plurality of view samples into the preset common latent space and the preset structured latent space respectively, and outputting the complementary information and structural information between the view samples, the method includes: constructing an undirected k-nearest neighbor graph using the preset common latent space; inputting the plurality of view samples into the undirected k-nearest neighbor graph to obtain the actual nearest neighbor relationships between the plurality of view samples; using the actual nearest neighbor relationships and the preset common latent space representation as input to a preset graph convolutional network, outputting the updated representation of the preset common latent space, and performing layer-by-layer convolution through the preset graph convolutional network to obtain the updated representation of each convolutional layer, and constructing the preset structured latent space based on the updated representation of each convolutional layer.

[0016] Based on the aforementioned technical means, this application embodiment can construct a k-nearest neighbor graph based on a common latent space, and utilize a graph convolutional network to mine information between samples in the common latent space representation. Information about a node's neighbors is collected through nonlinear transformations and aggregation functions, and used to update the node's features. Therefore, the topological structure of the data can be preserved, while making the features of neighboring nodes more similar. Furthermore, the interaction between samples is enhanced through the graph convolutional network, thereby learning a structured latent space representation. Similar samples have more similar features during the graph convolution process, exhibiting better structure.

[0017] A second aspect of this application provides a multi-view clustering method, comprising the following steps: acquiring training data carrying labels of real clustering results, wherein the training data includes multiple view samples; inputting the multiple view samples into a preset common latent space and a preset structured latent space respectively, and outputting complementary information and structural information between the view samples; calculating the actual similarity between the sample points of each view sample and the preset clustering center based on the complementary information and the structural information, determining the actual category of each view sample based on the actual similarity, clustering the multiple view samples based on the actual category to obtain a training clustering result; calculating a training loss value based on the training clustering result and the real clustering result; when the training loss value is greater than a convergence threshold, optimizing the clustering center from the current confidence allocation using the target distribution determined by the actual similarity, and updating the preset structured latent space, until the training loss value is less than or equal to the convergence threshold, stopping iterative training, obtaining a trained clustering model, and using the clustering model to cluster multiple views to be clustered to obtain the clustering result of the multiple views.

[0018] Based on the above technical means, the embodiments of this application train and correct the clustering model according to the training data results and the training loss value until the preset convergence value is met, and then stop training. In this way, the clustering results of multiple views to be clustered are obtained by using the trained clustering model, which has high accuracy and good clustering effect.

[0019] A third aspect of this application provides a multi-view clustering apparatus, comprising: a first acquisition module for acquiring multiple views to be clustered; an extraction module for extracting at least one image feature from each view to obtain multiple image features; and a first clustering module for inputting the multiple image features into a pre-trained clustering model, extracting complementary information and structural information between different views using a preset common latent space and a preset structured latent space of the clustering model, matching the actual category of each view using the complementary information and the structural information, clustering the multiple views based on the actual category, and outputting the clustering result of the multiple views.

[0020] Optionally, in one embodiment of this application, the first clustering module is further configured to acquire training data carrying labels of real clustering results, wherein the training data includes multiple view samples; input the multiple view samples into a preset common latent space and a preset structured latent space respectively, and output complementary information and structural information between the view samples; calculate the actual similarity between the sample points of each view sample and the preset cluster center based on the complementary information and the structural information, determine the actual category of each view sample based on the actual similarity, cluster the multiple view samples based on the actual category to obtain training clustering results; calculate the training loss value based on the training clustering results and the real clustering results; when the training loss value is greater than the convergence threshold, optimize the cluster center from the current confidence allocation using the target distribution determined by the actual similarity, and update the preset structured latent space, until the training loss value is less than or equal to the convergence threshold, stop iterative training, and obtain the trained clustering model.

[0021] Optionally, in one embodiment of this application, the first clustering module is further configured to calculate reconstruction loss and clustering loss based on the training clustering results and the real clustering results; obtain the weight parameters of the clustering loss; and calculate the training loss value based on the weight parameters, the clustering loss, and the reconstruction loss.

[0022] Optionally, in one embodiment of this application, it further includes: a computing unit, configured to obtain the latent space representation of any view in the preset samples before inputting the plurality of view samples into the preset common latent space and the preset structured latent space respectively, and outputting the complementary information and structural information between the view samples; and to calculate the mapping relationship between the latent space representation and the arbitrary view with the minimum reconstruction loss of the latent space representation as the optimization objective.

[0023] The first construction unit is used to construct the preset common latent space based on the mapping relationship and the preset multilayer perceptron.

[0024] Optionally, in one embodiment of this application, it further includes: a generation unit, configured to construct an undirected k-nearest neighbor graph using the preset common latent space before inputting the plurality of view samples into the preset common latent space and the preset structured latent space respectively, and outputting complementary information and structural information between the view samples; inputting the plurality of view samples into the undirected k-nearest neighbor graph to obtain the actual nearest neighbor relationships between the plurality of view samples;

[0025] The second construction unit is used to take the actual nearest neighbor relationship and the preset common latent space representation as input to the preset graph convolutional network, output the updated representation of the preset common latent space, and perform layer-by-layer convolution through the preset graph convolutional network to obtain the updated representation of each convolution layer, and construct the preset structured latent space based on the updated representation of each convolution layer.

[0026] A fourth aspect of this application provides a multi-view clustering apparatus, comprising: a second acquisition module for acquiring training data carrying labels of real clustering results, wherein the training data includes multiple view samples; a processing module for inputting the multiple view samples into a preset common latent space and a preset structured latent space respectively, and outputting complementary information and structural information between the view samples; a training module for calculating the actual similarity between the sample points of each view sample and the preset clustering center based on the complementary information and the structural information, determining the actual category of each view sample based on the actual similarity, and clustering the multiple view samples based on the actual category to obtain a training clustering result; and a second clustering module for calculating a training loss value based on the training clustering result and the real clustering result, optimizing the clustering center from the current confidence allocation using the target distribution determined by the actual similarity when the training loss value is greater than a convergence threshold, and updating the preset structured latent space, stopping iterative training when the training loss value is less than or equal to the convergence threshold, obtaining a trained clustering model, and using the clustering model to cluster multiple views to be clustered to obtain the clustering result of the multiple views.

[0027] A fifth aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-view clustering method as described in the above embodiments.

[0028] A sixth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the multi-view clustering method as described in the above embodiments.

[0029] Therefore, this application has at least the following beneficial effects:

[0030] 1. This application embodiment can extract image features from multiple views to be clustered, input them into a pre-trained clustering model, flexibly mine complementary information of different views through a common latent space representation, and obtain structural information between views using the structured latent space, thereby achieving classification of each view category and further obtaining clustering results for multiple views. Simultaneously considering complementary information between views and structural relationships between samples, it can not only flexibly fuse information from multiple views but also fully utilize structural information between samples, thus making the clustering results more accurate.

[0031] 2. The embodiments of this application can simultaneously optimize latent space representation learning and clustering, thereby making the learned structured latent space representation more suitable for clustering tasks. First, the soft assignment between each sample point in the latent space representation and the cluster centroid is calculated. Second, the structured latent space representation is updated, and the cluster centroid is refined by learning from the current high-confidence assignment using an auxiliary target distribution. By repeating this process until convergence is achieved, the feature representation of each sample is made closer to the cluster center, thereby improving cluster cohesion.

[0032] 3. In this embodiment of the application, the training loss value can be calculated based on the weight parameters of the reconstruction loss, clustering loss and clustering loss to obtain the training clustering result and the real clustering result, so that the learned latent space representation has a better cluster structure.

[0033] 4. The embodiments of this application can reconstruct each view based on the latent space representation of the view in the preset samples, so as to realize the mapping between the common latent space representation and the original image features of the original data, thereby more comprehensively describing the image data itself and making the latent space representation more accurate and robust.

[0034] 5. This application embodiment constructs a k-nearest neighbor graph based on a common latent space and utilizes a graph convolutional network to mine information between samples in the common latent space representation. Information about a node's neighbors is collected through nonlinear transformations and aggregation functions and used to update the node's features. Therefore, the topological structure of the data can be preserved, while making the features of neighboring nodes more similar. Furthermore, the interaction between samples is enhanced through the graph convolutional network, thereby learning a structured latent space representation. Similar samples have more similar features during the graph convolution process, exhibiting better structure.

[0035] 6. In this embodiment, the clustering model is trained and corrected based on the training data results and the training loss value until the preset convergence value is met, at which point the training stops. The clustering results of multiple views to be clustered are then obtained by using the trained clustering model, which has high accuracy and good clustering effect.

[0036] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0037] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0038] Figure 1 This is a flowchart of a multi-view clustering method provided according to an embodiment of this application;

[0039] Figure 2 This is a model framework diagram of SLRL provided according to an embodiment of this application;

[0040] Figure 3 This is a schematic diagram illustrating a visual comparison of the BBCSport and MSRCV1 datasets according to embodiments of this application;

[0041] Figure 4 The graph shows the sensitivity test results for parameters k and γ according to the embodiments of this application.

[0042] Figure 5 This is a schematic diagram illustrating the changes in ACC and objective function values ​​with the number of iterations on MSRCV1 and 100Leaves, according to an embodiment of this application.

[0043] Figure 6 This is a schematic diagram illustrating the training process of the SLRL model provided according to an embodiment of this application;

[0044] Figure 7 This is a flowchart of another multi-view clustering method provided according to an embodiment of this application;

[0045] Figure 8 This is a block diagram of a multi-view clustering device according to an embodiment of this application;

[0046] Figure 9 This is a block diagram of another multi-view clustering device provided according to an embodiment of this application;

[0047] Figure 10 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0048] Explanation of reference numerals in the attached drawings: First acquisition module-100, extraction module-200, first clustering module-300, second acquisition module-400, processing module-500, training module-600, second clustering module-700, memory-1001, processor-1002, communication interface-1003. Detailed Implementation

[0049] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0050] The following describes a multi-view clustering method, apparatus, electronic device, and storage medium according to embodiments of this application with reference to the accompanying drawings. Addressing the problems mentioned in the background art, this application provides a multi-view clustering method. In this method, image features of multiple views to be clustered are extracted and input into a pre-trained clustering model. Complementary information between different views is flexibly mined through a common latent space representation, and structural information between views is obtained using the structured latent space, thereby achieving classification of each view category and further obtaining the clustering result of multiple views. Simultaneously considering the complementary information between views and the structural relationships between samples, it can not only flexibly fuse information from multiple views but also fully utilize the structural information between samples, thus making the clustering result more accurate. Therefore, it solves the problems of related technologies that construct graphs based on original data matrices or shallow features and use different fusion strategies to fuse multiple graphs, which not only involve high computational cost but also heavily rely on the quality of the original graph construction, thus limiting the effectiveness of multi-view clustering.

[0051] Specifically, Figure 1 This is a flowchart illustrating a multi-view clustering method provided in an embodiment of this application.

[0052] like Figure 1 As shown, this multi-view clustering method includes the following steps:

[0053] In step S101, multiple views to be clustered are obtained.

[0054] It is understandable that multi-view data contains richer information than a single view, and reasonably integrating this information from different perspectives can lead to a more comprehensive analysis of the problem. Therefore, the embodiments of this application first need to obtain multiple views to be clustered, and by reasonably fusing the information from different views, adaptively classify the data into their respective categories without any label information.

[0055] In step S102, at least one image feature of each view is extracted to obtain multiple image features.

[0056] This application embodiment can select commonly used multi-view datasets to compare all methods, including 100leaves, Scene-15, MSRCV1, 3Soures, and BBCSport. Table 1 shows the statistical information for each dataset. Table 1 is a statistical information table for commonly used datasets.

[0057] Table 1

[0058] Dataset Sample size Number of views category Feature Dimension 100Leaves 1600 3 100 64 / 64 / 64 Scene-15 4485 3 15 20 / 59 / 40 MSRCV1 210 6 7 1302 / 48 / 512 / 100 / 256 / 210 3sources 169 3 6 3560 / 3631 / 3068 BBCSport 544 2 5 3183 / 3203

[0059] Details for each dataset are as follows:

[0060] (1) 100Leaves1: This dataset consists of 1600 images of 100 plant species. In this embodiment, three features were extracted from each plant sample: shape description, fine-scale edge data, and texture histogram.

[0061] (2) Scene-152: This dataset includes 15 different indoor and outdoor scene images, consisting of a total of 4485 images. For each image, three different features were extracted: GIST, PHOG (Pyramid Histogram of Oriented Gradients), and LBP.

[0062] (3) MSRCV1: This dataset contains 7 categories of common objects in daily life, including trees, buildings, airplanes, cows, faces, cars, and bicycles, with 30 images for each category. Six visual features were extracted from each image: 1302-dimensional Scale-Invariant Feature Transform (SIFT) features, 48-dimensional Color Moment (CMT) features, 512-dimensional GIST features, 100-dimensional Local Binary Patterns (LBP) features, 256-dimensional Histogram of Oriented Gradients (HOG) features, and 210-dimensional CensusTransform Histogram (CENTRIST) features.

[0063] (4) 3Soures3: This dataset consists of 948 news articles, covering 416 different news items. In this embodiment, 169 news items reported simultaneously by three media outlets were selected, encompassing six themes: business, entertainment, health, politics, sports, and technology. For each news item, features of three different dimensions—3560, 3631, and 3068—were extracted.

[0064] (5) BBCSport4: This dataset consists of 544 sports news articles, covering 5 different types of topics. In this embodiment, features of 3183 and 3203 dimensions were extracted for each document.

[0065] In step S103, multiple image features are input into a pre-trained clustering model. The complementary information and structural information between different views are extracted using the preset common latent space and preset structured latent space of the clustering model, respectively. The complementary information and structural information are used to match the actual category of each view. Based on the actual category, multiple views are clustered, and the clustering results of multiple views are output.

[0066] To better extract information from multi-view data, multi-view clustering algorithms generally need to follow two principles: consistency and complementarity. This application proposes a multi-view clustering method based on Structured Latent Representation Learning for Multi-view Clustering (SLRL), whose SLRL model framework is as follows: Figure 2 As shown, this embodiment of the application can extract image features from multiple views to be clustered and input them into a pre-trained clustering model. First, it learns a common latent space representation to flexibly mine complementary information between different views. Second, in order to utilize the structural information between samples, a k-nearest neighbor graph is constructed based on this latent space representation, and the interaction between samples is further enhanced through a graph convolutional network, thereby learning a structured latent space representation. In addition, the learning of the latent space representation is further constrained by the clustering loss, so that the learned latent space representation has a good cluster structure, thereby achieving the classification of each view category and further obtaining the clustering results of multiple views.

[0067] In one embodiment of this application, before inputting multiple view samples into a preset common latent space and a preset structured latent space respectively, and outputting complementary information and structural information between view samples, the method includes: obtaining the latent space representation of any view in the preset samples; calculating the mapping relationship between the latent space representation and any view with the minimum reconstruction loss of the latent space representation as the optimization objective; and constructing a preset common latent space based on the mapping relationship and a preset multilayer perceptron.

[0068] Understandably, the key to multi-view clustering lies in how to rationally utilize the consistency and complementarity among views to obtain more comprehensive information. Therefore, a certain strategy is needed to fuse information from multiple views. Considering the complementarity of multiple views, a latent space multi-view subspace clustering method has been proposed. This method finds a common latent space representation for all views and reconstructs each view based on this representation. Due to the complementarity of multiple views, the latent space representation can describe the data itself more comprehensively than a single view, thus making the latent space representation more accurate and robust.

[0069] This application embodiment can provide a multi-view dataset X = {X} with V views. (1) X (2) , ..., X (V)}, Where N is the number of samples in the data, d v Let be the original feature dimension of view v. The goal of SLRL is to learn a structured latent space representation for all views, making it well-suited for multi-view clustering tasks. Some of the symbols involved and their meanings are shown in Table 2. Table 2 is a table of symbols used and their explanations.

[0070] Table 2

[0071]

[0072] Specifically, for the v-th view The latent space of its nth sample is defined as h. n , for Based on the parameterized reconstruction network of the v-th view, the optimization objective for learning the common latent space representation can be defined as:

[0073]

[0074] Among them, L r The reconstruction loss for the common latent space representation aims to learn the mapping between the original data space and the common latent space. In this embodiment, a multilayer perceptron can be used as the encoder to realize the common latent space representation and the original features x of the original data. n The mapping. By minimizing the loss corresponding to formula (3-1), a common latent space representation H = {h1, h2, ..., h} can be learned. N}

[0075] In one embodiment of this application, before inputting multiple view samples into a preset common latent space and a preset structured latent space respectively, and outputting complementary information and structural information between view samples, the method includes: constructing an undirected k-nearest neighbor graph using the preset common latent space; inputting multiple view samples into the undirected k-nearest neighbor graph to obtain the actual nearest neighbor relationships between multiple view samples; using the actual nearest neighbor relationships and the preset common latent space representation as input to a preset graph convolutional network, outputting an updated representation of the preset common latent space, and performing layer-by-layer convolution through the preset graph convolutional network to obtain an updated representation of each convolution layer, and constructing a preset structured latent space based on the updated representation of each convolution layer.

[0076] In this embodiment, the common latent space representation learned in the above embodiments can flexibly integrate complementary information from multiple views, but it lacks a good cluster structure, making it difficult to achieve ideal clustering results when directly used for clustering. Furthermore, considering only the mining of complementary information from multiple views ignores the structural information between samples, which is crucial for clustering tasks. Considering that samples of the same class have similar characteristics, for each sample, this embodiment can select its k nearest neighbors as its neighbors. Therefore, it proposes constructing an undirected k-nearest neighbor graph for the learned common latent space representation H to mine the structural information of samples in the latent space. This embodiment defines the near neighbor graph as A∈R N*N For any two samples i and j, their latent space is represented as h. i and h j The distance between the two can be calculated in the following way.

[0077] For continuous data, such as images, this embodiment of the application can use a Gaussian kernel to measure the similarity between sample i and sample j. The nearest neighbor relationship between samples i and j can be calculated by the following formula:

[0078]

[0079] For discrete data, such as text, embodiments of this application can use dot product similarity to calculate the similarity between two data points. The nearest neighbor relationship can then be calculated as follows:

[0080]

[0081] Where, N k (h i Let represent the k nearest neighbors of the i-th sample. Thus, in this embodiment, a nearest neighbor graph A can be obtained from non-graph data, which can well represent the nearest neighbor relationships between samples.

[0082] Since nearest neighbor graphs effectively measure the distance between different samples, this application's embodiments utilize Graph Nervous Networks (GCNs) to mine information between samples in the common latent space representation in order to make the learned latent space representation more structured. GCNs collect information about a node's neighbors by applying multiple convolutional layers, using nonlinear transformations and aggregation functions, and then use this information to update the node's features. Therefore, the topological structure of the data can be preserved while making the features of neighboring nodes more similar. Through the graph convolutional network, the features of samples of the same class become more similar, resulting in a structured latent space representation.

[0083] The graph convolutional layer takes the latent space representation H and the nearest neighbor graph A as input, and outputs the updated latent space representation. For the weight matrix W, the feature representation H learned by the l-th layer of the GCN is... (l) It can be obtained through the following convolution operation:

[0084]

[0085] in, The correlation coefficient matrix has self-connected components. I is the unit diagonal matrix corresponding to A, and W(l) is the learnable parameter matrix of the l-th layer. φ(·) is the activation function, and ReLU can be used as the activation function in this embodiment. Therefore, in the l-th layer network, the latent space representation H(l-1) is normalized by the adjacency matrix. Propagation yields a new representation H(l). For a graph convolutional network with L layers, a structured latent space representation can ultimately be learned through layer-by-layer convolution. for For each sample in the graph, similar samples have more similar features during graph convolution, resulting in better structure.

[0086] In one embodiment of this application, the clustering model is trained based on training data carrying labels of real clustering results, including: acquiring training data carrying labels of real clustering results, wherein the training data includes multiple view samples; inputting the multiple view samples into a preset common latent space and a preset structured latent space respectively, and outputting complementary information and structural information between view samples; calculating the actual similarity between the sample points of each view sample and the preset cluster centers based on the complementary information and structural information, determining the actual category of each view sample based on the actual similarity, clustering multiple view samples based on the actual categories to obtain training clustering results; calculating the training loss value based on the training clustering results and the real clustering results; when the training loss value is greater than the convergence threshold, optimizing the cluster centers from the current confidence allocation using the target distribution determined by the actual similarity, and updating the preset structured latent space, until the training loss value is less than or equal to the convergence threshold, stopping iterative training, and obtaining the trained clustering model.

[0087] It is understandable that graph convolutional networks can yield more structured latent space representations. The learned public representations can preserve the inherent local structure of the data, but they still cannot be directly applied to clustering problems. Most existing works use traditional clustering methods (such as spectral clustering or k-means) on the learned features, and the clustering effect depends entirely on the learned features. Related techniques propose an end-to-end clustering method that can simultaneously optimize latent space representation learning and clustering, thus making the learned structured latent space representation more suitable for clustering tasks.

[0088] In this embodiment, a two-step alternating unsupervised algorithm can be used to improve clustering. In the first step, a soft assignment between each sample point and the cluster centroid in the latent space representation is computed. In the second step, the structured latent space representation is updated, and the cluster centroids are refined by learning from the current high-confidence assignments using an auxiliary target distribution. This process is repeated until convergence is achieved. Specifically, for the i-th sample and the j-th cluster, this embodiment can use a t-distribution as the kernel to measure the data representation. and cluster center vector u j The similarity between them can be calculated using the following formula:

[0089]

[0090] Among them, the cluster center ui is obtained by K-means. The result is obtained from the initialization. Q = [q] i [j] represents the probability of assigning sample i to cluster j, i.e., soft assignment. After obtaining the cluster distribution Q, the goal of this application embodiment is to optimize the latent space representation by learning high-confidence soft assignments. Specifically, this application embodiment can make the feature representation of each sample closer to the cluster center, thereby improving cluster cohesion. Therefore, p in the target distribution P ij It can be calculated in the following ways:

[0091]

[0092] In the target distribution P, each assignment in Q is normalized using a sum of squares to increase its confidence. Therefore, the clustering loss function can be defined as follows:

[0093]

[0094] By minimizing the KL divergence loss between the Q and P distributions, the target distribution P can facilitate the network in learning a more structured latent space representation suitable for clustering tasks. This means the data representations around the cluster centers are more similar, resulting in better clustering performance. The clustering results for multiple views to be clustered are obtained using the output of the trained clustering model, demonstrating high accuracy and good clustering performance.

[0095] In one embodiment of this application, calculating the training loss value based on the training clustering results and the actual clustering results includes: calculating the reconstruction loss and the clustering loss based on the training clustering results and the actual clustering results; obtaining the weight parameters of the clustering loss; and calculating the training loss value based on the weight parameters, the clustering loss, and the reconstruction loss.

[0096] Specifically, the loss function in this application embodiment can mainly include reconstruction loss and clustering loss. The reconstruction loss is used to constrain the error between the reconstructed features of the common latent space representation H and the original features of the view, thereby flexibly balancing consistency and complementarity. Then, by mining the structural characteristics between samples, a structured latent space representation is obtained. Clustering loss is used for distributed latent space representation Simultaneously constraining the learning of the common latent space representation and the graph representation, the learned latent space representation thus possesses a better cluster structure. Therefore, the overall loss function can be defined as:

[0097] L = L r +γL c (3-8)

[0098] Among them, L r and L c These are the reconstruction loss and clustering loss, respectively. γ>0 is a hyperparameter used to balance clustering optimization and learning of the common latent space representation.

[0099] In the field of multi-view clustering, for ease of explanation, the embodiments of this application may select the following nine methods to compare with the multi-view clustering method of the embodiments of this application.

[0100] a. Comparison Method

[0101] (1) SPCBestSV: This method performs spectral clustering on each view and selects the best set of results.

[0102] (2) LRRBestSV: This method proposes to find the lowest-rank representation among all data samples, thereby clustering the samples into their respective subspaces. LRR is used to cluster on each view separately, and the best result is selected.

[0103] (3) DiMSC: This method uses the Hilbert Schmidt Independence Criterion (HSIC) as a diversity constraint to enhance the complementarity between different views, and then uses spectral clustering to obtain the clustering results.

[0104] (4) AMGL: This method proposes a novel parameterless automatic weighted multi-graph learning model. This model can automatically learn the association graph and weights of different views. In addition, the method can converge to a local optimum and can be used for both multi-view clustering and semi-supervised classification.

[0105] (5) MLAN: This method proposes a novel multi-view learning model that can simultaneously perform clustering and local structure learning. The resulting optimal graph can be directly classified into specific classes. Furthermore, the model can automatically assign weights to each view without requiring additional weight and penalty parameters.

[0106] (6) LMSC: This method finds a common latent space representation for multiple views and reconstructs each view based on the learned latent space representation. Since the latent space representation can describe the data more comprehensively than a single view, and can also mine complementary information from the latent space of multiple views, the subspace representation is more effective and robust.

[0107] (7) MVGL: This method proposes a multi-view clustering approach based on graph learning. First, initial graphs are learned for different views, and then the initial graphs are further optimized by applying rank constraints on the Laplacian matrix. Finally, the optimized graphs are integrated into the global graph.

[0108] (8) GMC: This method proposes a novel multi-view fusion technique that can automatically weight each data graph matrix to obtain a unified graph matrix. The unified graph matrix then updates the data graph matrix of each view, enabling the learning of each view's graph matrix and the learning of the unified graph matrix to mutually reinforce each other.

[0109] (9) CGD: This method is the first to attempt to use a diffusion process for multi-view clustering. Using the graph matrices of different views as input, it captures the underlying manifold geometry of the original data points through an iterative cross-diffusion process. It utilizes complementary information between multiple graphs to learn an improved graph for each view. The improved view association graphs are then averaged to obtain a unified graph for clustering.

[0110] b. Experiment setup

[0111] For each experiment, 80% of the dataset was used as the training set and 20% as the test set. For fairness, all comparison methods were adjusted according to the source code and parameters provided by the authors of the paper to obtain the best clustering results. For the model in this embodiment, the ReLU function can be used as the non-linear activation function in the network. The latent space dimension F of each dataset ranges from {16, 32, 64, 128, 256}, with a default dimension of 64. The number of nearest neighbors k ranges from k∈{3, 15}, with a default setting of 10. Since the model is not sensitive to changes in the γ parameter, γ is uniformly set to 10 in the experiments of this embodiment. Batch gradient descent can be used to optimize the model's loss function, with a learning rate of 0.01. To avoid random errors, this embodiment can repeat the experiment 10 times and calculate the mean and standard deviation of the four clustering performance metrics for comparison.

[0112] c. Experiment and Results Analysis

[0113] The following table shows the mean and standard deviation of all methods for different metrics on the five datasets: 100Leaves, Scene-15, MSRCV1, BBCSport, and 3Sources. For each clustering metric, the best results are marked in bold. Tables 3-7 are shown below, with Table 3 providing a comparison of clustering performance on the 100Leaves dataset.

[0114] Table 3

[0115] method ACC NMI F_score ARI SPCBestSV 48.03±1.11 76.61±2.18 40.76±1.84 40.19±1.51 LRRBestSV 47.86±2.49 70.62±1.37 38.42±2.67 35.08±0.91 DiMSC 85.58±3.13 72.34±2.31 57.97±3.25 60.89±2.96 AMGL 76.76±1.93 87.57±2.86 57.66±1.30 56.12±1.94. MLAN 84.02±0.72 94.11±1.30 80.21±1.07 79.56±2.36 LMSC 74.37±2.06 86.41±2.73 64.31±2.84 63.30±2.95 MVGL 76.59±1.26 85.87±0.78 51 32±2.74 51.37±0.70 GMC 83.62±0.83 90.09±2.10. 80.94±2.65 73.79+0.68 CGD 86.34±1.21 94.92±1.93. 83.74±0.93 81.86±1.02 SLRL 93.75±0.94 97.03±1.33 88.89±1.72 88.67±1.01

[0116] Table 4 shows the clustering performance comparison on the Scene-15 dataset.

[0117] Table 4

[0118] method ACC NMI F_score ARI SPCBestSV 37.42±1.34 37.26±0.91 28.31±1.70 19.42±1.28 LRRBestSV 36.29±0.92 36.13±1.47 25.31±0.90 18.95±0.91 DiMSC 38.22±1.13 39.00±2.40 31.87±1.25 22.91±1.17 AMGL 33.92±2.33 37.35±1.33 25.02±0.83 17.39±1.10 MLAN 15.78±1.08 1 6.59±0.03 15.21±1.08 13.94±1.37 LMSC 42.68±1.40 34.98±2.21 27.77±1.27 23.26±0.93 MVGL 37.48±2.06 37.29±2.17 27.82±1.10 22.14±2.06 GMC 41.55±1.30 37.09±1.81 30.04±0.92 22.18±1.98 CGD 43.66±1.99 42.53±1.53 31.39±2.80 24.94±1.53 SLRL 47.52±1.87 53.25±1.02 42.31±0.97 31.88±1.24

[0119] Table 5 shows the clustering performance comparison on the MSRCV1 dataset.

[0120] Table 5

[0121] method ACC NMI F_score ARI SPCBestSV 51.45±2.41 50.07±0.85 48.68±1.64 47.70±1.89 LRRBestSV 66.75±3.06 59.68±1.55 60.44±3.03 58.24±1.17 DiMSC 74.33±2.61 66.75±2.73 73.79±0.68 72.95±1.13 AMGL 74.40±2.91 72.97±1.69 66.50±2.42 62.62±1.84 MLAN 71.41±1.82 74.40±0.73 66.73±2.66 60.88±1.38 LMSC 83.62±1.33 70.52±1.27 69.96±2.58 66.13±1.35 MVGL 90.72±2.46 82.51±0.47 81.21±1.50 76.64±1.22 GMC 90.55±2.29 82.76±2.03 78.48±1.94 76.95±2.27 CGD 91.29±1.09 83.63±2.51 80.98±0.82 77.79±0.81 SLRL 92.70±2.12 89.96±1.52 84.35±3.09 82.26±1.84

[0122] Table 6 is a comparison table of clustering performance on the BBCSport dataset.

[0123] Table 6

[0124] method ACC NMI F_score ARI SPCBestSV 35.05±2.47 30.71±1.09 39.84±2.61 31 92±1.65 LRRBestSV 89.68±1.91 77.34±2.90 80.18±1.39 73.75±2.26 DiMSC 84 98±1.11 89.08±1.51 91.27±1.20 91.63±1.75 AMGL 57.83±2.49 52.14±2.50 56.81±1.80 52.49±0.46 MLAN 97.23±3.13 90.68±1.69 94.77±2.94 92.07±3.08 LMSC 92.79±0.72 83.77±1.78 90.21±1.04 86.07±2.05 MVGL 80.89±1.59 79.90±2.38 79.55±1.11 74.81±2.35 GMC 85.56±2.30 81.01±0.97 80.85±1.26 81.15±0.37 CGD 97.37±1.03 91.09±2.76 93.97±1.75 93.73±1.20 SLRL 93.02±2.52 86.44±0.93 91.20±1.73 90.75±3.10

[0125] Table 7 shows the clustering performance comparison on the 3Sources dataset.

[0126] Table 7

[0127] method ACC NMI F_score ARI SPCBestSV 64.08±2.44 56.25±1.36 60.26±0.87 53.15±0.70 LRRBestSV 63.64±1.98 53.73±1.25 54.37±2.41 46.07±0.75 DiMSC 73.75±2.22 68.84±0.82 70.23±1.25 60.43±1.51 AMGL 63.88±1.08 60.89±1.91 62.75±1.27 54.91±1.03 MLAN 68.72±2.41 54.38±1.21 55.73±1.33 35.55±1.77 LMSC 72.22±2.08 68.40±0.91 70.61±1.15 57.76±0.55 MVGL 70.54±1.66 65.69±2.31 60.63±1.59 43.10±1.08 GMC 69.56±1.99 61.84±2.33 61.84±2.33 44.38±2.52 CGD 76.65±1.53 70.79±0.58 70.18±3.23 60.64±1.20 SLRL 81.62±1.62 72.33±2.05 77.90±1.32 69.86±1.27

[0128] From these experimental results, the following conclusions can be drawn:

[0129] (1) The proposed method achieved excellent clustering results on five commonly used multi-view datasets. Compared with the comparative methods, the proposed method achieved the best results on the 100Leaves, Scene-15, MSRCV1, and 3Sources datasets. Compared with the second-best method, the proposed method's SLRL improved the ACC by 7.41%, 3.86%, 1.41%, and 4.97%, respectively, and the NMI by 2.11%, 10.72%, 6.33%, and 1.54%, respectively. Although MLAN and CGD performed better on the BBCSport dataset, they were inferior to the proposed method's multi-view clustering method on other datasets. Overall, the proposed method's multi-view clustering device achieved ideal clustering performance on different datasets, demonstrating its superiority.

[0130] (2) On some datasets, the best results of single-view clustering are better than some multi-view clustering algorithms, indicating that it is not easy to make full use of the information of multiple views, and it is necessary to more reasonably explore the consistency and complementarity between views.

[0131] To more intuitively demonstrate the effectiveness of the multi-view clustering method in this application's embodiments, visualization analysis can be performed on the BBCSport and MSRCV1 datasets. Here, BSV represents visualizing all views and selecting the best-performing one. SLRL-H represents the visualization result of the common latent space representation H. For structured latent space representation The visualization effect. For example... Figure 3 As shown, on the BBCSport and MSRCV1 datasets, the latent space representation learned in this application has a better cluster structure than the original input features, indicating that fusing information from multiple views is beneficial to the clustering effect. Furthermore, on both datasets, compared to SLRL-H, Samples of the same class are more closely clustered, while samples of different classes are more dispersed, resulting in a better cluster structure. This demonstrates that learning through graph representation and clustering constraints enables the model to learn features more suitable for clustering.

[0132] Secondly, the clustering model of the present application embodiment can be tested through parameter sensitivity experiments and convergence experiments.

[0133] a. Parameter sensitivity experiment

[0134] (1) k-value sensitivity experiment

[0135] The nearest neighbor number k is a crucial parameter in constructing a k-nearest neighbor graph, significantly impacting the performance of most graph-based algorithms. To examine the model's sensitivity to k, this application's embodiments designed a k-sensitivity experiment on the MSRCV1 dataset. Figure 4 As shown, for k∈{3, 15}, the model in this embodiment is insensitive to the value of k within a certain range. This demonstrates that the multi-view clustering method in this embodiment can learn structural information between samples even when there is limited neighborhood information or spurious connections exist.

[0136] (2) Gamma sensitivity experiment

[0137] In the objective function, γ is a parameter that balances common representation learning and clustering constraints. Taking the 100Leaves dataset as an example... Figure 4 As shown, the value of γ affects the clustering indices ACC and NMI, where γ∈{10} -5 10 -4 10 -3 10 -2 10 -1 10 0 10 1 10 2 10 3}

[0138] To reduce the influence of random results, the experiment was repeated 10 times for each value of γ, and the curves were plotted using the average values ​​of ACC and NMI. It can be observed that ACC and NMI maintain relatively stable values ​​with changes in γ, indicating that the multi-view clustering method of this application is insensitive to γ ​​over a large range.

[0139] b. Convergence test

[0140] The embodiments of this application have undergone convergence experiments on the MSRCV1 and 100Leaves datasets. Figure 4 In the diagram, the x-axis represents the number of algorithm iterations, and the curves show the changes in ACC and target loss with each iteration. Experiments demonstrate that, as... Figure 5 As shown, with the increase of the number of iterations, the objective function becomes non-increasing and gradually converges to a certain fixed value, while the ACC gradually increases and tends to stabilize. Furthermore, the objective function initially decreases rapidly, but after about 30 iterations, the changes in both the objective function and ACC tend to stabilize, demonstrating that the embodiments of this application have good convergence.

[0141] The multi-view clustering method proposed in this application extracts image features from multiple views to be clustered, inputs them into a pre-trained clustering model, flexibly mines complementary information between different views through a common latent space representation, and utilizes the structured latent space to obtain structural information between views, thereby achieving classification of each view category and further obtaining the clustering results of multiple views. Simultaneously considering the complementary information between views and the structural relationships between samples, it can not only flexibly fuse information from multiple views but also fully utilize the structural information between samples, thus making the clustering results more accurate. Therefore, it solves the problems of related technologies that construct graphs based on original data matrices or shallow features and use different fusion strategies to fuse multiple graphs, which not only involve high computational cost but also heavily rely on the quality of the original graph construction, thus limiting the effectiveness of multi-view clustering.

[0142] Based on the multi-view clustering method described in the above embodiments, which mainly illustrate the online application of multi-view clustering, this application also proposes another multi-view clustering method to illustrate how to perform offline training. The training process is as follows: Figure 6 As shown. The embodiments of this application and the above embodiments have different focuses in their descriptions, and steps not fully described in each embodiment can be referred to each other.

[0143] Specifically, such as Figure 7 As shown, this multi-view clustering method includes the following steps:

[0144] In step S701, training data carrying labels of real clustering results is obtained, wherein the training data includes multiple view samples.

[0145] In step S702, multiple view samples are input into a preset common latent space and a preset structured latent space respectively, and complementary information and structural information between view samples are output.

[0146] In step S703, the actual similarity between the sample points of each view sample and the preset cluster center is calculated based on complementary information and structural information. The actual category of each view sample is determined based on the actual similarity. Multiple view samples are clustered based on the actual category to obtain the training clustering result.

[0147] In step S704, the training loss value is calculated based on the training clustering results and the actual clustering results. When the training loss value is greater than the convergence threshold, the cluster centers are optimized from the current confidence allocation using the target distribution determined by the actual similarity, and the preset structured latent space is updated. The iterative training stops when the training loss value is less than or equal to the convergence threshold, and the trained clustering model is obtained. The clustering model is then used to cluster multiple views to be clustered, and the clustering results of multiple views are obtained.

[0148] Next, referring to the accompanying drawings, a multi-view clustering device according to an embodiment of this application is described.

[0149] Figure 8 This is a block diagram of a multi-view clustering device according to an embodiment of this application.

[0150] like Figure 8 As shown, the multi-view clustering device 10 includes: a first acquisition module 100, an extraction module 200, and a first clustering module 300.

[0151] The first acquisition module 100 is used to acquire multiple views to be clustered; the extraction module 200 is used to extract at least one image feature from each view to obtain multiple image features; the first clustering module 300 is used to input the multiple image features into a pre-trained clustering model, extract complementary information and structural information between different views using the preset common latent space and preset structured latent space of the clustering model, match the actual category of each view using the complementary information and structural information, cluster the multiple views based on the actual category, and output the clustering results of the multiple views.

[0152] In one embodiment of this application, the first clustering module 300 is further configured to acquire training data carrying labels of real clustering results, wherein the training data includes multiple view samples; input the multiple view samples into a preset common latent space and a preset structured latent space respectively, and output complementary information and structural information between view samples; calculate the actual similarity between the sample points of each view sample and the preset cluster centers based on the complementary information and structural information, determine the actual category of each view sample based on the actual similarity, cluster the multiple view samples based on the actual categories to obtain training clustering results; calculate the training loss value based on the training clustering results and the real clustering results; when the training loss value is greater than the convergence threshold, optimize the cluster centers from the current confidence allocation using the target distribution determined by the actual similarity, and update the preset structured latent space, until the training loss value is less than or equal to the convergence threshold, stop iterative training, and obtain the trained clustering model.

[0153] In one embodiment of this application, the first clustering module 300 is further configured to calculate the reconstruction loss and clustering loss based on the training clustering results and the actual clustering results; obtain the weight parameters of the clustering loss; and calculate the training loss value based on the weight parameters, the clustering loss and the reconstruction loss.

[0154] In one embodiment of this application, the apparatus 10 further includes: a computing unit, configured to, before inputting multiple view samples into a preset common latent space and a preset structured latent space respectively, and outputting complementary information and structural information between the view samples, obtain the latent space representation of any view in the preset samples; calculate the mapping relationship between the latent space representation and any view with the minimum reconstruction loss of the latent space representation as the optimization objective; and a first construction unit, configured to construct the preset common latent space based on the mapping relationship and a preset multilayer perceptron.

[0155] In one embodiment of this application, the apparatus 10 further includes: a generation unit, configured to construct an undirected k-nearest neighbor graph using the preset common latent space before inputting multiple view samples into a preset common latent space and a preset structured latent space respectively, and outputting complementary information and structural information between the view samples; inputting multiple view samples into the undirected k-nearest neighbor graph to obtain actual nearest neighbor relationships between the multiple view samples; and a second construction unit, configured to use the actual nearest neighbor relationships and the preset common latent space representation as input to a preset graph convolutional network, output an updated representation of the preset common latent space, and perform layer-by-layer convolution through the preset graph convolutional network to obtain an updated representation of each convolution layer, and construct a preset structured latent space based on the updated representation of each convolution layer.

[0156] The multi-view clustering device proposed in this application extracts image features from multiple views to be clustered, inputs them into a pre-trained clustering model, flexibly mines complementary information between different views through a common latent space representation, and utilizes the structured latent space to obtain structural information between views, thereby achieving classification of each view category and further obtaining the clustering result of multiple views. Simultaneously considering the complementary information between views and the structural relationships between samples, it can not only flexibly fuse information from multiple views but also fully utilize the structural information between samples, thus making the clustering result more accurate. Therefore, it solves the problems of related technologies that construct graphs based on original data matrices or shallow features and use different fusion strategies to fuse multiple graphs, which not only involve high computational cost but also heavily rely on the quality of the original graph, thus limiting the effectiveness of multi-view clustering.

[0157] Figure 9 This is a block diagram of another multi-view clustering device according to an embodiment of this application.

[0158] like Figure 9 As shown, the multi-view clustering device 20 includes: a second acquisition module 400, a processing module 500, a training module 600, and a second clustering module 700.

[0159] The second acquisition module 400 is used to acquire training data carrying labels of real clustering results, wherein the training data includes multiple view samples; the processing module 500 is used to input multiple view samples into a preset common latent space and a preset structured latent space respectively, and output complementary information and structural information between view samples; the training module 600 is used to calculate the actual similarity between the sample points of each view sample and the preset cluster centers based on the complementary information and structural information, determine the actual category of each view sample based on the actual similarity, and cluster multiple view samples based on the actual categories to obtain training clustering results; the second clustering module 700 is used to calculate the training loss value based on the training clustering results and the real clustering results, and when the training loss value is greater than the convergence threshold, optimize the cluster centers from the current confidence allocation using the target distribution determined by the actual similarity, and update the preset structured latent space, until the training loss value is less than or equal to the convergence threshold, stop iterative training, obtain the trained clustering model, and use the clustering model to cluster multiple views to be clustered to obtain the clustering results of multiple views.

[0160] It should be noted that the foregoing explanation of the multi-view clustering method embodiment also applies to the multi-view clustering device of this embodiment, and will not be repeated here.

[0161] The multi-view clustering device proposed in this application extracts image features from multiple views to be clustered, inputs them into a pre-trained clustering model, flexibly mines complementary information between different views through a common latent space representation, and utilizes the structured latent space to obtain structural information between views, thereby achieving classification of each view category and further obtaining the clustering result of multiple views. Simultaneously considering the complementary information between views and the structural relationships between samples, it can not only flexibly fuse information from multiple views but also fully utilize the structural information between samples, thus making the clustering result more accurate. Therefore, it solves the problems of related technologies that construct graphs based on original data matrices or shallow features and use different fusion strategies to fuse multiple graphs, which not only involve high computational cost but also heavily rely on the quality of the original graph, thus limiting the effectiveness of multi-view clustering.

[0162] Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0163] The memory 1001, the processor 1002, and the computer program stored on the memory 1001 and capable of running on the processor 1002.

[0164] When the processor 1002 executes the program, it implements the multi-view clustering method provided in the above embodiments.

[0165] Furthermore, electronic devices also include:

[0166] Communication interface 1003 is used for communication between memory 1001 and processor 1002.

[0167] The memory 1001 is used to store computer programs that can run on the processor 1002.

[0168] The memory 1001 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0169] If the memory 1001, processor 1002, and communication interface 1003 are implemented independently, then the communication interface 1003, memory 1001, and processor 1002 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0170] Optionally, in a specific implementation, if the memory 1001, processor 1002, and communication interface 1003 are integrated on a single chip, then the memory 1001, processor 1002, and communication interface 1003 can communicate with each other through an internal interface.

[0171] The processor 1002 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0172] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the multi-view clustering method described above.

[0173] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0174] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0175] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0176] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0177] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0178] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A multi-view clustering method, characterized in that, Includes the following steps: Get multiple views of the data to be clustered; Extract at least one image feature from each view to obtain multiple image features; The multiple image features are input into a pre-trained clustering model. The complementary information and structural information between different views are extracted using the preset common latent space and preset structured latent space of the clustering model, respectively. The complementary information and the structural information are used to match the actual category of each view. The multiple views are clustered based on the actual category, and the clustering results of the multiple views are output. Before inputting multiple view samples into the preset common latent space and the preset structured latent space respectively, and outputting the complementary information and structural information between the view samples, the process includes: Construct an undirected k-nearest neighbor graph using the preset public hidden space; Input the multiple view samples into the undirected k-nearest neighbor graph to obtain the actual nearest neighbor relationships between the multiple view samples; The actual nearest neighbor relationship and the preset common latent space representation are used as inputs to a preset graph convolutional network. The updated representation of the preset common latent space is output, and the network is convolved layer by layer to obtain the updated representation of each convolutional layer. The preset structured latent space is constructed based on the updated representation of each convolutional layer.

2. The method according to claim 1, characterized in that, The clustering model is trained based on training data carrying labels of the actual clustering results, including: Obtain training data carrying labels of real clustering results, wherein the training data includes multiple view samples; The multiple view samples are respectively input into the preset common latent space and the preset structured latent space, and the complementary information and structural information between the view samples are output. The actual similarity between the sample points of each view sample and the preset cluster center is calculated based on the complementary information and the structural information. The actual category of each view sample is determined based on the actual similarity. The multiple view samples are clustered based on the actual category to obtain the training clustering result. The training loss value is calculated based on the training clustering results and the actual clustering results. When the training loss value is greater than the convergence threshold, the cluster centers are optimized from the current confidence assignment using the target distribution determined by the actual similarity, and the preset structured latent space is updated. The iterative training stops when the training loss value is less than or equal to the convergence threshold, and the trained clustering model is obtained.

3. The method according to claim 2, characterized in that, The step of calculating the training loss value based on the training clustering results and the true clustering results includes: Calculate the reconstruction loss and clustering loss based on the training clustering results and the actual clustering results; Obtain the weight parameters of the clustering loss, and calculate the training loss value based on the weight parameters, the clustering loss, and the reconstruction loss.

4. The method according to claim 2, characterized in that, Before inputting the multiple view samples into the preset common latent space and the preset structured latent space respectively, and outputting the complementary information and structural information between the view samples, the process includes: Obtain the latent space representation of any view in a preset sample; With the goal of minimizing the reconstruction loss of the latent space representation, the mapping relationship between the latent space representation and the arbitrary view is calculated. The preset common hidden space is obtained based on the mapping relationship and the preset multilayer perceptron.

5. A multi-view clustering method, characterized in that, Includes the following steps: Obtain training data carrying labels of real clustering results, wherein the training data includes multiple view samples; The multiple view samples are respectively input into a preset common latent space and a preset structured latent space, and the complementary information and structural information between the view samples are output. The actual similarity between the sample points of each view sample and the preset cluster center is calculated based on the complementary information and the structural information. The actual category of each view sample is determined based on the actual similarity. The multiple view samples are clustered based on the actual category to obtain the training clustering result. The training loss value is calculated based on the training clustering results and the actual clustering results. When the training loss value is greater than the convergence threshold, the cluster centers are optimized from the current confidence assignment using the target distribution determined by the actual similarity, and the preset structured latent space is updated. The iterative training stops when the training loss value is less than or equal to the convergence threshold, and the trained clustering model is obtained. The clustering model is then used to cluster multiple views to be clustered to obtain the clustering results of the multiple views. Before inputting the multiple view samples into the preset common latent space and the preset structured latent space respectively, and outputting the complementary information and structural information between the view samples, the process includes: Construct an undirected k-nearest neighbor graph using the preset public hidden space; Input the multiple view samples into the undirected k-nearest neighbor graph to obtain the actual nearest neighbor relationships between the multiple view samples; The actual nearest neighbor relationship and the preset common latent space representation are used as inputs to a preset graph convolutional network. The updated representation of the preset common latent space is output, and the network is convolved layer by layer to obtain the updated representation of each convolutional layer. The preset structured latent space is constructed based on the updated representation of each convolutional layer.

6. A multi-view clustering device, characterized in that, Implementing the multi-view clustering method as described in any one of claims 1-4, comprising: The first acquisition module is used to acquire multiple views to be clustered; The extraction module is used to extract at least one image feature from each view, resulting in multiple image features; The first clustering module is used to input the multiple image features into a pre-trained clustering model, extract complementary information and structural information between different views using the preset common latent space and preset structured latent space of the clustering model, match the actual category of each view using the complementary information and the structural information, cluster the multiple views based on the actual category, and output the clustering result of the multiple views.

7. A multi-view clustering device, characterized in that, Implementing the multi-view clustering method as described in claim 5 includes: The second acquisition module is used to acquire training data carrying labels of real clustering results, wherein the training data includes multiple view samples; The processing module is used to input the multiple view samples into a preset common latent space and a preset structured latent space respectively, and output complementary information and structural information between the view samples; The training module is used to calculate the actual similarity between the sample points of each view sample and the preset cluster center based on the complementary information and the structural information, determine the actual category of each view sample based on the actual similarity, and cluster the multiple view samples based on the actual category to obtain the training clustering result; The second clustering module is used to calculate the training loss value based on the training clustering results and the real clustering results. When the training loss value is greater than the convergence threshold, the cluster centers are optimized from the current confidence assignment using the target distribution determined by the actual similarity, and the preset structured latent space is updated. Iterative training is stopped when the training loss value is less than or equal to the convergence threshold, and the trained clustering model is obtained. The clustering model is then used to cluster multiple views to be clustered to obtain the clustering results of the multiple views.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the multi-view clustering method as described in any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the multi-view clustering method as described in any one of claims 1-5.

Images