Explainable multi-clustering method and system based on weakly supervised learning
By generating parallel nonlinear feature subspaces through deep neural networks and combining them with weakly supervised prior knowledge, the feature subspaces are optimized, solving the interpretability and quality issues of multi-clustering algorithms in big data mining and achieving high-quality multi-clustering pattern discovery.
Patent Information
- Application Number
- CN202411812139.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-10
- Publication Date
- 2025-12-12
- Estimated Expiration
- 2044-12-10
AI Technical Summary
Existing multi-clustering algorithms struggle to generate interpretable, high-quality multi-clustering patterns in big data mining scenarios under the guidance of weakly supervised prior knowledge, and their subspace saliency and interpretability are insufficient.
Multiple parallel nonlinear feature subspaces are generated using deep neural networks. Combined with weakly supervised prior knowledge and redundant control constraints, the feature subspaces are optimized through reconstruction loss, clustering constraint loss, and difference constraint loss. K-means clustering is then applied to generate multi-cluster results.
It enhances the saliency and interpretability of the feature subspace, improves the quality and diversity of multi-clustering, and enhances the interpretability of multi-clustering.
Smart Images
Figure CN119740060B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data mining classification and interpretable artificial intelligence technology, and particularly relates to an interpretable multi-clustering method and system based on weakly supervised learning. Background Technology
[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.
[0003] With the surge in big data, the diversity and complexity of data are also constantly increasing. Traditional clustering methods can only provide a single clustering result, limiting data exploration to a single possible partition. In contrast, multi-clustering can simultaneously or sequentially reveal multiple non-redundant, meaningful clustering schemes, uncovering multiple interesting hidden structures within the data from different perspectives. However, how to explore salient clustering spaces and generate interpretable multi-clustering under the guidance of weakly supervised prior knowledge remains a critical challenge for current multi-clustering theory and technology in big data mining applications.
[0004] Current multi-clustering algorithms can be categorized into two types based on whether they reference prior information: unsupervised multi-clustering algorithms and semi-supervised multi-clustering algorithms. Unsupervised multi-clustering algorithms do not rely on any external prior information during optimization; they only achieve multi-clustering by adaptively learning the inherent attributes and features of the data. Conversely, semi-supervised multi-clustering algorithms introduce known clusters as reference information during model training, generating alternative clusters with low overlap with known clusters. Furthermore, based on the different clustering feature spaces, multi-clustering algorithms can also be divided into algorithms based on the original feature space and algorithms based on feature subspaces. Algorithms based on the original feature space explore multiple non-redundant clusters in the high-dimensional feature space of the data, while algorithms based on subspaces project the data into different subspaces and generate multiple clusters by constraining the redundancy of the subspaces.
[0005] In recent years, research on multi-clustering theory and techniques has developed rapidly, and related results have been widely applied in fields such as image data analysis and text data mining. However, since relying solely on known clusters cannot generate reliable subspaces, and subspaces and multi-clustering suffer from problems such as low significance and poor interpretability, there is still considerable room for improvement in multi-clustering research in exploring interpretable, high-quality, and diverse clustering patterns based on known weakly supervised prior knowledge. Summary of the Invention
[0006] To address at least one of the technical problems mentioned above, this invention provides an interpretable multi-clustering method and system based on weakly supervised learning. This method utilizes the strong representational capabilities of deep neural networks to generate multiple parallel nonlinear subspaces, combines weakly supervised prior knowledge with redundant control constraints to enhance the saliency and interpretability of the subspaces, and applies it to the discovery of high-quality, interpretable multi-clustering patterns in large-scale data.
[0007] To achieve the above objectives, the present invention adopts the following technical solution:
[0008] A first aspect of the present invention provides an interpretable multi-clustering method based on weakly supervised learning, comprising the following steps:
[0009] Obtain the original dataset, perform numerical vector representation on the data, and obtain the data representation matrix;
[0010] The data representation matrix is projected onto different nonlinear feature subspaces to obtain the features of the corresponding subspaces, and the reconstruction loss is calculated.
[0011] A sample feature annotation set is constructed based on the original dataset, a clustering constraint matrix is constructed based on the sample feature annotation set, and the clustering constraint loss of the nonlinear feature subspace is calculated based on the clustering constraint matrix.
[0012] Calculate the difference constraint loss of the nonlinear eigenspace;
[0013] An objective function is constructed based on reconstruction loss, clustering constraint loss, and difference constraint loss to determine the optimized feature subspace set.
[0014] The optimized feature subspace set is subjected to K-means clustering to obtain multi-clustering results that conform to weakly supervised prior knowledge.
[0015] Furthermore, multiple parallel deep neural networks are used to project the original high-dimensional data onto different nonlinear feature subspaces to obtain the data features of the corresponding subspaces.
[0016] Furthermore, the formula for calculating the clustering constraint matrix is as follows:
[0017]
[0018] in, Let be the similarity between the i-th sample and the j-th sample in the h-th subspace. This represents the class label of the i-th sample in the h-th subspace. Let the class label of the j-th sample in the h-th subspace be denoted as . When two samples are in the h-th subspace, they should be assigned to the same cluster; otherwise, they belong to different classes.
[0019] Furthermore, the formula for calculating the clustering constraint loss of the nonlinear feature subspace is as follows:
[0020]
[0021] in, This represents the sum of clustering constraint losses for the H feature subspaces. Let i be the i-th feature in the h-th feature subspace matrix. It is the j-th feature in the h-th feature subspace matrix;
[0022] Furthermore, the formula for calculating the difference constraint loss of the nonlinear eigenspace is as follows:
[0023]
[0024] in, Z represents the sum of the difference constraint losses of H feature subspaces. h and Z ′ h Let h and h represent two parallel subspaces, where h ≠ h. ′ , where I represents an orthogonal identity matrix.
[0025] Furthermore, the objective function is:
[0026]
[0027] Where j(Θ) represents the total loss of the model; F θ (·) represents the deep decoding network used for data reconstruction, and θ represents the parameters of the decoding network. This represents the low-dimensional embedding representation used for data reconstruction, where H represents the number of features, and concat(·) represents the concatenation operation; α and β represent two hyperparameters used to balance the effects of different losses. This represents the sum of clustering constraint losses for the H feature subspaces. Let H represent the sum of the difference constraint losses of the H feature subspaces.
[0028] Furthermore, the original dataset includes a text dataset and an image dataset. For the text data, words with variance less than a set variance value are first removed, and then word2vec is used to convert the text data into a vectorized text representation matrix. For the image data, image2vec is used to vectorize the original color image into an image representation matrix, and then normalization is performed to eliminate the influence of dimensions.
[0029] A second aspect of the present invention provides an interpretable multi-clustering system based on weakly supervised learning, comprising:
[0030] The vector representation module is configured to: acquire the original dataset, perform numerical vector representation on the data, and obtain the representation matrix of the data;
[0031] The feature space optimization module is configured to: optimize the projection of the data representation matrix onto different nonlinear feature subspaces to obtain the features of the corresponding subspaces, and calculate the reconstruction loss;
[0032] A sample feature annotation set is constructed based on the original dataset, a clustering constraint matrix is constructed based on the sample feature annotation set, and the clustering constraint loss of the nonlinear feature subspace is calculated based on the clustering constraint matrix.
[0033] Calculate the difference constraint loss of the nonlinear eigenspace;
[0034] An objective function is constructed based on reconstruction loss, clustering constraint loss, and difference constraint loss to determine the optimized feature subspace set.
[0035] The multi-clustering output module is configured to perform K-means clustering on the optimized feature subspace set to obtain multi-clustering results that conform to weakly supervised prior knowledge.
[0036] A third aspect of the present invention provides a computer-readable storage medium.
[0037] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps in the interpretable multi-clustering method based on weakly supervised learning as described above.
[0038] A fourth aspect of the present invention provides a computer device.
[0039] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps in the interpretable multi-clustering method based on weakly supervised learning as described above.
[0040] Compared with the prior art, the beneficial effects of the present invention are:
[0041] This invention utilizes multiple parallel deep neural networks to achieve data projection, enhancing the nonlinear representation capability of the feature subspace. The parallel structure also ensures the diversity of the feature subspace and multiple clusters to a certain extent. Embedding weakly supervised prior knowledge representing different clustering patterns into the parallel feature subspace enhances the saliency and interpretability of the subspace, helping to improve the quality of multiple clusters. Redundant control constraints reduce the similarity between subspaces, further improving the diversity of multiple clusters. By establishing the association between subspaces and specific clustering patterns through weakly supervised prior knowledge, different subspaces represent clustering patterns from different perspectives, improving the interpretability of multiple clusters.
[0042] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0043] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0044] Figure 1 This is a flowchart of an interpretable multi-clustering method based on weakly supervised learning provided in an embodiment of the present invention. Detailed Implementation
[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0046] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0047] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0048] To address the problems existing in current multi-clustering algorithms mentioned in the background, this invention utilizes the strong representational ability of deep neural networks to generate multiple parallel nonlinear subspaces, combines weakly supervised prior knowledge and redundant control constraints to enhance the saliency and interpretability of the subspaces, and applies them to the discovery of high-quality, interpretable multi-clustering patterns in large-scale data.
[0049] Example 1
[0050] This embodiment provides an interpretable multi-clustering method based on weakly supervised learning, including the following steps:
[0051] Step 1: Obtain the original dataset, convert the data into numerical vector representations, and obtain the representation matrix X of the corresponding data;
[0052] In this embodiment, the original dataset includes a text dataset and an image dataset;
[0053] For example, a text dataset can be web page text data, and an image dataset can be an image dataset from the field of computer vision.
[0054] For text data, we first remove rare words, stop words, and other words with small variance, and then use word2vec to convert the text data into a vectorized text representation matrix.
[0055] For image data, the original color image is first vectorized into an image representation matrix using image2vec, and then further normalized to eliminate the influence of dimensions.
[0056] Step 2: Project the data representation matrix x onto different nonlinear feature subspaces to obtain the features of the corresponding subspaces;
[0057] In this embodiment, multiple parallel deep neural networks are used to project the original high-dimensional data onto different nonlinear feature subspaces, as shown below:
[0058]
[0059] Among them, z h Let AE be the representation matrix of the h-th eigenspace. h (·) represents the h-th depth autoencoder, and H represents the number of nonlinear feature subspaces;
[0060] The deep autoencoder in this embodiment can be a multi-layer perceptron (MLP). An MLP is a feedforward artificial neural network model composed of multiple fully connected hidden layers. By stacking multiple hidden layers, MLPs can capture complex features and patterns, automatically extracting meaningful features from the data, thereby providing better representations for subsequent tasks.
[0061] Step 3: Based on the coarse-grained sample feature annotation set, construct a clustering constraint matrix that reflects the data under different clustering modes, construct a clustering constraint loss based on the clustering constraint matrix, and embed the clustering constraints into different feature subspaces based on the clustering constraint loss;
[0062] Specifically, it includes:
[0063] Step 301: Obtain a coarse-grained set of sample feature annotations, i.e., weakly supervised prior knowledge;
[0064] In this embodiment, the label set of a portion of the original data (e.g., 20% of the original data) is first used as a coarse-grained sample feature annotation set. The annotation information l i (1≤i≤N) roughly represents the category label of the i-th sample.
[0065] Step 302: Construct a clustering constraint matrix with reference to the coarse-grained sample feature annotation set L;
[0066] In this embodiment, the clustering constraint matrix is represented by a sample similarity matrix. If two samples can be classified into one class from a certain perspective, but do not belong to another perspective, then this similarity can be encoded into a sample similarity matrix:
[0067]
[0068] Among them, Let be the similarity between the i-th sample and the j-th sample in the h-th subspace. This represents the class label of the i-th sample in the h-th subspace. This represents the class label of the j-th sample in the h-th subspace. This indicates that two samples should be assigned to the same cluster in the h-th subspace, otherwise they belong to different classes.
[0069] Step 303: Calculate the sum of clustering constraint losses for the H feature subspaces;
[0070] In this embodiment, clustering constraint loss is constructed using the sample similarity matrix. Enforcing different feature spaces focuses on different features of the original data and follows specific clustering constraints. The specific calculation process is as follows:
[0071]
[0072] in, Let H represent the sum of clustering constraint losses for the H feature subspaces. Let i be the i-th feature in the h-th feature subspace matrix. It is the j-th feature in the h-th feature subspace matrix;
[0073] when When sample i and sample j are in the h-th subspace Z h They will be drawn closer to each other; conversely, when When sample i and sample j are in the h-th subspace Z h They will move away from each other. In this way, the h-th subspace Z... h The clustering results will be consistent with weakly supervised prior knowledge, which will be constrained by the sample similarity matrix.
[0074] The above scheme embeds clustering constraints into different feature subspaces by subsequently constructing a clustering constraint loss.
[0075] Step 4: Calculate the sum of the difference constraint losses of the H feature subspaces;
[0076] In this embodiment, adaptive difference constraints are used to control redundancy between feature subspaces, thereby enhancing the diversity of the feature space and its clusters; the calculation process is as follows:
[0077]
[0078] in, Z represents the sum of the difference constraint losses of H feature subspaces. h and Z ′ h Let h and h represent two parallel subspaces, where h ≠ h. ′ , where I represents an orthogonal identity matrix.
[0079] Step 5: Construct a unified objective function, jointly optimize the reconstruction loss, clustering constraint loss, and difference constraint loss, and generate multiple diverse saliency subspaces. The calculation process is as follows:
[0080]
[0081] Where J(Θ) represents the total loss of the model; f θ (·) represents the deep decoding network used for data reconstruction, and θ represents the parameters of the decoding network; represents a low-dimensional embedding representation used for data reconstruction, concat(·) represents the concatenation operation; α and β represent two hyperparameters used to balance the effects of different losses.
[0082] Step 6: Optimize the resulting feature subspace set The data is input into the K-means clustering algorithm, which produces H clustering results that conform to the weakly supervised prior knowledge. The clustering process is as follows:
[0083]
[0084] This invention utilizes the strong representational capabilities of deep neural networks to generate multiple parallel nonlinear subspaces, combines weakly supervised prior knowledge with redundant control constraints to enhance the saliency and interpretability of the subspaces, and applies them to the discovery of high-quality, interpretable multi-clustering patterns in large-scale data.
[0085] Example 2
[0086] This embodiment provides an interpretable multi-clustering system based on weakly supervised learning, including:
[0087] The vector representation module is configured to: acquire the original dataset, perform numerical vector representation on the data, and obtain the representation matrix of the data;
[0088] The feature subspace projection module is configured to project the representation matrix of the data onto different nonlinear feature subspaces.
[0089] The weakly supervised knowledge fusion module is configured to: construct a clustering constraint matrix reflecting the data under different clustering modes based on a coarse-grained set of sample feature annotations; construct a clustering constraint loss based on the clustering constraint matrix; and embed the clustering constraints into different feature subspaces based on the clustering constraint loss.
[0090] The data multi-clustering module is configured to cluster in the feature subspace set to obtain multi-clustering results that conform to weakly supervised prior knowledge.
[0091] Example 3
[0092] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the interpretable multi-clustering method based on weakly supervised learning described above.
[0093] Example 4
[0094] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the interpretable multi-clustering method based on weakly supervised learning as described above.
[0095] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0096] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0097] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0098] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0099] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0100] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. An interpretable multi-clustering method based on weakly supervised learning, characterized in that, The method comprises the following steps: obtaining an original data set, performing numerical vector representation on data to obtain a representation matrix of the data; projecting the representation matrix of the data into different nonlinear feature subspaces to obtain features of the corresponding subspaces and calculating a reconstruction loss; constructing a sample feature label set based on the original data set, constructing a clustering constraint matrix based on the sample feature label set, and calculating a clustering constraint loss of the nonlinear feature subspaces based on the clustering constraint matrix; calculating a difference constraint loss of the nonlinear feature subspaces, and the calculation formula of the difference constraint loss of the nonlinear feature subspaces is: wherein, denotes the sum of the diversity constraint losses of H feature subspaces, and denote two parallel subspaces respectively and , denotes an orthogonal identity matrix; constructing an objective function based on the reconstruction loss, the clustering constraint loss and the difference constraint loss, and determining an optimized feature subspace set; performing K-means clustering on the optimized feature subspace set to obtain a multi-clustering result meeting weakly supervised prior knowledge; projecting original high-dimensional data into different nonlinear feature subspaces by using multiple parallel deep neural networks to obtain data features of the corresponding subspaces; The original data set comprises a text data set and an image data set. For text data, first, words with a variance less than a set variance value are removed, and then word2vec is used to convert the text data into a vectorized text representation matrix. For image data, image2vec is used to vectorize original color images into an image representation matrix, and then further normalization is performed to eliminate the dimension effect.
2. The weakly supervised learning based interpretable multi-clustering method of claim 1, wherein, The calculation formula of the clustering constraint matrix is: wherein, is the similarity between the first sample and the second sample in the first subspace, is the similarity between the first sample and the second sample in the second subspace, is the similarity between the first sample and the second sample in the third subspace, is the similarity between the first sample and the second sample in the fourth subspace, is the class label of the first sample in the first subspace, is the class label of the first sample in the second subspace, is the class label of the first sample in the third subspace, is the class label of the first sample in the fourth subspace, is the class label of the second sample in the first subspace, is the class label of the second sample in the second subspace, is the class label of the second sample in the third subspace, is the class label of the second sample in the fourth subspace.
3. The weakly supervised learning based interpretable multi-clustering method of claim 2, wherein, The calculation formula of the clustering constraint loss of the nonlinear feature subspaces is: in, This represents the sum of clustering constraint losses for the H feature subspaces. For the first h The th eigensubspace matrix in the th eigenspace matrix One characteristic, For the first h The th eigensubspace matrix in the th eigenspace matrix One characteristic.
4. The weakly supervised learning based interpretable multi-clustering method of claim 1, wherein, The objective function is: wherein, denotes the total loss of the model; denotes a deep decoding network for data reconstruction, denotes parameters of the decoding network, denotes a low-dimensional embedding representation for data reconstruction, H denotes the number of features, denotes a concatenation operation; denotes two hyper-parameters for balancing the influence of different losses, denotes the sum of clustering constraint losses of H feature subspaces, denotes the sum of diversity constraint losses of H feature subspaces.
5. An interpretable multi-clustering system based on weakly supervised learning, characterized in that, comprises: a vector representation module configured to obtain an original data set, perform numerical vector representation on data to obtain a representation matrix of the data; a feature space optimization module configured to optimize projecting the representation matrix of the data into different nonlinear feature subspaces to obtain features of the corresponding subspaces and calculate a reconstruction loss, and the calculation formula of a difference constraint loss of the nonlinear feature subspaces is: wherein, denotes the sum of the diversity constraint losses of H feature subspaces, and denotes two parallel subspaces and , denotes an orthogonal identity matrix; constructing a sample feature label set based on the original data set, constructing a clustering constraint matrix based on the sample feature label set, and calculating a clustering constraint loss of the nonlinear feature subspaces based on the clustering constraint matrix; calculating a difference constraint loss of the nonlinear feature subspaces; constructing an objective function based on the reconstruction loss, the clustering constraint loss and the difference constraint loss, and determining an optimized feature subspace set; a multi-clustering output module configured to perform K-means clustering on the optimized feature subspace set to obtain a multi-clustering result meeting weakly supervised prior knowledge; projecting original high-dimensional data into different nonlinear feature subspaces by using multiple parallel deep neural networks to obtain data features of the corresponding subspaces; The original data set comprises a text data set and an image data set. For text data, first, words with a variance less than a set variance value are removed, and then word2vec is used to convert the text data into a vectorized text representation matrix. For image data, image2vec is used to vectorize original color images into an image representation matrix, and then further normalization is performed to eliminate the dimension effect.
6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by the processor, implements the steps in the weakly supervised learning based interpretable multi-clustering method of any one of claims 1-4.
7. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps in the weakly supervised learning based interpretable multi-clustering method of any one of claims 1-4 when executing the program.