Weak supervised learning-based interpretable multi-clustering method and system

Abstract

The invention belongs to the technical field of data mining classification and interpretable artificial intelligence, provides an interpretable multi-clustering method and system based on weak supervised learning, and adopts the technical scheme that a plurality of parallel deep neural networks are utilized to realize data projection, the nonlinear representation capability of a feature subspace is enhanced, and the data projection efficiency is improved. The parallel structure can ensure the diversity of the feature subspace and the multi-cluster to a certain extent; according to the method, weak supervision priori knowledge representing different clustering modes is embedded into parallel feature subspaces, so that the significance and interpretability of the subspaces are enhanced, and the multi-clustering quality is improved; according to the method, the similarity between the subspaces is reduced by using redundancy control constraints, and the diversity of multiple clusters is further improved; according to the method, association between the subspaces and the specific clustering modes is established through weak supervision priori knowledge, different subspaces represent the clustering modes of different perspectives, and the interpretability of multi-clustering is improved.

Description

Technical Field

[0001] The present invention belongs to the field of data mining classification and explainable artificial intelligence technology, and in particular to an explainable multi-clustering method and system based on weakly supervised learning. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the surge in big data, the diversity and complexity of data are also increasing. Traditional clustering methods can only provide a single clustering result, limiting data exploration to a single possible partition. In contrast, multi-clustering can reveal multiple non-redundant and meaningful clustering schemes simultaneously or sequentially, and can reveal multiple interesting hidden structures within the data from different perspectives. However, how to explore the clustering space with significant patterns and generate interpretable multi-clustering under the guidance of weakly supervised prior knowledge is a difficulty that needs to be solved in the current application of multi-clustering theory and technology in big data mining scenarios.

[0004] The current multi-clustering algorithms can be divided into two categories according to whether they refer to prior information, including unsupervised multi-clustering algorithms and semi-supervised multi-clustering algorithms. Unsupervised multi-clustering algorithms do not rely on any external prior information during the algorithm optimization process, and only achieve multi-cluster discovery of data by adaptively learning the intrinsic attribute characteristics of the data. In contrast, semi-supervised multi-clustering algorithms introduce known clusters as reference information during the model training process to generate alternative clusters with low overlap with known clusters. In addition, according to the different clustering feature spaces, multi-clustering algorithms can also be divided into algorithms based on the original feature space and algorithms based on feature subspaces. The algorithms based on the original space explore multiple non-redundant clusters in the high-dimensional feature space of the data, while the algorithms based on subspaces project the data into different subspaces and generate multiple clusters by constraining the redundancy of the subspaces.

[0005] In recent years, the research on multi-clustering theory and technology has developed rapidly, and related results have been widely used in fields such as image data analysis and text data mining. However, since only referring to known clusters cannot generate reliable subspaces, and subspaces and multi-clustering have problems such as low significance and poor interpretability, there is still much room for improvement in multi-clustering research by referring to known weakly supervised prior knowledge to explore interpretable, high-quality diversified clustering patterns. Summary of the invention

[0006] In order to solve at least one technical problem existing in the above-mentioned background technology, the present invention provides an interpretable multi-clustering method and system based on weakly supervised learning, which utilizes the strong representation ability of deep neural networks to generate multiple parallel nonlinear subspaces, combines weakly supervised prior knowledge with redundant control constraints to enhance the significance and interpretability of the subspace, and is applied to high-quality, interpretable multi-clustering pattern discovery of large-scale data.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A first aspect of the present invention provides an interpretable multi-clustering method based on weakly supervised learning, comprising the following steps:

[0009] Get the original data set, perform numerical vector representation on the data, and obtain the data representation matrix;

[0010] Project the data representation matrix into different nonlinear feature subspaces to obtain the features of the corresponding subspaces and calculate the reconstruction loss;

[0011] A sample feature annotation set is constructed based on the original data set, a clustering constraint matrix is ​​constructed based on the sample feature annotation set, and the clustering constraint loss of the nonlinear feature subspace is calculated based on the clustering constraint matrix;

[0012] Compute the difference-constrained loss in the nonlinear feature subspace;

[0013] Construct an objective function based on reconstruction loss, clustering constraint loss and difference constraint loss to determine the optimized feature subspace set;

[0014] The optimized feature subspace set is clustered using K-means to obtain multi-clustering results that conform to weakly supervised prior knowledge.

[0015] Furthermore, multiple parallel deep neural networks are used to project the original high-dimensional data into different nonlinear feature subspaces to obtain the data features of the corresponding subspaces.

[0016] Furthermore, the clustering constraint matrix calculation formula is:

[0017]

[0018] in, is the similarity between the i-th sample and the j-th sample in the h-th subspace, represents the category label of the i-th sample in the h-th subspace, represents the category label of the jth sample in the hth subspace, when When , the two samples should be classified into the same cluster in the hth subspace, otherwise they belong to different classes.

[0019] Furthermore, the calculation formula of the clustering constraint loss of the nonlinear feature subspace is:

[0020]

[0021] in, represents the sum of clustering constraint losses of H feature subspaces, is the i-th feature in the h-th feature subspace matrix, is the jth feature in the hth feature subspace matrix;

[0022] Furthermore, the calculation formula of the difference constraint loss of the nonlinear feature subspace is:

[0023]

[0024] in, represents the sum of the difference constraint losses of H feature subspaces, Z h and Z ′ h They represent two parallel subspaces and h≠h ′ , I represents an orthogonal identity matrix.

[0025] Furthermore, the objective function is:

[0026]

[0027] Among them, j(Θ) represents the total loss of the model; F θ (·) represents the deep decoding network used for data reconstruction, θ represents the parameters of the decoding network, represents the low-dimensional embedding representation used for data reconstruction, H represents the number of features, concat(·) represents the concatenation operation; α and β represent two hyperparameters used to balance the impact of different losses. represents the sum of clustering constraint losses of H feature subspaces, Represents the sum of the difference constraint losses of H feature subspaces.

[0028] Furthermore, the original data set includes a text data set and an image data set. For the text data, the words whose variance is less than the set variance value are first removed, and then word2vec is used to convert the text data into a vectorized text representation matrix; for the image data, image2vec is used to vectorize the original color image into an image representation matrix, and then further normalized to eliminate the dimension effect.

[0029] A second aspect of the present invention provides an interpretable multi-clustering system based on weakly supervised learning, comprising:

[0030] A vector representation module is configured to: obtain an original data set, perform numerical vector representation on the data, and obtain a representation matrix of the data;

[0031] A feature space optimization module is configured to: optimize the projection of the data representation matrix into different nonlinear feature subspaces to obtain the features of the corresponding subspaces, and calculate the reconstruction loss;

[0032] A sample feature annotation set is constructed based on the original data set, a clustering constraint matrix is ​​constructed based on the sample feature annotation set, and the clustering constraint loss of the nonlinear feature subspace is calculated based on the clustering constraint matrix;

[0033] Compute the difference-constrained loss in the nonlinear feature subspace;

[0034] Construct an objective function based on reconstruction loss, clustering constraint loss and difference constraint loss to determine the optimized feature subspace set;

[0035] The multi-clustering output module is configured to: perform K-means clustering on the optimized feature subspace set to obtain a multi-clustering result that conforms to weakly supervised prior knowledge.

[0036] A third aspect of the present invention provides a computer-readable storage medium.

[0037] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the above-mentioned interpretable multi-clustering method based on weakly supervised learning.

[0038] A fourth aspect of the present invention provides a computer device.

[0039] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the above-mentioned interpretable multi-clustering method based on weakly supervised learning are implemented.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention utilizes multiple parallel deep neural networks to realize data projection, thereby enhancing the nonlinear representation capability of feature subspaces. The parallel structure can also ensure the diversity of feature subspaces and multiple clusters to a certain extent. Weakly supervised prior knowledge representing different clustering patterns is embedded into parallel feature subspaces, thereby enhancing the significance and interpretability of subspaces, and helping to improve the quality of multiple clusters. Redundant control constraints are utilized to reduce the similarity between subspaces, thereby further improving the diversity of multiple clusters. The association between subspaces and specific clustering patterns is established through weakly supervised prior knowledge, and different subspaces represent clustering patterns from different perspectives, thereby improving the interpretability of multiple clusters.

[0042] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0044] Figure 1 It is a flow chart of an interpretable multi-clustering method based on weakly supervised learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0048] In response to the problems existing in the current multi-clustering algorithms mentioned in the background technology, the present invention utilizes the strong representation ability of deep neural networks to generate multiple parallel nonlinear subspaces, combines weakly supervised prior knowledge with redundant control constraints to enhance the significance and interpretability of the subspaces, and applies it to high-quality, interpretable multi-clustering pattern discovery of large-scale data.

[0049] Embodiment 1

[0050] This embodiment provides an interpretable multi-clustering method based on weakly supervised learning, comprising the following steps:

[0051] Step 1: Get the original data set, represent the data in numerical vector form, and obtain the corresponding data representation matrix X;

[0052] In this embodiment, the original data set includes a text data set and an image data set;

[0053] For example, the text dataset may be web page text data, and the image dataset may be an image dataset in the field of computer vision.

[0054] For text data, we first remove rare words, stop words and other words with small variance, and then use word2vec to convert the text data into a vectorized text representation matrix.

[0055] For image data, image2vec is first used to vectorize the original color image into an image representation matrix, and then further normalized to eliminate the dimension effect.

[0056] Step 2: Project the data representation matrix x onto different nonlinear feature subspaces to obtain the features of the corresponding subspaces;

[0057] In this embodiment, multiple parallel deep neural networks are used to project the original high-dimensional data into different nonlinear feature subspaces, which can be expressed as:

[0058]

[0059] Among them, z h Represents the representation matrix of the h-th feature subspace, AE h (·) represents the h-th deep autoencoder, H represents the number of nonlinear feature subspaces;

[0060] The deep autoencoder in this embodiment can adopt a multi-layer perceptron (MLP). MLP is a feed-forward artificial neural network model consisting of multiple fully connected hidden layers. By stacking multiple hidden layers, MLP can capture complex features and patterns and automatically extract meaningful features from data, thereby providing better representation for subsequent tasks.

[0061] Step 3: Based on the coarse-grained sample feature annotation set, a clustering constraint matrix reflecting the data under different clustering modes is constructed, and a clustering constraint loss is constructed based on the clustering constraint matrix. Based on the clustering constraint loss, the clustering constraint is embedded into different feature subspaces;

[0062] Specifically include:

[0063] Step 301: Obtain a coarse-grained sample feature annotation set, i.e., weakly supervised prior knowledge;

[0064] In this embodiment, the label set of some samples in the original data (such as 20% of the original data) is first used as the coarse-grained sample feature annotation set. The annotation information i (1≤i≤N) roughly represents the category label of the i-th sample.

[0065] Step 302: Construct a clustering constraint matrix with reference to the coarse-grained sample feature annotation set L;

[0066] In this embodiment, the clustering constraint matrix is ​​represented by a sample similarity matrix. If two samples can be classified into one category from a certain perspective and do not belong to another perspective, then this similarity can be encoded into a sample similarity matrix:

[0067]

[0068] Among them, is the similarity between the i-th sample and the j-th sample in the h-th subspace, represents the category label of the i-th sample in the h-th subspace, represents the category label of the jth sample in the hth subspace, It means that two samples should be classified into the same cluster in the hth subspace, otherwise they belong to different classes.

[0069] Step 303: Calculate the sum of clustering constraint losses of H feature subspaces;

[0070] In this embodiment, the clustering constraint loss is constructed using the sample similarity matrix Forced constraints on different feature spaces focus on different features of the original data and follow specific clustering constraints. The specific calculation process is:

[0071]

[0072] in, Represents the sum of the clustering constraint losses of H feature subspaces. is the i-th feature in the h-th feature subspace matrix, is the jth feature in the hth feature subspace matrix;

[0073] when When sample i and sample j are in the hth subspace Z h When When sample i and sample j are in the hth subspace Z h In this way, the h-th subspace Z h The sample similarity matrix will be constrained to produce clustering results consistent with weakly supervised prior knowledge.

[0074] The above scheme embeds clustering constraints into different feature subspaces by subsequently constructing a clustering constraint loss.

[0075] Step 4: Calculate the sum of the difference constraint losses of H feature subspaces;

[0076] In this embodiment, adaptive difference constraints are used to control the redundancy between feature subspaces and enhance the diversity of feature spaces and clusters therein; the calculation process is:

[0077]

[0078] in, represents the sum of the difference constraint losses of H feature subspaces, Z h and Z ′ h They represent two parallel subspaces and h≠h ′ , I represents an orthogonal identity matrix.

[0079] Step 5: Construct a unified objective function to jointly optimize the reconstruction loss, clustering constraint loss, and difference constraint loss to generate multiple diverse saliency subspaces The calculation process is:

[0080]

[0081] Among them, J(Θ) represents the total loss of the model; f θ (·) represents the deep decoding network used for data reconstruction, θ represents the parameters of the decoding network; represents the low-dimensional embedding representation for data reconstruction, concat(·) represents the concatenation operation; α and β represent two hyperparameters used to balance the impact of different losses.

[0082] Step 6: Set the optimized feature subspace Input to the K-means clustering algorithm, clustering to obtain H multi-clustering results that conform to weakly supervised prior knowledge The clustering process is:

[0083]

[0084] The present invention utilizes the strong representation ability of deep neural networks to generate multiple parallel nonlinear subspaces, combines weakly supervised prior knowledge with redundant control constraints to enhance the significance and interpretability of the subspaces, and applies it to high-quality, interpretable multi-clustering pattern discovery of large-scale data.

[0085] Embodiment 2

[0086] This embodiment provides an interpretable multi-clustering system based on weakly supervised learning, including:

[0087] A vector representation module is configured to: obtain an original data set, perform numerical vector representation on the data, and obtain a representation matrix of the data;

[0088] A feature subspace projection module is configured to: project the representation matrix of the data into different nonlinear feature subspaces respectively;

[0089] A weakly supervised knowledge fusion module is configured to: construct a clustering constraint matrix reflecting data under different clustering modes based on a coarse-grained sample feature annotation set, construct a clustering constraint loss based on the clustering constraint matrix, and embed the clustering constraint into different feature subspaces based on the clustering constraint loss;

[0090] The data multi-clustering module is configured to: cluster in a feature subspace set to obtain a multi-clustering result that conforms to weakly supervised prior knowledge.

[0091] Embodiment 3

[0092] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the interpretable multi-clustering method based on weakly supervised learning as described above are implemented.

[0093] Embodiment 4

[0094] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the interpretable multi-clustering method based on weakly supervised learning as described above are implemented.

[0095] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0096] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0097] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0099] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An interpretable multi-clustering method based on weakly supervised learning, characterized by: The steps include: Get the original data set, perform numerical vector representation on the data, and obtain the data representation matrix; Project the data representation matrix into different nonlinear feature subspaces to obtain the features of the corresponding subspaces and calculate the reconstruction loss; A sample feature annotation set is constructed based on the original data set, a clustering constraint matrix is ​​constructed based on the sample feature annotation set, and the clustering constraint loss of the nonlinear feature subspace is calculated based on the clustering constraint matrix; Compute the difference-constrained loss in the nonlinear feature subspace; Construct an objective function based on reconstruction loss, clustering constraint loss and difference constraint loss to determine the optimized feature subspace set; The optimized feature subspace set is clustered using K-means to obtain multi-clustering results that conform to weakly supervised prior knowledge.

2. The interpretable multi-clustering method based on weakly supervised learning according to claim 1, characterized in that: Multiple parallel deep neural networks are used to project the original high-dimensional data into different nonlinear feature subspaces to obtain the data features of the corresponding subspaces.

3. The interpretable multi-clustering method based on weakly supervised learning according to claim 1, characterized in that: The clustering constraint matrix calculation formula is: in, is the similarity between the i-th sample and the j-th sample in the h-th subspace, represents the category label of the i-th sample in the h-th subspace, represents the category label of the jth sample in the hth subspace, when When , the two samples should be classified into the same cluster in the hth subspace, otherwise they belong to different classes.

4. The interpretable multi-clustering method based on weakly supervised learning as claimed in claim 3, characterized in that: The calculation formula of the clustering constraint loss in the nonlinear feature subspace is: in, represents the sum of clustering constraint losses of H feature subspaces, is the i-th feature in the h-th feature subspace matrix, is the jth feature in the hth feature subspace matrix.

5. The interpretable multi-clustering method based on weakly supervised learning according to claim 1, characterized in that: The calculation formula of the difference constraint loss of nonlinear feature subspace is: in, represents the sum of the difference constraint losses of H feature subspaces, Z h and Z ′ h They represent two parallel subspaces and h≠h ′ , I represents an orthogonal identity matrix.

6. The interpretable multi-clustering method based on weakly supervised learning according to claim 1, characterized in that: The objective function is: Among them, J(Θ) represents the total loss of the model; f θ (·) represents the deep decoding network used for data reconstruction, θ represents the parameters of the decoding network, represents the low-dimensional embedding representation used for data reconstruction, H represents the number of features, concat(·) represents the concatenation operation; α and β represent two hyperparameters used to balance the impact of different losses. represents the sum of clustering constraint losses of H feature subspaces, Represents the sum of the difference constraint losses of H feature subspaces.

7. The interpretable multi-clustering method based on weakly supervised learning according to claim 1, characterized in that: The original data set includes a text data set and an image data set. For the text data, we first remove words whose variance is less than the set variance value, and then use word2vec to convert the text data into a vectorized text representation matrix; for the image data, we use image2vec to vectorize the original color image into an image representation matrix, and then further normalize it to eliminate the dimension effect.

8. An interpretable multi-clustering system based on weakly supervised learning, characterized by: include: A vector representation module is configured to: obtain an original data set, perform numerical vector representation on the data, and obtain a representation matrix of the data; A feature space optimization module is configured to: optimize the projection of the data representation matrix into different nonlinear feature subspaces to obtain the features of the corresponding subspaces, and calculate the reconstruction loss; A sample feature annotation set is constructed based on the original data set, a clustering constraint matrix is ​​constructed based on the sample feature annotation set, and the clustering constraint loss of the nonlinear feature subspace is calculated based on the clustering constraint matrix; Compute the difference-constrained loss in the nonlinear feature subspace; Construct an objective function based on reconstruction loss, clustering constraint loss and difference constraint loss to determine the optimized feature subspace set; The multi-clustering output module is configured to: perform K-means clustering on the optimized feature subspace set to obtain a multi-clustering result that conforms to weakly supervised prior knowledge.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the interpretable multi-clustering method based on weakly supervised learning as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the interpretable multi-clustering method based on weakly supervised learning as described in any one of claims 1 to 7 are implemented.

Images