Clustering method and device based on gaussian mixture model, equipment and storage medium

Abstract

Embodiments of the present application disclose a clustering method and device based on a Gaussian mixture model, equipment and a storage medium. The method comprises: obtaining original data related to a target task, wherein the target task is any one of a customer clustering task, an image clustering task, a text clustering task, and a biological information clustering task; dividing the original data into a plurality of density regions based on a Gaussian mixture model, wherein the difference between different density regions is greater than a preset threshold; constructing an adjacency matrix of the density regions; and performing spectral clustering on the density regions according to the adjacency matrix to obtain a clustering result of the target task. The present application can accurately cluster the original data related to the target task and provide a good basis for subsequent processing of the target task.

Description

Technical Field

[0001] This invention relates to the field of data clustering technology, and in particular to clustering methods, apparatus, devices and storage media based on Gaussian mixture models. Background Technology

[0002] Clustering is a general data processing technique that uses artificial intelligence to divide samples from a given dataset into different groups. Clustering is widely used in retail, media, image / text analysis, and bioinformatics research for clustering data.

[0003] However, there is a significant drawback to using existing clustering methods to cluster data: the effectiveness drops rapidly when there is uneven density in the data distribution, and clustering algorithms often face the problem of having a large number of parameters and being difficult to select. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention aims to provide a clustering method, apparatus, device, and storage medium based on a Gaussian mixture model, which can accurately cluster raw data related to a target task, providing a solid foundation for subsequent processing of the target task.

[0005] The technical solution of this invention is implemented as follows:

[0006] In a first aspect, embodiments of the present invention provide a clustering method based on a Gaussian mixture model, comprising:

[0007] Obtain raw data related to the target task, where the target task is any one of customer clustering, image clustering, text clustering, or bioinformatics clustering.

[0008] Based on the Gaussian mixture model, the original data is divided into several density regions, where the difference between different density regions is greater than a preset threshold.

[0009] Construct the adjacency matrix of the density region;

[0010] Based on the adjacency matrix, spectral clustering is performed on the density regions to obtain the clustering results for the target task.

[0011] Optionally, based on a Gaussian mixture model, the original data is divided into several density regions, including:

[0012] The density of the original data is fitted based on the Gaussian mixture model to determine the density inflection points.

[0013] Based on density inflection points, the original data is divided into several density regions.

[0014] Optionally, the density of the original data is fitted based on a Gaussian mixture model to determine density inflection points, including:

[0015] Generate an increasing density variable sequence of the original data, wherein the original data is sorted in ascending order of density in the increasing density variable sequence;

[0016] The increasing density variable sequence was fitted using a Gaussian mixture model to determine the density inflection points.

[0017] Optionally, construct the adjacency matrix of the density regions, including:

[0018] Based on the mutual nearest neighbor MNN relationship, each density region is clustered to determine the density ratio of each density region;

[0019] Construct the adjacency matrix of each density region based on its density ratio.

[0020] Optionally, based on the adjacency matrix, spectral clustering is performed on the density regions to obtain the clustering results for the target task, including:

[0021] Construct the Laplacian matrix based on the adjacency matrix;

[0022] Spectral clustering of density regions is performed based on the eigenvectors of the Laplacian matrix to obtain the clustering results for the target task.

[0023] Optionally, after obtaining the clustering results for the target task, the following may also be included:

[0024] Generate processing information corresponding to the clustering results;

[0025] Processing information is sent to the user terminal, which guides the user in processing different clustering results of the target task.

[0026] Optionally, the spectral clustering can be any one of the following: -NN-based spectral clustering, k-NN-based spectral clustering, Gaussian kernel-based spectral clustering, kernel parameter self-adjustment spectral clustering, or common nearest neighbor-based spectral clustering.

[0027] Secondly, embodiments of the present invention provide a clustering device based on a Gaussian mixture model, comprising: a data acquisition module, a region partitioning module, a matrix construction module, and a spectral clustering module;

[0028] The data acquisition module is used to acquire raw data related to the target task, wherein the target task is any one of customer clustering task, image clustering task, text clustering task, or bioinformatics clustering task;

[0029] The region segmentation module is used to divide the original data into several density regions based on the Gaussian mixture model, wherein the difference between different density regions is greater than a preset threshold.

[0030] The matrix construction module is used to construct the adjacency matrix of density regions;

[0031] The spectral clustering module is used to perform spectral clustering on density regions based on the adjacency matrix to obtain the clustering results for the target task.

[0032] Thirdly, embodiments of the present invention provide an electronic device, including: a processor, the processor being configured to implement a clustering method based on a Gaussian mixture model having any of the features described in the first aspect when executing a computer program.

[0033] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a clustering method based on a Gaussian mixture model having any of the features of the first aspect described above.

[0034] This invention provides a clustering method, apparatus, device, and storage medium based on a Gaussian mixture model. By using a Gaussian mixture model, the original data is divided into several density regions, and then an adjacency matrix is ​​constructed for each density region. Based on this adjacency matrix, spectral clustering is performed on the density regions to obtain the clustering results for the target task. Compared with existing technologies, this invention proposes a novel approach using a Gaussian mixture model for density stratification. This allows for understanding the multi-density clustering distribution during the adjacency matrix construction process. Furthermore, a new density ratio is developed based on nearest neighbors and density stratification, effectively reducing density imbalance among different instances in the clusters, improving practical applicability, and resulting in a more balanced adjacency matrix that better adapts to multi-density data. This enables accurate clustering of the original data related to the target task, providing a solid foundation for subsequent processing of the target task. Attached Figure Description

[0035] Figure 1 A flowchart illustrating a clustering method based on a Gaussian mixture model provided in an embodiment of the present invention;

[0036] Figure 2 The true distribution of six multi-density synthetic datasets;

[0037] Figure 3 Visualization of the Jain dataset;

[0038] Figure 4 The visualization results for dataset D1;

[0039] Figure 5 Visualization results for dataset D2;

[0040] Figure 6 The visualization results for dataset D3;

[0041] Figure 7 Visualization results for the Pathbased dataset;

[0042] Figure 8 A visualization of the Compound dataset;

[0043] Figure 9 The true distribution of six single-density synthetic datasets;

[0044] Figure 10 This is a visualization of the clustering results of Stratified multi-densityspectral clustering using Gaussian mixture model (SMDSC) on six single-density synthetic datasets.

[0045] Figure 11 For different and their effects on SMDSC;

[0046] Figure 12 Comparing the NMI results of SMDSC with those of the non-hierarchical spectral clustering algorithm;

[0047] Figure 13 Here are some examples from four image datasets;

[0048] Figure 14 A schematic diagram of a clustering device based on a Gaussian mixture model provided in an embodiment of the present invention;

[0049] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0050] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0051] The terms "first," "second," etc., used in the specification, claims, and drawings of this invention are used to distinguish different objects, not to define a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0052] It should also be noted that the various embodiments of the present invention described below can be executed individually or in combination with each other, and the embodiments of the present invention do not impose specific limitations in this regard.

[0053] Figure 1 This is a flowchart illustrating a clustering method based on a Gaussian mixture model, provided by an embodiment of the present invention. The method disclosed in this embodiment is applicable to electronic devices (such as computer equipment, servers, etc.). Figure 1 As shown, the method includes:

[0054] S110. Obtain raw data related to the target task, wherein the target task is any one of customer clustering task, image clustering task, text clustering task, or bioinformatics clustering task.

[0055] The target task is related to the clustering scenario of user needs, that is, the target task is any one of customer clustering task, image clustering task, text clustering task, or bioinformatics clustering task.

[0056] For example, customer clustering tasks can help a retailer segment its customers into different groups: high-end consumers, mid-range consumers, and low-end consumers, in order to develop corresponding marketing strategies; image clustering tasks can divide pixels in an image into different regions, thus enabling more accurate clustering; text clustering tasks can divide text data into different topics or categories (e.g., dividing a news article into different topics: politics, economics, and sports, in order to further summarize different topics). In addition, text clustering tasks can be used for positive news analysis to monitor public opinion; bioinformatics clustering tasks can divide gene or protein sequences into different families or categories, such as transcription factors, enzymes, and receptors, in order to better understand the functions and structures of different families.

[0057] The electronic device first determines the target task, and then acquires the raw data related to the target task through the acquisition unit.

[0058] S120. Based on the Gaussian mixture model, the original data is divided into several density regions, where the difference between different density regions is greater than a preset threshold.

[0059] Specifically, the method of "dividing the original data into several density regions based on the Gaussian mixture model" in step S120 can include the following two steps:

[0060] Step A1: Fit the density of the original data based on the Gaussian mixture model to determine the density inflection points.

[0061] Electronic devices can first generate an increasing density variable sequence of raw data, in which the raw data is sorted in ascending order of density; then, the increasing density variable sequence is fitted based on a Gaussian mixture model to determine the density inflection points.

[0062] The original data consists of multiple samples. In clustering problems, sample density usually refers to how densely packed the samples are. High-density regions are those where there are many objects around the sample and they are relatively close together, while low-density regions are the opposite.

[0063] Sample density is defined based on nearest neighbor distance. Given a dataset, the nearest neighbor distance of a sample is the average distance to its nearest neighbors:

[0064]

[0065] d(x i ,x j ) represents sample x i and x j The Euclidean distance, N k (x i ) represents x i The set of k nearest neighbors, sample x i density ρ i It can be easily defined as the reciprocal of the k-nearest neighbor distance:

[0066]

[0067] Equation (2) implies that the sparser the sample distribution, the lower the density. After sorting the samples in ascending order of density, the density histogram can be easily obtained. Next, a Gaussian mixture model is used to fit the density histogram. Given an increasing density variable sequence P = {ρ...} l |l∈1,2,...,n}, where ρ l Represents the sorted sample x l The density. A Gaussian mixture model consisting of Q components can be described as:

[0068]

[0069] Where Θ=(α1,α2,...,α Q ; θ1,θ2,...,θ Q α represents the model parameters. q Let α represent the q-th coefficient. q ≥0 and φ(ρ|θ q The q-th fractional model is called the Gaussian distribution density function, which is defined as:

[0070]

[0071] Finding the parameter set Θ that maximizes P(ρ|Θ) is called the maximum likelihood estimate of Θ. To estimate Θ, we first need to define the log-likelihood function:

[0072]

[0073] Where γ lq Represents ρ l The probability from the q-th sub-model. Directly solving for the parameters in formula (5) is quite complex. Another solution is to use the iterative EM algorithm, which mainly consists of two steps:

[0074] Step E: Based on the current model parameters, the sub-model... q For the observed data ρ l The responsiveness is calculated as follows:

[0075]

[0076] M-step: The parameters for the next iteration are calculated as follows:

[0077]

[0078] The EM algorithm requires initial parameter initialization, typically performed using the K-means algorithm. Afterward, E-steps and M-steps iterate continuously, stopping when the log-likelihood function value no longer shows significant change. The probability density curve can be plotted using the acquired parameters. Regions of different densities can be distinguished by the troughs between adjacent peaks. Samples with densities located at troughs are called transition samples. This means that different density layers are distinguished by the density of these transition samples.

[0079] Let P = {ρ} l |l∈1,2,...,n} represents the set arranged in ascending order of sample density, i.e., ρ l-1 <ρ l <ρ l+1 Using formula (3), the turning point sample x l density ρ l The following conditions must be met:

[0080] P(ρ l |Θ)<P(ρ l-1 |Θ)andP(ρ) l+1 |Θ)>P(ρ l |Θ),1<l<n. (10)

[0081] Step A2: Divide the original data into several density regions based on the density inflection points.

[0082] Different density layers are determined based on the density of the transition samples. The corresponding density regions are defined by Δ={π q|q=1,2,...,Q} represents, where π q ={x h |h=1,2,...,H q Let} represent the sample in the q-th density region. Another key issue is choosing an appropriate number of Gaussian components to fit the distribution of the curve and density histogram. Based on clustering metrics, the Davies Bouldin Index (DBI) is used to evaluate the quality of density stratification, and it is defined as follows:

[0083]

[0084] in It is a density layer π q The divergence within, Q is the number of layers, ρ l It is sample x l density, c q It is a density layer π q The average density value is defined as:

[0085]

[0086] Formula (11) aims to find the optimal density stratification so that the density difference of samples within the same density stratum is small and the difference between different density stratums is more significant.

[0087] Given an interval [2, T], with a step size of 1 for Q, the corresponding minimum value is calculated using formula (11). The value is used to set the final number of density regions Q.

[0088] In this step, a Gaussian mixture model is first used to fit the local density of the samples to find density inflection points, dividing the samples into different density regions and effectively separating samples with large density differences. This results in minimal density differences within the same density level and significant density differences between different levels. Each density level is then clustered separately to find multiple density peaks, generating initial sub-clusters.

[0089] S130, the adjacency matrix of the constructed density region.

[0090] Specifically, the method for "constructing the adjacency matrix of the density region" in step S130 may include the following two steps:

[0091] Step B1: Based on the mutual nearest neighbor MNN relationship, cluster each density region separately to determine the density ratio of each density region.

[0092] Mutual nearest neighbors refer to two samples considering each other as their neighbors. Specifically, given a dataset U and any sample x i x kIndicates sample x i The k-th nearest neighbor, x i The set of k nearest neighbors is defined as:

[0093] N k (x i )={x j |x j ∈U and d(x) i ,x j )<d(x i ,x k (13)

[0094] Sample x i The reverse k-nearest neighbor set R k (x i ) is a neighbor containing x i The object is defined as:

[0095] R k (x i )={x j |x j ∈U and x i ∈N k (x j (14)

[0096] Therefore, the set of MNNs is represented as M k (x i ), can be described as:

[0097] M k (x i ) = N k (x i )∩R k (x i (15)

[0098] Iterate through all samples in the dataset. If two objects satisfy the MNN relationship, set the elements of the adjacency matrix to 1; otherwise, set them to 0. The detailed definition is as follows:

[0099]

[0100] Adjacency matrices based on MNNs help to reasonably distinguish the boundaries of different clusters, especially when the density differences between clusters are significant and they are close together. Objects in high-density regions are less likely to consider samples in low-density regions as neighbors. Therefore, the relationships in MNNs are difficult to satisfy, and the boundaries are easier to determine.

[0101] While the adjacency matrix constructed by MNN is more efficient, it loses the distance information between samples. This means that the element values ​​in the matrix, i.e., 1 or 0, can indicate whether a relationship exists between objects, but not the strength of that relationship. Here, based on formula (16), a new adjacency matrix considering similarity is defined as:

[0102]

[0103] Let Λ = {π} m |m=1,...,Q} is a set of density regions arranged in descending order of average density, c m Representing the density region π m The average value. Based on the region with the maximum average density π1, the density ratio of each region can be calculated as follows:

[0104]

[0105] Step B2: Construct the adjacency matrix of the density regions based on the density ratio of each density region.

[0106] The tendency for samples from different density regions to be balanced during measurement allows spectral clustering to avoid distributing more high-density regions across multiple parts, thus preventing the accidental merging of low-density regions. Applying a certain degree of similarity contraction to different low-density regions can balance the similarity matrix. The density ratio r is defined based on formula (18). m The similarity of samples in low-density regions, i.e., when m>1, can be enhanced by the following formula:

[0107]

[0108] Where ψ(x) i ) represents x i The index of the density layer, r m x represents i and x j The density ratio of samples belonging to the same density region. This formula indicates that when calculating samples in low-density regions, similarity will be increased to a certain extent based on the density ratio. Simply tightening low-density regions based on the density ratio can enhance the similarity of samples in density layer π2 (e.g., samples c and d in cluster C3). Figure 6 This enhancement of low-density regions increases the similarity of boundary samples (e.g., samples a and b) of C1 and C2, leading to the merging of clusters C1 and C2. To further adapt to the distribution of complex datasets, a scaling factor λ is defined according to formula (18) to further amplify or reduce the density ratio.

[0109] τ m =λ*r m,m=1,...,Q. (20)

[0110] The final adjacency matrix is ​​defined to account for density differences in different regions:

[0111]

[0112] Similarity scaling based on density hierarchy helps to build a more balanced adjacency matrix, which can avoid spectral clustering focusing on high-density regions and thus obtain better results.

[0113] In this step, a more balanced adjacency matrix can be designed based on the density ratio of different density layers.

[0114] S140. Based on the adjacency matrix, perform spectral clustering on the density regions to obtain the clustering results for the target task.

[0115] Specifically, the method in step S140, "performing spectral clustering of the density regions based on the adjacency matrix to obtain the clustering results for the target task," can include the following two steps:

[0116] Step C1: Construct the Laplace matrix based on the adjacency matrix.

[0117] Based on the adjacency matrix defined in the above steps, the overall algorithm flow of hierarchical multi-density spectral clustering is described as follows.

[0118] Let U = {x} i |i∈1,2,...,n} is a sample set, where x i It is a sample containing d features, C = {C u |u∈1,2,...,K} is a set of clusters, where C u Let G = (V, W) represent the u-th cluster, which is a graph model where V represents the vertex set and W represents the edge set. The adjacency matrix of graph G can be constructed in various ways, such as considering neighbors or defining a distance threshold. Let the objects in U be the vertices in the graph, then the adjacency matrix of the graph can be represented as:

[0119] A={a ij} n×n (twenty two)

[0120] Where a ij =0 means x i and x j There are no edges connecting them; otherwise, there are edges connecting them, and the similarity is a. ij Therefore, the diagonal matrix D can be represented as:

[0121] D={d ij} n×n (twenty three)

[0122]

[0123] The Laplace matrix L of graph G can be further defined as:

[0124] L = DA (25)

[0125] As one of the most widely used spectral clustering algorithms, the Ng-Jordan-Weiss (NJW) method uses a normalized similarity matrix as the graph Laplacian matrix. This scheme normalizes the diagonal elements while scaling the off-diagonal elements accordingly. In this setting, the normalized Laplacian matrix L... nor Defined as:

[0126]

[0127] Step C2: Perform spectral clustering on the density region based on the eigenvectors of the Laplacian matrix to obtain the clustering results for the target task.

[0128] Eigenvectors corresponding to the smallest eigenvalues ​​of the Laplacian matrix can be used for meaningful clustering. In formula (26), L nor The eigenvectors corresponding to the K smallest eigenvalues ​​are used for two-stage clustering, a process performed using the K-means algorithm. The similarity matrix generated in step S130 is applied to formula (22), and the complete clustering result can be obtained according to traditional spectral clustering.

[0129] In this step, the Laplacian matrix was constructed from the adjacency matrix, and the data was clustered in two stages using the eigenvectors of the Laplacian matrix.

[0130] In one embodiment, the spectral clustering is any one of the following: spectral clustering based on ω-NN, spectral clustering based on k-NN, spectral clustering based on Gaussian kernel, spectral clustering with self-adjusting kernel parameters, and spectral clustering based on common nearest neighbors.

[0131] This invention constructs a novel spectral clustering technique to adapt to more complex multi-density data distributions. First, the data is density-stratified using the probability density of a Gaussian mixture model; the number of strata is automatically determined by a defined divergence index. Effective density stratification helps distinguish samples of different densities, preventing clustering techniques from focusing on high-density regions while ignoring low-density regions. Then, a novel density ratio based on nearest neighbors and density stratification is established, effectively reducing density imbalances among different instances during clustering and improving practical applicability, particularly advantageous in clusters with unclear boundaries. Finally, an optimized adjacency matrix and multi-density spectral clustering techniques are introduced to enhance the clustering performance of multi-density data while maintaining the clustering capabilities of single-density data.

[0132] Optionally, after step S140 is completed, processing information corresponding to the clustering results can be generated; the processing information can be sent to the user terminal, wherein the processing information is used to guide the user to process different clustering results of the target task.

[0133] The following are some exemplary implementations used to illustrate the experimental results of the Gaussian mixture model-based clustering method and other comparative algorithms provided in this invention on different datasets, comprising five subsections. After introducing the experimental setup, Section (2) compares the clustering performance of SMDSC on 12 synthetic datasets, including 6 types of multi-density data and 6 classic single-density data. In Section (3), the core parameters k and λ of SMDSC are analyzed in detail. To further compare the performance of each algorithm more comprehensively, ten real datasets are used to test the effects in Section (4). Finally, to verify the advantages of optimizing the adjacency matrix, three different combinations of defined adjacency matrices are used in Section (5) to evaluate the performance gap.

[0134] (1) Experimental setup

[0135] To test the impact of the proposed algorithm on multi-density data, Section (2) uses six multi-density synthetic datasets: Jain, D1, D2, D3, Pathbased, and Compound. These six datasets contain different shapes, boundary distances, complexities, and varying degrees of density differences, which is beneficial for analyzing the performance gaps of different methods on various multi-density data. In addition to testing multi-density data, the clustering ability of the method in this patent is intended to be maintained on traditional single-density data. To this end, six other single-density synthetic datasets were also used to test the performance of SMDSC on similar density data, including six complex datasets: Aggregation, Zigzag, Spiral, R15, Flame, and Complex9. Figures 2 to 10 In this context, the horizontal and vertical coordinates of the image are the two-dimensional coordinates from the original dataset.

[0136] Table 1 Dataset Information

[0137]

[0138] In addition, to verify the algorithm's performance on real-world problems, ten real-world datasets from the UCI database were tested: Seeds, Banknote, Hepatitis, Appendicitis, Glass, Newthyroid, Cleveland, Balance, Vehicle, and Satimage. In summary, detailed information about the datasets used is presented in Table 1; each dataset was standardized before clustering.

[0139] To comprehensively evaluate the effectiveness of the proposed method, three types of clustering algorithms were used for comparative experiments:

[0140] 1) Spectral clustering algorithms: spectral clustering based on ò-NN (SP-ò), spectral clustering based on k-NN (SP-k), spectral clustering based on Gaussian kernel (SP-G), spectral clustering with self-adjusted kernel parameters (SP-S), and spectral clustering based on common nearest neighbor (SP-C).

[0141] 2) Density-based clustering, including density-based spatial clustering of applications with noise (DB), density peak clustering (DP), variational density peak clustering (VD), and density peak clustering with connectivity estimation (DC).

[0142] 3) Ensemble clustering methods, including Locally Weighted Graph Partitioning (LW) and Spectral Ensemble Clustering (SEC).

[0143] Table 2 Parameter settings for different clustering methods

[0144]

[0145] For SMDSC, due to the adjustable scaling parameter λ, σ in Equation (21) is fixed at 1 to reduce the number of parameters. Additionally, the number of density layers is automatically determined simply by setting the number of maximum Gaussian components to a fixed value. Here, T is fixed at 10 and not adjusted. Except for DB and VD, each algorithm requires the actual number of clusters as a parameter input. The settings for other comparison algorithms are implemented based on suggestions in the original paper or optimal settings from the parameter pool; detailed settings for various algorithms are reported in Table 2.

[0146] Normalized mutual information (NMI) and the adjusted rand index (ARI) were used as standards to evaluate the performance of the clustering algorithms for experimental analysis. To clarify the results, the best results (i.e., higher NMI, ARI, and lower ranking) are highlighted in bold in Tables 3, 4, and 5. Furthermore, the second-best results are underlined to make the information in the tables easier to understand.

[0147] (2) Synthetic dataset testing

[0148] 1) Results of multi-density data

[0149] Figure 2 The true distribution of six multi-density synthetic datasets is shown in the figure, with various types of clusters distributed in each subplot, displayed with different colors and symbols.

[0150] Visualization of clustering results from different algorithms, as shown below Figures 3 to 8 As shown, the NMI and ARI values ​​are recorded in Table 3 accordingly.

[0151] Each clustering algorithm exhibits significant differences across various datasets. Compared to other spectral clustering methods SP-o, SP-k, and SP-G, these methods provide relatively good results on data with small density differences and well-defined boundaries, such as the Jain dataset. With appropriate parameter selection, all three methods can construct good clusters, thus achieving excellent results. However, due to the large density differences in datasets D1, D2, and D3, their results are poor, such as... Figure 4-6 Subgraphs (a), (b), and (c) are shown. The boundary densities of different clusters abruptly change, and the boundaries between clusters are not very clear. In this case, if the chosen parameter is small, it is difficult for low-density cluster samples to form connections in the graph. If the chosen parameter is large, high-density data will connect with low-density data. As for the more complex mixed data Pathbased and Compound, SP-o, SP-k, and SP-G are more difficult to handle. For the other two spectral methods, SP-S and SP-C, except for SP-S which achieves better clustering results on D1, both methods perform poorly on other data. The self-adjusting patterns in SP-S are difficult to adapt to various distributions, while the common nearest neighbors in SP-C are scarce on multi-density data, thus always giving poor results.

[0152] For the density-based clustering algorithm DB, consistent with SP-ò, the parameters are difficult to tune for multi-density data, and DB performs poorly on all datasets. For DP, density peaks always fall in high-density regions, causing high-density clusters to be split into multiple parts, while low-density clusters are clustered into nearby high-density regions. DP's clustering performance is poor across all multi-density data. Compared to the original DP algorithm, the improved VD and DC algorithms achieve better results. VD uses density stratification to distinguish different samples, effectively improving the clustering performance of multi-density data, achieving the best results on Jain, D1, Pathbased, and Compound. For the remaining D2 and D3, only a few samples are mis-clustered. In comparison, DC's results are not as good as VD's, but it shows a significant improvement over the original DP on Jain and Compound.

[0153] For ensemble clustering algorithms, they showed poor results on all multi-density datasets. This indicates that multi-density data remains challenging even when combining the results of multiple base clusterings. Finally, for SMDSC, excellent results were achieved on all six multi-density datasets, especially for complex compounds, where SMDSC provided complete clustering.

[0154] Experimental results show that density stratification is effective for multi-density data. By enhancing the similarity of low-density clusters to a certain extent, a more balanced similarity matrix can be constructed, thus avoiding spectral clustering from focusing on high-density clusters while ignoring low-density clusters. Simultaneously, the similarity matrix construction based on MNN can effectively distinguish the boundaries of various clusters, further improving the spectral clustering effect for multiple data distributions. Table 3 records the NMI and ARI results of different clustering methods on six multi-density datasets.

[0155] Table 3 Clustering results of different clustering methods on six multi-density datasets.

[0156]

[0157]

[0158] As shown in Table 3, SMDSC exhibits the best clustering performance across all multi-density synthetic data. While spectral methods perform well on Jain and D1 in individual cases, they are generally unsuitable for multi-density data. For VD, the clustering results are always significant. The remaining comparison algorithms, however, demonstrate poor performance. Finally, considering all multi-density data, the ranking results for each algorithm are summarized in the last row of Table 3. Consistent with the experimental visualization results, SMDSC demonstrates superior ranking, ranking first with an average ranking performance of 1.2, indicating the effectiveness of the proposed SMDSC in multi-density clustering.

[0159] 2) Single-density data results

[0160] The actual distributions of the six single-density synthetic datasets are as follows: Figure 9 As shown.

[0161] SMDSC primarily deals with multi-density data; therefore, the visualization results for single-density synthetic data only show the proposed method. Figure 10 The results of SMDSC on different single-density datasets are recorded. As can be seen from the figure, SMDSC exhibits excellent clustering performance in various complex shapes. Although a small number of boundary objects are incorrectly clustered, all clusters are correctly distinguished overall.

[0162] Table 4. Clustering results of different clustering methods on six single-density datasets.

[0163]

[0164]

[0165] To analyze the differences between SMDSC and other comparative methods, Table 4 records the NMI and ARI results of different algorithms on single-density data. Considering the NMI index, SP-ò, SP-k, and SP-G achieve better results in most cases. In particular, for Zigzag, Spiral, and R15, these algorithms almost all achieve superior results.

[0166] On Spiral, R15, and Complex9, SMDSC achieved performance tied for first place with other methods. For Aggregation and Zigzag, SMDSC ranked second with a slight difference of 0.01. Consistent with SMDSC, VD and DC showed significant performance in most cases. As for other methods, clustering results were better if the distribution boundaries of the clusters were relatively clear, such as in R15. However, performance decreased significantly if the boundaries were ambiguous, such as in Flame.

[0167] The results from ARI are similar to those from NMI. SMDSC achieved state-of-the-art results on almost all datasets, and its clustering results for VD and DC were superior to other methods. The ranking results further validated that SMDSC can effectively cluster multi-density data while maintaining the clustering ability of traditional single-density data.

[0168] (3) Parameter Analysis

[0169] Section (1) introduced that SMDSC has all parameters fixed except for k and λ. To analyze the impact of the two key parameters k and λ on SMDSC, this section presents the NMI results of the synthetic dataset under 12 different parameters in the form of three-dimensional plots, such as... Figure 11 As shown.

[0170] The number of nearest neighbors in the experiments ranged from 5 to 30, with a step size of 1. The scaling factor λ iterated from 1 to 9 with a step size of 2. Due to the distribution and size of the datasets, different datasets typically achieve optimal results at specific values ​​of k. Therefore, the optimal value of parameter k inevitably needs to be heuristically explored. For Jain, clustering performance is best if k is greater than 6. For D3, the value of k is best below 25. As for other data, D1 is almost unaffected by k, while the others require appropriate values ​​of k for better performance. Figure 11 It can be seen that the parameter k of SMDSC is crucial for complex distributions and real datasets of different sizes.

[0171] In comparison, the effect of the parameter λ is more noteworthy. As analyzed above, λ should yield better results in more complex data distributions. Figure 11 (e) and Figure 11 (f) shows that Pathbased and Compound achieve better results when λ > 7 or λ < 3, respectively. This experiment demonstrates that the scaling factor definition is practical for clusters with multiple densities and complex distributions. Furthermore, the heuristic search range is controllable, and the algorithm can quickly obtain the optimal value.

[0172] In experiments with parameter λ, for relatively simple multi-density data such as Jain, D1, D2, and D3, λ requires almost no adjustment. If λ is 1, the algorithm can directly obtain the optimal result. Similarly, for each single-density data (e.g., Figure 11 (g) to Figure 11 As shown in (l), if λ is 1, each algorithm can achieve optimal results. Especially for the five datasets Zigzag, Spiral, R15, Flame, and Complex9, if SMDSC takes a suitable k, the clustering results are almost unaffected by λ. The above analysis shows that the definition of λ is necessary for complex datasets. However, in a more general case, the SMDSC algorithm can directly cluster to good results without the parameter λ. In this case, the algorithm depends only on the neighbor parameter k. From Figure 11 Based on the results and the experience gained from the experiment, it is recommended that k be in the range of 10-15 and λ be in the range of 3-7.

[0173] (4) Real dataset testing

[0174] Results on real-world datasets are more useful for testing the performance of SMDSC on practical problems. Since high-dimensionality data is not easily visualized, the NMI and ARI metrics are used to test the performance of various algorithms. Table 5 records the experimental results on different datasets. Considering NMI, SMDSC achieves relatively good results on seven real-world datasets. Particularly for Banknote and Newthyroid, SMDSC significantly outperforms other comparative methods. For other data, SMDSC also demonstrates excellent performance. Regarding the ARI metric, SMDSC is slightly lower than DP on Banknote, but performs better on the remaining data. Compared to the spectral clustering algorithms SP-ò, SP-k, SP-G, SP-S, and SP-C, SMDSC achieves better results in most cases, demonstrating the effectiveness of density-based hierarchical clustering of the dataset. Density-based algorithms and ensemble algorithms generally show poor results.

[0175] Overall, SMDSC performs better than the twelve algorithms tested, ranking 1.4. The statistical ranking further demonstrates that the proposed method is effective and usable in two or higher dimensions, contributing to the solution of complex real-world clustering problems.

[0176] Table 5. Clustering results of different clustering methods on ten real datasets.

[0177]

[0178]

[0179] (5) Compare spectral clustering without hierarchical structure

[0180] To further illustrate the effect of density stratification, this section uses three non-stratified spectral clustering algorithms to compare their clustering performance with that of the proposed method. The settings for the three comparison methods are as follows:

[0181] 1) SP-G: Refer to the SMDSC settings, σ is set to 1.

[0182] 2) SP-MNN: In this method, the adjacency matrix a ij Depends on x i and x j Does the relationship between neighbors satisfy the condition of mutual nearest neighbors? If it does, then a ij =1; otherwise a ij The value is 0. Consistent with the settings of SMDSC, this method achieves the optimal clustering result in the interval [5,30] with a step size of 1.

[0183] 3) SP-MNN-G: If x i and x j It is MNN, aij Using the Gaussian kernel function exp(-||x) i -x j || 2 / 2σ 2 ) calculate, otherwise, a ij It is 0. Similarly, according to the SMDSC settings, σ is also set to 1 in this method.

[0184] like Figure 12 As shown, the NMI results for spectral clustering considering density stratification are superior to the non-stratified strategy across ten datasets. The improvement is particularly pronounced for Banknote, Appendicitis, and Newthyroid. While the improvement trend is not significant for other datasets, SMDSC achieves the best results.

[0185] In summary, the experiments demonstrate that the density stratification strategy using Gaussian mixture models is effective, and density stratification-based spectral clustering can effectively improve the performance of multi-density data.

[0186] With the development of image acquisition technology and the widespread use of smartphones, the amount of image data people acquire has exploded. Effective methods are needed to process and utilize this high-dimensional image data with multi-density distributions. Image data contains rich visual information, but understanding and analyzing it is a complex task. Image clustering can reveal the similarities and differences between images. Image clustering first requires extracting pixels from the image, using pixels as features, and finally grouping highly similar images into the same cluster and low-similarity images into different clusters. The image clustering process can be divided into steps such as data collection, data preprocessing, feature extraction, feature dimensionality reduction, clustering algorithm selection, cluster analysis, and cluster evaluation. Currently, image clustering is widely used in image retrieval, image clustering, image understanding, and image analysis and mining. The SMDSC algorithm is applied to image clustering analysis to verify the practical application capability of this invention.

[0187] Figure 13 Four image datasets were used, including Coil20. [The datasets used include OlivettiFaces (Olivetti), USPS, and Yale. The Coil20 dataset contains image sequences of 20 different objects, each image being 128×128 pixels. Rotation angles {5°, 10°, ..., 360°} are applied to each image, resulting in a total of 72×20 = 1440 test images. The Olivetti dataset is a classic computer vision dataset containing 400 grayscale face images of 40 individuals. Each person has 10 images with different poses and expressions, including information such as whether they are wearing glasses, frontal view, and profile view. Each image is cropped to 64×64 pixels. The USPS dataset is a commonly used handwritten digit recognition dataset, provided by the United States Postal Service (USPS), containing handwritten digit images scanned from US mail. Each image is a 16×16 pixel grayscale image representing a handwritten digit (between 0 and 9), totaling 9298 images. In this experiment, 1000 images were randomly selected for testing. The Yale dataset contains facial images of 15 individuals, each with 11 different expressions and poses, totaling 165 images. Each image is grayscale and 320×243 pixels in size. Table 6 records detailed information for the four datasets. Figure 13 This section displays partial images from four image datasets. Principal component analysis was used to extract the principal components of the datasets before the experiment, with the cumulative contribution rate set to 90%.

[0188] Table 7 records the experimental results of different algorithms on various image datasets. From the NMI metric, SMDSC achieved the best results on Coil20, Olivetti, and Yale. Among the five spectral clustering algorithms compared, SP-k performed better. For the remaining methods, DP and the recently proposed DC method performed excellently. The ARI results are similar; SMDSC is consistently the best or second best among various methods. The average ranking results further confirm that SMDSC has certain advantages for complex real-world image data, achieving an average ranking of 2.1 across multiple comparison methods and test datasets.

[0189] Table 6. Basic information about the four image datasets

[0190]

[0191] Table 7 shows the clustering results of different clustering methods on four image datasets.

[0192]

[0193]

[0194] This invention provides a clustering method based on a Gaussian mixture model, comprising: acquiring raw data related to a target task, wherein the target task is any one of customer clustering, image clustering, text clustering, or bioinformatics clustering; dividing the raw data into several density regions based on a Gaussian mixture model, wherein the difference between different density regions is greater than a preset threshold; constructing an adjacency matrix for the density regions; and performing spectral clustering on the density regions according to the adjacency matrix to obtain the clustering result for the target task. Compared with existing technologies, this invention proposes a novel idea of ​​using a Gaussian mixture model for density stratification, thereby enabling the understanding of multi-density clustering distributions during the adjacency matrix construction process. Furthermore, it develops a new density ratio based on mutual nearest neighbors and density stratification, effectively reducing density imbalance among different instances in the clusters, improving practical applicability, and resulting in a more balanced adjacency matrix that better adapts to multi-density data. This allows for accurate clustering of raw data related to the target task, providing a solid foundation for subsequent processing of the target task.

[0195] Figure 14 This is a schematic diagram of a clustering device based on a Gaussian mixture model provided in an embodiment of the present invention, as shown below. Figure 14 As shown, it includes: a data acquisition module 141, a region division module 142, a matrix construction module 143, and a spectral clustering module 144.

[0196] The data acquisition module 141 is used to acquire raw data related to the target task, wherein the target task is any one of customer clustering task, image clustering task, text clustering task, and bioinformatics clustering task;

[0197] The region segmentation module 142 is used to divide the original data into several density regions based on the Gaussian mixture model, wherein the difference between different density regions is greater than a preset threshold.

[0198] Matrix construction module 143 is used to construct the adjacency matrix of the density region;

[0199] The spectral clustering module 144 is used to perform spectral clustering on density regions based on the adjacency matrix to obtain the clustering results for the target task.

[0200] Optionally, the region division module 142 is specifically used to fit the density of the original data based on the Gaussian mixture model to determine the density inflection data points; and to divide the original data into several density regions based on the density inflection data points.

[0201] Optionally, the region division module 142 is specifically used to generate an increasing density variable sequence of the original data, wherein the original data is sorted in ascending order of density in the increasing density variable sequence; and the increasing density variable sequence is fitted based on a Gaussian mixture model to determine the density inflection data points.

[0202] Optionally, the matrix construction module 143 is specifically used to cluster each density region based on the mutual nearest neighbor MNN relationship, determine the density ratio of each density region, and construct the adjacency matrix of the density region according to the density ratio of each density region.

[0203] Optionally, the spectral clustering module 144 is specifically used to construct a Laplacian matrix based on the adjacency matrix; and to perform spectral clustering on the density region based on the eigenvectors of the Laplacian matrix to obtain the clustering result of the target task.

[0204] Optionally, the spectral clustering module 144 is also used to generate processing information corresponding to the clustering results and send the processing information to the user terminal, wherein the processing information is used to guide the user to process different clustering results of the target task.

[0205] Optionally, the spectral clustering can be any one of the following: spectral clustering based on ω-NN, spectral clustering based on k-NN, spectral clustering based on Gaussian kernel, spectral clustering with self-adjusting kernel parameters, or spectral clustering based on common nearest neighbors.

[0206] The clustering device based on Gaussian mixture model provided in the embodiments of the present invention can execute the clustering method based on Gaussian mixture model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0207] This invention also provides an electronic device, including: a processor, which is configured to implement a clustering method based on a Gaussian mixture model having any of the features described in the above embodiments when executing a computer program.

[0208] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 15 As shown, the device includes a processor 20, a memory 21, an input device 22, and an output device 23; the number of processors 20 in the device can be one or more. Figure 15 Taking a processor 20 as an example; the processor 20, memory 21, input device 22, and output device 23 in the device can be connected via a bus or other means. Figure 15 Taking the bus connection as an example, a bus can refer to one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus that uses any of the various bus architectures.

[0209] The memory 21, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in this embodiment of the invention. The processor 20 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 21, thereby implementing the methods described above.

[0210] The memory 21 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on device usage. Furthermore, the memory 21 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory 21 may further include memory remotely located relative to the processor 20, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0211] Input device 22 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 23 may include display devices such as a display screen.

[0212] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a clustering method based on a Gaussian mixture model as provided in this invention. This method may specifically include, but is not limited to, the content disclosed in the above-described method embodiments.

[0213] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0214] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0215] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0216] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, Ruby, and Go, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0217] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0218] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0219] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0220] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0221] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A clustering method based on Gaussian mixture models, characterized in that, include: Obtain raw data related to the target task, wherein the target task is any one of customer clustering task, image clustering task, text clustering task, and bioinformatics clustering task; Based on the Gaussian mixture model, the original data is divided into several density regions, wherein the difference between different density regions is greater than a preset threshold. Construct the adjacency matrix of the density region; Based on the adjacency matrix, spectral clustering is performed on the density region to obtain the clustering result for the target task.

2. The clustering method based on Gaussian mixture model according to claim 1, characterized in that, The Gaussian mixture model divides the original data into several density regions, including: The density of the original data is fitted based on the Gaussian mixture model to determine the density inflection points. Based on the density inflection points, the original data is divided into several density regions.

3. The clustering method based on Gaussian mixture model according to claim 2, characterized in that, The process of fitting the density of the original data based on a Gaussian mixture model to determine density inflection points includes: Generate an increasing density variable sequence of the original data, wherein the original data is sorted in ascending order of density in the increasing density variable sequence; The increasing density variable sequence is fitted using a Gaussian mixture model to determine the density inflection points.

4. The clustering method based on Gaussian mixture model according to claim 1, characterized in that, Constructing the adjacency matrix of the density region includes: Based on the mutual nearest neighbor MNN relationship, each density region is clustered to determine the density ratio of each density region. Construct the adjacency matrix of the density regions based on the density ratio of each density region.

5. The clustering method based on Gaussian mixture model according to claim 1, characterized in that, The step of performing spectral clustering on the density region based on the adjacency matrix to obtain the clustering result for the target task includes: Construct the Laplace matrix based on the adjacency matrix; Based on the eigenvectors of the Laplacian matrix, spectral clustering is performed on the density region to obtain the clustering result for the target task.

6. The clustering method based on Gaussian mixture model according to claim 1, characterized in that, After obtaining the clustering results of the target task, the process also includes: Generate processing information corresponding to the clustering results; The processing information is sent to the user terminal, wherein the processing information is used to guide the user to process different clustering results of the target task.

7. The clustering method based on Gaussian mixture model according to claim 1 or 5, characterized in that, The spectral clustering can be any one of the following: -NN-based spectral clustering, k-NN-based spectral clustering, Gaussian kernel-based spectral clustering, kernel parameter self-adjustment spectral clustering, and common nearest neighbor-based spectral clustering.

8. A clustering device based on a Gaussian mixture model, characterized in that, include: The module includes a data acquisition module, a region partitioning module, a matrix construction module, and a spectral clustering module. The data acquisition module is used to acquire raw data related to the target task, wherein the target task is any one of customer clustering task, image clustering task, text clustering task, and bioinformatics clustering task; The region division module is used to divide the original data into several density regions based on a Gaussian mixture model, wherein the difference between different density regions is greater than a preset threshold. The matrix construction module is used to construct the adjacency matrix of the density region; The spectral clustering module is used to perform spectral clustering on the density region based on the adjacency matrix to obtain the clustering result of the target task.

9. An electronic device, characterized in that, include: A processor configured to implement the clustering method based on a Gaussian mixture model as described in any one of claims 1-7 when executing a computer program.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the clustering method based on the Gaussian mixture model as described in any one of claims 1-7.

Images