Index evaluation method and index evaluation system based on grouping stability
By generating sub-data through uniform downsampling and the k-means algorithm and calculating the clustering stability index, the problem that the evaluation indicators of the evaluation results cannot effectively reflect the stability of the clustering results, which cannot be solved in the existing technology, is solved. A reliable clustering result evaluation method and system is provided, which improves the credibility of the clustering results and the robustness of the decision.
Patent Information
- Application Number
- CN202510336437.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-03-21
- Filing Date
- 2025-03-21
- Publication Date
- 2025-09-23
Smart Images
Figure CN120687946A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an evaluation method and an evaluation system, and in particular to an indicator evaluation method and an indicator evaluation system based on clustering stability. Background Art
[0002] In data analysis, it is usually hoped that a stable indicator can be used to evaluate the rationality of a clustering result under various data randomness and changes.
[0003] See also Figure 8 As shown, for example, a raw data set 12 may contain multiple data points that are randomly distributed. When clustering the data points, data points with similar characteristics are assigned the same cluster label, and all data points are assigned their own cluster labels to divide the data points into multiple groups. For example, if the raw data set 12 needs to be divided into three groups, the data points in the left half are divided into two clusters, upper and lower, and the data points in the right half are grouped into the same cluster, thereby completing the clustering of the three groups.
[0004] However, the quality or rationality of the clustering results usually varies due to the subjective consciousness of the judge. Therefore, it is difficult to define the quality of the clustering results only by indicators. For example, Figure 8 and Figure 9 It is the clustering performance under two different clustering algorithms. Under the subjective judgment, it is usually considered Figure 9 The clustering performance is more suitable, but the existing index scores (silhouette coefficients) are evaluated as 0.50 and 0.26 respectively, which shows that Figure 8 The clustering performance is better under the indicator scores.
[0005] Since existing indicators cannot be directly applied to most situations, we hope to use a new clustering stability evaluation method to improve the above shortcomings. Summary of the Invention
[0006] In view of the above problems, the present invention provides an index evaluation method based on clustering stability and an evaluation system thereof, which are used to provide a clustering stability index for users to refer to in order to evaluate the stability or quality of the clustering results.
[0007] The index evaluation method based on clustering stability includes the following steps:
[0008] Uniformly downsample the original data to be clustered to generate array sub-data;
[0009] Calculating a plurality of similarities between the original data to be grouped and the set of sub-data based on at least one statistical test;
[0010] retaining the plurality of sub-data of the array whose similarities are greater than a similarity threshold as the sub-data of the array to be analyzed;
[0011] The array of sub-data to be analyzed is grouped according to a grouping algorithm to generate a plurality of sub-data grouping results;
[0012] Aggregating a plurality of clustering label models of the plurality of sub-data clustering results, and generating a plurality of aggregated sub-data clustering results according to the aggregated plurality of clustering label models;
[0013] A clustering stability index is calculated according to the clustering results of the plurality of aggregated sub-data.
[0014] The index evaluation system based on clustering stability includes a processing device. The processing device uniformly downsamples a raw data to be clustered to generate array sub-data, and the processing device calculates multiple similarities between the raw data to be clustered and the array sub-data based on at least one statistical test. The processing device retains the array sub-data with multiple similarities greater than a similarity threshold as array sub-data to be analyzed, and the processing device clusters the array sub-data to be analyzed based on a clustering algorithm to generate multiple sub-data clustering results. The processing device aggregates multiple clustering label combinations of the multiple sub-data clustering results, and generates multiple aggregated sub-data clustering results based on the aggregated multiple clustering label combinations, and the processing device calculates a clustering stability index based on the multiple aggregated sub-data clustering results.
[0015] This clustering stability-based index evaluation method and its index evaluation system provide a reference index for evaluating the stability of clustering results, thereby providing a reliable basis for decision-making. The clustering stability index can also provide a consistent reference index for users to evaluate between clustering results of different clustering methods, allowing users to rely on these clustering results with greater confidence to make decisions. In addition, the clustering stability index can also quantify uncertainty, which is crucial for risk management, decision-making, and system robustness. This allows users to be more cautious about the clustering results and helps reduce misleading results. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 Schematic diagram of the process of the index evaluation method based on clustering stability of the present invention.
[0017] Figure 2 Schematic diagram of uniform downsampling based on the clustering stability indicator evaluation method of the present invention.
[0018] Figure 3A 1 is a flow chart of a first embodiment of the pre-processing of the index evaluation method based on clustering stability of the present invention.
[0019] Figure 3B 4 is a flow chart of a second embodiment of the pre-processing of the index evaluation method based on clustering stability of the present invention.
[0020] Figure 4 This is another flow chart of the index evaluation method based on clustering stability of the present invention.
[0021] Figure 5 This is another flow chart of the index evaluation method based on clustering stability of the present invention.
[0022] Figure 6 Schematic diagram of the final clustering results of the clustering stability-based index evaluation method of the present invention.
[0023] Figure 7 Schematic diagram of the block diagram of the index evaluation system based on clustering stability of the present invention.
[0024] Figure 8 Schematic diagram of clustering original data using a clustering algorithm.
[0025] Figure 9 This is a schematic diagram of clustering the original data using another clustering algorithm. DETAILED DESCRIPTION
[0026] See also Figure 1 As shown, the present invention provides an index evaluation method based on clustering stability, comprising the following steps:
[0027] Step S13: uniformly downsample the original data to be grouped to generate array sub-data. Figure 2 As shown, the original data to be clustered 10 includes multiple data points 101, and multiple sub-data groups 10a-10j are generated through uniform downsampling. For example, uniform downsampling retains data points 101 at equal intervals and generates multiple sub-data groups 10a-10j using different sampling intervals. Therefore, the sampling points of each sub-data group 10a-10j are not completely the same, and the sampling points of each sub-data group 10a-10j are also not completely the same as the data points 101 of the original data to be clustered 10, thereby generating multiple sub-data groups 10a-10j that are variants of the original data to be clustered 10.
[0028] Step S14: Calculate multiple similarities between the original data 10 to be clustered and the array sub-data based on at least one statistical test. In this embodiment, the statistical test is a comprehensive evaluation of at least one test, including but not limited to a Chi-Squared Test, a Student's t-test, or an F-test.
[0029] Step S15: Retain the plurality of sub-data sets whose similarities are greater than a similarity threshold as the sub-data sets to be analyzed. For example, if the similarity threshold is 80%, it means that the data point 101 of the retained sub-data sets to be analyzed is more than 80% identical to the data point 101 of the original data 10 to be clustered.
[0030] Step S16: Cluster the sub-data to be analyzed in the array according to a clustering algorithm to generate a plurality of sub-data clustering results. In this embodiment, the clustering algorithm is a k-means clustering algorithm. For example, when selecting a clustering algorithm, the parameters of the selected clustering algorithm must also be set. For example, when the selected clustering algorithm is the k-means algorithm, the parameters of the number of clusters must also be set. For example, when n_cluster=3 is set, that is, when k-means clustering is set, all data points 101 must be divided into 3 groups. Figure 2 As shown, the multiple data points 101 of the original data to be clustered 10 are clustered into three groups using k-means, each represented by a different color, and the multiple sub-group data 10a~10j are also clustered into three groups using k-means, each represented by a different color.
[0031] Step S17 : Aggregating the plurality of clustering label models of the plurality of sub-data clustering results, and generating a plurality of aggregated sub-data clustering results according to the aggregated plurality of clustering label models.
[0032] Step S18: Calculate a clustering stability index based on the plurality of aggregated sub-data clustering results.
[0033] The index evaluation method based on clustering stability of the present invention provides a reference index, namely the clustering stability index, which can be used to evaluate the stability of the clustering results, thereby providing a reliable basis for decision-making. And the clustering stability index can allow users to evaluate the clustering results of running different clustering methods with a consistent reference index, which allows users to rely on these clustering results to make decisions with more confidence. In addition, the clustering stability index can also quantify uncertainty, which is crucial for risk management, decision-making and system robustness, which allows users to treat the clustering results more cautiously and helps reduce misleading results. For example, when the clustering stability index is lower, it represents that the uncertainty of this clustering result is higher, and vice versa, it represents that the uncertainty of this clustering result is lower.
[0034] In addition, before step S13, the following steps are further included:
[0035] Step S11: Receive original data.
[0036] Step S12: pre-process the original data to generate the original data to be grouped 10.
[0037] When receiving the original data, the present invention further organizes the original data through pre-processing to make the original data consistent and more easily executed by the next steps, thereby improving processing efficiency and accuracy.
[0038] See also Figure 3A As shown, in a first embodiment of the pre-processing, the original data includes at least one numerical feature or one text feature. Further, step S12 further includes the following sub-steps:
[0039] Step S121: Determine whether the original data set contains the text feature.
[0040] Step S122 : When the original data set contains the text feature, convert the text feature in the original data set into a converted numerical feature to generate the original data to be clustered 10 .
[0041] Step S123 : When the original data group does not contain the text feature, the original data group is directly used as the original data to be clustered 10 .
[0042] Because clustering algorithms require purely numerical input, if the raw data contains text features, these features will be converted, for example, using a One Hot Encoder. This conversion method is not limited to the One Hot Encoder; any preprocessing method that can convert text to numerical values, such as the Ordinal Encoder and Binary Encoder, can be used.
[0043] Also, see Figure 3B As shown, in a second embodiment of the pre-processing, the original data includes at least one numerical feature or one text feature. Further, step S12 further includes the following sub-steps:
[0044] Step S121 ′: determining whether the original data set contains the text feature.
[0045] Step S122': If the text feature is present in the original data set, the text feature in the original data set is converted into a converted numerical feature, and the converted numerical feature is normalized to generate the original data to be clustered 10. In this embodiment, the text feature is converted into the converted numerical feature using a one-hot encoder. The converted numerical feature and the numerical feature are normalized using a min-max standard scaling algorithm.
[0046] Step S123 ′: when the original data set does not contain the text feature, normalize the numerical feature of the original data set to generate the original data to be clustered 10 .
[0047] Because clustering methods based on Euclidean distance often favor features with larger scales when the scales of the numerical features differ significantly, this paper also uses the Min Max Standard Scale algorithm to normalize the multiple numerical features, converting them to a range of numerical values with the same scale. This normalization method is not limited to the Min Max Standard Scale algorithm; any preprocessing method that can normalize numerical features can be used, such as Standard Transformation and Box-Cox Transformation.
[0048] For further information, see Figure 4 As shown, step S17 further includes the following sub-steps:
[0049] Step S171 : Based on an original data clustering label model of the original data clustering result, overfitting the clustering label models of the plurality of sub-data clustering results is modeled and predicted by a decision tree classifier to generate the plurality of aggregated sub-data clustering results.
[0050] The present invention uses the decision tree to perfectly compare input (X) with output (Y) without controlling growth and pruning. Since the multiple sub-data clustering results will only generate the corresponding number of clusters when clustering, for example, the array of sub-data to be analyzed is divided into 3 groups, but the same cluster label model will not necessarily be generated. For example, the cluster label model of the first group of sub-data to be analyzed may be ABC, but the cluster label model of the second group of sub-data to be analyzed may be CBA. Therefore, the present invention aggregates the multiple cluster label models of the multiple sub-data clustering results, unifies the cluster label models of all sub-data clustering results, and avoids the situation of erroneous judgment due to different cluster label models.
[0051] For example, there are 10 groups of sub-data to be analyzed in the reserved array, and the reserved data points are shown in Table 1.
[0052]
[0053]
[0054] Table 1
[0055] The original data 10 to be clustered has data points ① to 10, sub-data 1 has data points ② to 10, sub-data 2 has data points ①, ③ to 10, sub-data 3 has data points ① to ②, ④ to 10, sub-data 4 has data points ① to ③, ⑤ to 10, sub-data 5 has data points ① to ④, ⑥ to 10, sub-data 6 has data points ① to ⑤, ⑦ to 10, sub-data 7 has data points ① to ⑥, ⑧ to 10, sub-data 8 has data points ① to ⑦, ⑨ to 10, sub-data 9 has data points ① to ⑧, ⑩, and sub-data 10 has data points ① to 9. Furthermore, the data points of sub-data 1 to 10, which are the sub-data to be analyzed in the array, are more than 80% identical to data point 101 of the original data 10 to be clustered.
[0056] For example, the aggregation of multiple clustering label models for the multiple sub-data clustering results is performed by the following steps, but is not limited thereto. Step 1: Establish a clustering label model (clustering label model 1), such as M1, for the original data 10 to be clustered. Step 2: Establish a clustering label model (clustering label model 11), such as M11, for sub-data 1. Step 3: Use clustering label model 1 (M1) to predict sub-data 1 and generate clustering result 1_1. Clustering result 1_1 is shown in Table 2 below.
[0057] Subdata 1 ② ③ ④ ⑤ ⑥ ⑦ ⑧ ⑨ ⑩ Grouping labels C C B B A A A A A
[0058] Table 2
[0059] Step 4: Use cluster label model 11 (M11) to predict sub-data 1 again, generating cluster result 11_1. Cluster result 11_1 is shown in Table 3 below.
[0060] Subdata 1 ② ③ ④ ⑤ ⑥ ⑦ ⑧ ⑨ ⑩ Grouping labels A A B B C C C C C
[0061] Table 3
[0062] Step 5: Use the clustering result 1_1 as the X (input) of the decision tree training, and use the clustering result 11_1 as the Y (output) of the decision tree training to train the decision tree 1. Step 6: Use the clustering label model 1 (M1) to predict the original data 10 to be clustered, and generate the clustering result 1_11.
[0063] The clustering results 1_11 are shown in Table 4 below.
[0064]
[0065] Table 4
[0066] In step 7, the trained decision tree 1 is used to predict the clustering result 1_11 and directly transform the clustering result 1_11 to obtain the clustering result 1_11_C. The clustering result 1_11_C is shown in Table 5 below.
[0067]
[0068] Table 5
[0069] Then, repeat the above steps ② to ⑦ to obtain the clustering results 2 to 10_11_C respectively. The clustering results 2 to 10_11_C are shown in Tables 6 to 14 below.
[0070]
[0071] Table 6 (Clustering results 2_11_C)
[0072]
[0073] Table 7 (Clustering results 3_11_C)
[0074]
[0075]
[0076] Table 8 (Clustering results 4_11_C)
[0077]
[0078] Table 9 (Clustering results 5_11_C)
[0079]
[0080] Table 10 (Clustering results 6_11_C)
[0081]
[0082] Table 11 (Clustering Results 7_11_C)
[0083]
[0084] Table 12 (Clustering results 8_11_C)
[0085]
[0086] Table 13 (Clustering results 9_11_C)
[0087]
[0088] Table 14 (Clustering results 10_11_C)
[0089] In addition, step S18 further includes the following sub-steps:
[0090] Step S181 : Calculating a plurality of clustering probabilities of a plurality of clustering labels of a plurality of data points 101 of the original data to be clustered 10 according to the plurality of aggregated sub-data clustering results.
[0091] Step S182: averaging the highest clustering probabilities of the plurality of data points 101 of the original data to be clustered 10 as the clustering stability indicator.
[0092] For example, after the aforementioned clustering results 1 to 10_11_C are obtained, the clustering probability of each clustering label of the data point 101 of the original data 10 to be clustered can be calculated based on the clustering labels of the multiple aggregated sub-data clustering results. For example, taking data point ① as an example, among the clustering results 1 to 10_11_C, there are 9 clustering results with the clustering label A, 1 clustering result with the clustering label B, and 0 clustering results with the clustering label C. Therefore, the clustering probability of data point ① with the clustering label A can be calculated as (9 / 10)% = 90%, the clustering probability of data point ① with the clustering label B can be calculated as (1 / 10)% = 10%, and the clustering probability of data point ① with the clustering label C can be calculated as (0 / 10)% = 0%. Similarly, the clustering probabilities of data points ② to 10 with the clustering labels A to C can be calculated respectively, as shown in Table 15 below.
[0093]
[0094] Table 15 Next, the cluster stability index is calculated according to the following formula:
[0095]
[0096] The above M is the number of data points 101 of the original data 10 to be clustered. For example, M=10. i ) is the highest clustering probability of each data point ①~⑩. Taking the above example, a1~a 10 The grouping stability index is 90%, 80%, 60%, 70%, 80%, 70%, 80%, 90%, 100%, and 100%. Therefore, the grouping stability index can be calculated according to the formula as follows:
[0097]
[0098] See also Figure 5 As shown, in this embodiment, step S16' further includes the following steps:
[0099] Step S16 ′: clustering the set of sub-data to be analyzed according to a clustering algorithm to generate a plurality of sub-data clustering results; and clustering the original data to be clustered 10 according to the clustering algorithm to generate an original data clustering result.
[0100] The index evaluation method based on cluster stability further includes the following steps:
[0101] Step S19: Generate a final clustering result based on the original data clustering result and the plurality of aggregated sub-data clustering results. In this embodiment, the final clustering result corresponds to the clustering stability index.
[0102] For example, the final clustering result is to use the highest probability among the clustering probabilities of data points ①-10 being assigned labels A-C as the cluster label for data points ①-10. For example, the probability of data point ① being assigned label A is 90%, the probability of being assigned label B is 10%, and the probability of being assigned label C is 0%. Therefore, the cluster label for data point ① can be determined as A. Similarly, the cluster labels for data points ②-10 can be determined sequentially to generate the final clustering result, as shown in Table 16 below.
[0103]
[0104] Table 16 (Final clustering results)
[0105] In addition, if the highest probability of a data point in the clustering probability is the same for more than two data points, the classification label of the original data clustering result shall prevail. Figure 6 As shown, if the data point The probability of label A is 50%, the probability of label B is 50%, and the probability of label C is 0%. It is impossible to decide based on the highest probability among the clustering probabilities. In this case, the classification label of the original data clustering result will be referred to. For example, the original data clustering result is for data point The classification label is B, and the final clustering result 11 for the data point The classification label is B.
[0106] In this embodiment, a higher number of the grouping stability index represents a higher stability of the final grouping result.
[0107] See also Figure 7As shown, the present invention also provides an index evaluation system based on clustering stability, which is used to execute the index evaluation method based on clustering stability. The index evaluation system based on clustering stability includes a processing device 20. The processing device 20 uniformly downsamples a raw data to be clustered to generate an array of sub-data, and the processing device 20 calculates multiple similarities between the raw data to be clustered and the array of sub-data based on at least one statistical test. The processing device 20 retains the array of sub-data with multiple similarities greater than a similarity threshold as the array of sub-data to be analyzed, and the processing device 20 clusters the array of sub-data to be analyzed based on a clustering algorithm to generate multiple sub-data clustering results. The processing device 20 aggregates multiple clustering label models of the multiple sub-data clustering results, and generates multiple aggregated sub-data clustering results based on the aggregated multiple clustering label models, and the processing device 20 calculates a clustering stability index based on the multiple aggregated sub-data clustering results.
[0108] Furthermore, the clustering stability-based index evaluation system further includes a data storage device 30. The data storage device 30 stores raw data. The processing device 20 is communicatively connected to the data storage device 30 to read the raw data and pre-process the raw data to generate the raw data to be clustered. In this embodiment, the processing device 20 is communicatively connected to the data storage device 30 via a network 40.
[0109] In one embodiment, the statistical test is a Chi-Squared Test, a Student's t-test, or an F-test. The clustering algorithm is a k-means clustering algorithm.
[0110] In a first embodiment of pre-processing, the raw data includes at least one numerical feature or one text feature. When pre-processing the raw data set, the processing device 20 determines whether the raw data set contains the text feature. If the raw data set contains the text feature, the processing device 20 converts the text feature in the raw data set into a converted numerical feature to generate the raw data set to be clustered. If the raw data set does not contain the text feature, the processing device 20 directly uses the raw data set as the raw data to be clustered.
[0111] In a second embodiment of pre-processing, the raw data includes at least one numerical feature or one text feature. When pre-processing the raw data set, the processing device 20 determines whether the raw data set contains the text feature. If the raw data set contains the text feature, the processing device 20 converts the text feature into a converted numerical feature and normalizes the converted numerical feature to generate the raw data to be clustered. If the raw data set does not contain the text feature, the processing device 20 normalizes the numerical feature of the raw data set to generate the raw data to be clustered.
[0112] In one embodiment, the processing device 20 converts the text feature into the converted numerical feature using a one-hot encoder and standardizes the converted numerical feature and the numerical feature using a min-max standard scaling algorithm.
[0113] When the processing device 20 aggregates the multiple cluster label models of the multiple sub-data clustering results and generates the multiple aggregated sub-data clustering results based on the aggregated multiple cluster label models, the processing device 20 performs overfitting modeling and prediction on the multiple cluster label models of the multiple sub-data clustering results through a decision tree classifier based on an original data clustering label model of the original data clustering results to generate the multiple aggregated sub-data clustering results.
[0114] When the processing device 20 calculates the clustering stability index based on the multiple aggregated sub-data clustering results, the processing device 20 calculates multiple clustering probabilities of multiple clustering labels of multiple data points of the original data to be clustered based on the multiple aggregated sub-data clustering results, and averages the multiple highest clustering probabilities of the multiple data points of the original data to be clustered as the clustering stability index.
[0115] When the processing device 20 clusters the plurality of arrays of sub-data to be analyzed according to the clustering algorithm to generate a plurality of sub-data clustering results, the processing device 20 also clusters the raw data to be clustered according to the clustering algorithm to generate a raw data clustering result. The processing device 20 generates a final clustering result based on the raw data clustering result and the plurality of aggregated sub-data clustering results. The final clustering result corresponds to the clustering stability indicator.
[0116] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any technician familiar with this profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.
Claims
1. A clustering stability-based index evaluation method comprising the following steps: Uniformly downsample the original data to be clustered to generate array sub-data; Calculating a plurality of similarities between the original data to be grouped and the set of sub-data based on at least one statistical test; retaining the plurality of sub-data of the array whose similarities are greater than a similarity threshold as the sub-data of the array to be analyzed; The array of sub-data to be analyzed is grouped according to a grouping algorithm to generate a plurality of sub-data grouping results; Aggregating a plurality of clustering label models of the plurality of sub-data clustering results, and generating a plurality of aggregated sub-data clustering results according to the aggregated plurality of clustering label models; A clustering stability index is calculated according to the clustering results of the plurality of aggregated sub-data.
2. The index evaluation method based on clustering stability according to claim 1, wherein: Before uniformly downsampling the original data to be grouped to generate the array sub-data, the following steps are further included: receiving raw data; The original data is pre-processed to generate the original data to be grouped.
3. The index evaluation method based on clustering stability according to claim 2, characterized in that: The original data includes at least one numerical feature or one text feature; The step of pre-processing the raw data set includes the following sub-steps: Determine whether the text feature exists in the original data group; When the original data set contains the text feature, converting the text feature in the original data set into a converted numerical feature to generate the original data to be grouped; When the original data group does not contain the text feature, the original data group is directly used as the original data to be grouped.
4. The index evaluation method based on clustering stability according to claim 2, characterized in that: The original data includes at least one numerical feature or one text feature; The step of pre-processing the raw data set includes the following sub-steps: Determine whether the text feature exists in the original data group; When the original data set contains the text feature, converting the text feature in the original data set into a converted numerical feature, and normalizing the converted numerical feature to generate the original data to be grouped; When the original data set does not contain the text feature, the numerical feature of the original data set is normalized to generate the original data to be grouped.
5. The index evaluation method based on clustering stability according to claim 1, characterized in that: The step of aggregating the plurality of clustering label models of the plurality of sub-data clustering results and generating the plurality of aggregated sub-data clustering results according to the aggregated plurality of clustering label models includes the following sub-steps: Based on an original data clustering label model of the original data clustering result, a decision tree classifier is used to overfit the clustering label models of the plurality of sub-data clustering results to generate the plurality of aggregated sub-data clustering results.
6. The index evaluation method based on clustering stability according to claim 1, characterized in that: The step of calculating the clustering stability index based on the plurality of aggregated sub-data clustering results includes the following sub-steps: Calculating a plurality of clustering probabilities of a plurality of clustering labels of a plurality of data points of the original data to be clustered according to the plurality of aggregated sub-data clustering results; The average of the multiple highest clustering probabilities of the multiple data points of the original data to be clustered is used as the clustering stability indicator.
7. The index evaluation method based on clustering stability according to claim 1, characterized in that: The step of clustering the array of sub-data to be analyzed according to the clustering algorithm to generate a plurality of sub-data clustering results further includes the following steps: Clustering the original data to be clustered according to the clustering algorithm to generate an original data clustering result; The clustering stability-based index evaluation method further comprises the following steps: Generating a final clustering result according to the original data clustering result and the plurality of aggregated sub-data clustering results; The final clustering result corresponds to the clustering stability index.
8. An index evaluation system based on clustering stability, characterized in that: Includes: a processing device for uniformly downsampling the original data to be grouped to generate array sub-data; The processing device calculates a plurality of similarities between the original data to be grouped and the array sub-data based on at least one statistical test; The processing device retains the plurality of sub-data sets having similarities greater than a similarity threshold as the sub-data sets to be analyzed, and the processing device clusters the sub-data sets to be analyzed according to a clustering algorithm to generate a plurality of sub-data clustering results; The processing device aggregates the plurality of cluster label models of the plurality of sub-data clustering results and generates a plurality of aggregated sub-data clustering results according to the aggregated plurality of cluster label models. The processing device also calculates a clustering stability index according to the plurality of aggregated sub-data clustering results.
9. The index evaluation system based on clustering stability according to claim 8, characterized in that: Further included are: A data storage device storing original data; The processing device is communicatively connected to the data storage device to read the original data and pre-process the original data to generate the original data to be grouped.
10. The index evaluation system based on clustering stability according to claim 9, characterized in that: The original data includes at least one numerical feature or one text feature; Wherein, the processing device determines whether the text feature exists in the original data group when pre-processing the original data group; When the original data set contains the text feature, the processing device converts the text feature in the original data set into a converted numerical feature to generate the original data set to be grouped; When the original data group does not contain the text feature, the processing device directly uses the original data group as the original data to be grouped.
11. The index evaluation system based on clustering stability according to claim 9, characterized in that: The original data includes at least one numerical feature or one text feature; Wherein, the processing device determines whether the text feature exists in the original data group when pre-processing the original data group; When the original data set contains the text feature, the processing device converts the text feature in the original data set into a converted numerical feature and standardizes the converted numerical feature to generate the original data to be grouped; When the original data set does not contain the text feature, the processing device normalizes the numerical feature of the original data set to generate the original data to be grouped.
12. The index evaluation system based on clustering stability according to claim 8, characterized in that: When the processing device aggregates the multiple cluster label models of the multiple sub-data clustering results and generates the multiple aggregated sub-data clustering results based on the aggregated multiple cluster label models, the processing device performs overfitting modeling and prediction on the multiple cluster label models of the multiple sub-data clustering results through a decision tree classifier based on an original data clustering label model of the original data clustering results to generate the multiple aggregated sub-data clustering results.
13. The index evaluation system based on clustering stability according to claim 8, characterized in that: When the processing device calculates the clustering stability index based on the multiple aggregated sub-data clustering results, the processing device calculates multiple clustering probabilities of multiple clustering labels of multiple data points of the original data to be clustered based on the multiple aggregated sub-data clustering results, and averages multiple highest clustering probabilities of the multiple data points of the original data to be clustered as the clustering stability index.
14. The index evaluation system based on clustering stability according to claim 8, characterized in that: When the processing device groups the set of sub-data to be analyzed according to the grouping algorithm to generate a plurality of sub-data grouping results, the processing device groups the original data to be grouped according to the grouping algorithm to generate an original data grouping result; The processing device generates a final grouping result based on the original data grouping result and the plurality of aggregated sub-data grouping results; The final clustering result corresponds to the clustering stability index.