Method, device and electronic equipment for classifying service data
By dividing the data into two parts for clustering and classification voting, and using centroid mapping and set conditions to fuse the results, the high computational cost and accuracy problems of clustering algorithms under big data are solved, achieving efficient and accurate clustering results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA CONSTRUCTION BANK
- Filing Date
- 2023-06-15
- Publication Date
- 2026-06-05
Smart Images
Figure CN116680615B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of data processing technology, and in particular to a method, apparatus and electronic device for classifying business data. Background Technology
[0002] With the advent of the big data era, the amount of data in various industries is enormous, making it difficult to analyze the massive amounts of data in each industry and to mine and find useful information from databases. Currently, data classification is usually carried out using unsupervised learning clustering methods, but due to the large amount of data, the clustering computation cost is high and it is difficult to evaluate the clustering effect. Summary of the Invention
[0003] This disclosure aims to at least partially address one of the technical problems in the related art.
[0004] The first aspect of this disclosure proposes a method for classifying business data, comprising: grouping all business data into a first data set and a second data set, wherein the first data set includes multiple data groups; clustering the first business data in each data group of the first data set to obtain one or more clusters for each data group; obtaining the centroids of each cluster in each data group, and performing classification voting on the second business data in the second data set based on the centroids of the clusters to obtain the cluster to which the second business data belongs; and fusing the clustering results corresponding to the first data set and the classification voting results corresponding to the second data set when the classification voting results of the second data set meet the set conditions to obtain the target clustering result of the all business data.
[0005] A second aspect of this disclosure provides a business data classification apparatus, comprising a grouping module for grouping all business data to obtain a first data set and a second data set, wherein the first data set includes multiple data groups; a clustering module for clustering first business data in each data group of the first data set to obtain one or more clusters for each data group; a voting module for obtaining the centroids of each cluster in each data group and, based on the centroids of the clusters, performing classification voting on second business data in the second data set to obtain the cluster to which the second business data belongs; and a classification module for fusing the clustering results corresponding to the first data set and the classification voting results corresponding to the second data set when the classification voting results of the second data set meet the set conditions, to obtain the target clustering result of the all business data.
[0006] A third aspect of this disclosure provides an electronic device, comprising:
[0007] At least one processor; and
[0008] A memory that is communicatively connected to at least one processor; wherein,
[0009] The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform the business data classification method provided in the first aspect of this disclosure.
[0010] A fourth aspect of this disclosure provides a computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are used to cause a computer to execute a classification method for business data provided in a first aspect of this disclosure.
[0011] A fifth aspect of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the business data classification method provided in the first aspect of this disclosure.
[0012] This application can solve the failure problem of clustering algorithms when processing large amounts of data, reduce the amount of computation in the clustering process, and evaluate the effect of clustering to ensure the accuracy of clustering results. Attached Figure Description
[0013] Figure 1 This is a flowchart of a business data classification method provided in an embodiment of this disclosure;
[0014] Figure 2 This is a flowchart of another method for classifying business data provided in this embodiment of the disclosure;
[0015] Figure 3 This is a flowchart of another method for classifying business data provided in this embodiment of the disclosure;
[0016] Figure 4 This is a flowchart of another method for classifying business data provided in this embodiment of the disclosure;
[0017] Figure 5 This is a structural block diagram of a business data classification device provided in an embodiment of this disclosure;
[0018] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation
[0019] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.
[0020] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0021] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.
[0022] The following description, in conjunction with the accompanying drawings, describes a method, apparatus, and electronic device for classifying business data according to embodiments of the present disclosure.
[0023] Figure 1 This is a flowchart of a business data classification method provided in an embodiment of this disclosure, such as... Figure 1 As shown, the method includes the following steps:
[0024] S101, group all business data to obtain a first data set and a second data set, wherein the first data set includes multiple data groups.
[0025] The embodiments disclosed herein are applicable to scenarios in various industries where business data is classified and analyzed, such as healthcare, government affairs, telecommunications, insurance, education, and power.
[0026] Given the massive volume of business data, the entire dataset can be grouped for analysis. Optionally, the entire dataset can be grouped evenly, or it can be grouped according to a specific ratio to obtain a first dataset and a second dataset.
[0027] Furthermore, all business data in the first data set is divided into multiple data groups. To facilitate comparative analysis of each data group, the amount of business data in each data group is the same.
[0028] For example, assuming the total amount of business data is 100,000, the 100,000 can be divided into a first data set and a second data set, meaning that each of the first and second data sets contains 50,000 business data. The business data in the first data set can be divided into multiple data groups. For example, if the 50,000 business data in the first data set is divided into 5 groups, then each data group contains 10,000 business data.
[0029] S102, cluster the first business data in each data group in the first data set to obtain one or more clusters for each data group.
[0030] Optionally, the method for clustering the first business data in each data group of the first data set can be K-means clustering or common clustering algorithms such as DBSCAN clustering.
[0031] S103, obtain the centroid of each cluster in each data group, and based on the centroid of the cluster, classify and vote on the second business data in the second data set to obtain the cluster to which the second business data belongs.
[0032] In some implementations, the centroid of each cluster in each data group is obtained, and the centroid represents its corresponding cluster; for example, if the first business data corresponding to the centroid is relatively good, then all the first business data in the cluster to which the centroid is located are relatively good.
[0033] It's understandable that the centroid is a representation of the clustering result; the mean of all data points in a cluster is usually called the centroid of that cluster. For example, if business data is mapped to points in a two-dimensional plane, then the x-coordinate of the centroid of the business data in a cluster is the mean of the x-coordinates of all points representing the business data in the cluster, and the y-coordinate of the centroid is the mean of the y-coordinates of all points representing the business data in the cluster.
[0034] Optionally, in each data group, the distance between each second business data and the centroid of each cluster in the data group can be calculated. The smaller the distance between the second business data and the centroid of the cluster, the more similar the characteristics of the second business data are to the characteristics of the first business data in the cluster to which the centroid belongs, and the more likely the second business data belongs to the cluster to which the centroid belongs.
[0035] In some implementations, the cluster corresponding to the second business data can be determined as the cluster containing the centroid that is closest to the second business data. In other words, the second business data can be assigned a corresponding cluster in each data group.
[0036] Optionally, a second voting process can be conducted for the clusters corresponding to the second business data in each data group. The distance between the centroid of the corresponding cluster in each selected data group and the second business data can be compared, and the cluster containing the centroid with the smallest distance can be taken as the cluster to which the second business data belongs. After obtaining the clusters to which all the second business data belongs, the classification voting results corresponding to the second data set are obtained.
[0037] For example, assuming there are 5 data groups, the distance between the second business data and the centroid of each type of cluster in each data group is calculated, and the cluster corresponding to the second business data is determined in the 5 data groups; further, the distance between the centroid of the 5 selected clusters and the second business data is compared again, and the cluster containing the centroid with the smallest distance is taken as the cluster to which the second business data belongs.
[0038] S104, if the classification voting results of the second data set meet the set conditions, the clustering results corresponding to the first data set and the classification voting results corresponding to the second data set are merged to obtain the target clustering result of the full business data.
[0039] Optionally, the conditions can be set to whether the voting results of the categories are balanced, or whether the number of successful votes in the category voting results is greater than a preset number.
[0040] For example, whether the classification voting results are balanced can be determined based on the distance between the second business data and the centroid of each cluster. For instance, for any cluster, the distance between all the second business data within that cluster and the centroid of the current cluster is calculated to determine the average distance. If the average distance is less than a set threshold, it indicates that the average distance is small, meaning that all the second business data within the cluster are relatively close to the centroid of the current cluster. When the average distance is less than the set threshold, the classification voting results are deemed to meet the set conditions.
[0041] For example, whether the success rate of the classification voting results is greater than the preset ratio can also be determined based on the distance between the second business data and the centroid of its respective cluster. A distance threshold is set. If the distance between the second business data and the centroid of its respective cluster is less than the distance threshold, it is recorded as a successful vote; conversely, if the distance between the second business data and the centroid of its respective cluster is not less than the distance threshold, it is recorded as a failed vote. The number of successful votes is obtained by counting all the successful votes of the second business data in the second data set. When the number of successful votes is greater than the preset number, it indicates that there are more successful votes of the second business data, and it is determined that the classification voting results of the second data set meet the set conditions.
[0042] Furthermore, the clustering results of the first dataset and the classification voting results of the second dataset are merged, that is, the clusters to which the first business data belongs and the clusters to which the second business data belongs are merged. This fusion is to combine the results of the clusters to which the first business data and the second business data belong to determine the business data in each cluster, and finally obtain the target clustering result of the full set of business data.
[0043] In this embodiment, the entire set of business data is grouped into two datasets for analysis. All business data in the first dataset are clustered to obtain multiple clusters, and the centroid of each cluster is obtained. Based on the centroids of each second business data item in the second dataset and each cluster, the cluster to which the second business data belongs is determined, thus obtaining the final target clustering result. In this embodiment, the clusters of the first business data in the first dataset are used as the classification basis to avoid the clustering algorithm failing with large amounts of data and reduce the computational load of the clustering process. Based on the centroids of each cluster, the second business data undergoes a classification vote to determine its corresponding cluster, resulting in better classification performance. Furthermore, the clustering results of the first dataset and the classification vote results of the second dataset are merged based on whether the classification vote results meet set conditions, ensuring the accuracy of the final target clustering result.
[0044] Figure 2 A flowchart of another business data classification method provided in this disclosure embodiment is shown below. Figure 2 As shown, the method includes the following steps:
[0045] S201, group all business data to obtain a first data set and a second data set, wherein the first data set includes multiple data groups.
[0046] Optionally, the full volume of business data is obtained by preprocessing the full volume of initial business data. The preprocessing may include data preprocessing and feature engineering.
[0047] In some implementations, data preprocessing of initial business data may include: outlier and numerical truncation, missing value handling, and normalization.
[0048] In some implementations, feature engineering processing of preprocessed initial business data can include: discrete feature construction, continuous feature construction, temporal feature construction, spatial feature construction, and text feature construction, etc.
[0049] In this embodiment of the disclosure, the method for implementing step S201 can be implemented in any of the ways described in the various embodiments of the disclosure. This is not limited here and will not be described in detail.
[0050] S202, cluster the first business data in each data group in the first data set to obtain one or more clusters for each data group.
[0051] Optionally, in this embodiment of the disclosure, each data group includes the same clusters.
[0052] In this embodiment of the disclosure, the method for implementing step S202 can be implemented in any of the ways described in the various embodiments of the disclosure. This is not limited here and will not be described in detail.
[0053] S203, obtain the centroid of each cluster in each data group.
[0054] After clustering the data sets to obtain one or more clusters, the centroid of each cluster in each data set is obtained. Optionally, the centroid is generally the mean of all first business data in the cluster.
[0055] S204, for each data group, obtain the distance between the centroids of the second business data and the various clusters of the data group.
[0056] Since each data group includes one or more clusters, and each cluster corresponds to a centroid, when determining the cluster to which the second business data belongs, the distance between the second business data and the centroids of each cluster in the data group can be used to determine this.
[0057] The smaller the distance between the second business data and the centroid of a certain cluster in the data group, the more similar the second business data is to that centroid, and the more likely the second business data belongs to that cluster.
[0058] S205, determine the cluster with the smallest distance within the data group, and use it as a candidate cluster for mapping the second business data to the data group.
[0059] Since the smaller the distance between the second business data and the centroid of the cluster in the data group, the more likely the second business data belongs to the cluster of that centroid, the cluster with the smallest distance to the second business data in the data group is determined as a candidate cluster of the second business data.
[0060] It is understandable that the first data set includes multiple data groups, and the second business data determines a candidate cluster from each data group. Therefore, the first data set includes multiple candidate clusters of the second business data.
[0061] S206, vote on the candidate clusters of all data groups, and obtain the target cluster to which the second business data belongs based on the voting results.
[0062] Optionally, each candidate cluster in the data group can be distinguished. For example, the candidate clusters in the data group can be arranged from largest to smallest according to their centroids, and the arranged candidate clusters can be labeled. The labels can be numbers such as 1, 2, 3, 4, etc. Each label can represent the excellent, good, average, and poor labels of the business data.
[0063] After determining the candidate clusters for each data group, the corresponding labels for the candidate clusters can be obtained; for example, labels such as "excellent" or "good" can be determined. A vote is then taken among the candidate clusters for all data groups; in this embodiment, the vote can be considered as the number of times a candidate cluster is selected.
[0064] Optionally, for candidate cluster i, obtain the number of votes mapped to candidate cluster i in all data groups, where i is a positive integer, 1≤i≤J, and J is the maximum number of clusters; if the number of votes for candidate cluster i is greater than the first set threshold, determine that the target cluster to which the second business data belongs is candidate cluster i with a number of votes greater than the first set threshold.
[0065] Optionally, the first set threshold can be determined by the maximum number of clusters. To ensure the reliability of cluster determination, a ratio between 50% and 90% of the maximum number of clusters can be used. For example, the first set threshold can be set to 80% of the maximum number of clusters. When the maximum number of clusters is 100, the first set threshold is 80.
[0066] For example, suppose there are 5 data groups, each containing 4 clusters, and each cluster is labeled with a number from 1 to 4. If the candidate clusters for the second business data across all data groups are determined to be "1, 1, 1, 2, 1", then the number of votes for candidate cluster 1 is 4. If the number of votes for candidate cluster 1 (4) is greater than a first preset threshold, then the target cluster to which the second business data belongs is determined to be candidate cluster 1. Optionally, if no candidate cluster has a number of votes greater than the first preset threshold, a cross-group comparison is performed on the distance between the second business data and the centroids of all clusters in all data groups to obtain the cluster with the smallest distance to the second business data among all centroids, which is then taken as the target cluster to which the second business data belongs.
[0067] For example, assuming that the number of votes for each candidate cluster in the 5 data groups is no greater than the first set threshold, the distance between the second business data and the centroids of all clusters in all data groups is compared to determine the centroid with the smallest distance to the second business data. The cluster corresponding to the centroid with the smallest distance to the second business data is the target cluster to which the second business data belongs.
[0068] S207, if the classification voting results of the second data set meet the set conditions, the clustering results corresponding to the first data set and the classification voting results corresponding to the second data set are merged to obtain the target clustering result of the full business data.
[0069] In this embodiment, the process of determining the category to which the second business data belongs from multiple candidate categories is recorded as a vote. If the voting result of the target category to which the second business data belongs is greater than a first preset threshold, the number of valid votes is incremented by 1; if the voting result of the target category to which the second business data belongs is not greater than the first preset threshold, the number of valid votes remains unchanged. In other words, when determining the target category based on candidate categories, only the process of determining the target category when the voting result is greater than the first preset threshold is a valid voting process. Therefore, all the second business data in the second data set are statistically analyzed to determine the number of valid votes in the second data set.
[0070] The classification voting results of the second data set are based on the number of valid votes in the second data set and the cluster to which the second business data belongs. The larger the number of valid votes, the more second business data belong to the target cluster determined when the number of votes exceeds the first set threshold. The result of the target cluster to which the second business data belongs is more reliable and has stronger reference value.
[0071] Optionally, the set condition can be the voting success rate. That is, when the number of valid votes in the classification voting results of the second dataset is greater than the voting success rate, the classification voting results of the second dataset are determined to meet the set condition.
[0072] Furthermore, after determining the target clustering result for all business data, labels for each business data point can be generated based on the cluster to which it belongs; and sample business data can be generated based on the business data and its labels. In other words, after determining the target clustering result for all business data, the corresponding label for each business data point is obtained, and sample business data is generated based on the business data and its labels.
[0073] In this embodiment, the centroid of each cluster in the data group is obtained. Based on the distance between the second business data and each centroid, candidate clusters of the second business data are determined in each data group, avoiding the error of analyzing a single data group. The number of votes for the same candidate cluster in each data group is counted. The larger the number of votes, the more likely the second business data belongs to that candidate cluster. Therefore, candidate clusters whose voting data meets the first preset threshold are used as target clusters of the second business data, making the classification of the second business data more accurate. When the voting data is not greater than the first preset threshold, the cluster with the smallest centroid distance to the second business data among all clusters in the data group is used as the target cluster of the second business data. In this way, all the second business data in the second data set are classified, which not only ensures the consistency of the clustering results, but also avoids the problem of clustering failure due to excessively large data.
[0074] Figure 3 A flowchart of another business data classification method provided in this disclosure embodiment is shown below. Figure 3 As shown, the method includes the following steps:
[0075] S301, group all business data to obtain a first data set and a second data set, wherein the first data set includes multiple data groups.
[0076] In this embodiment of the disclosure, the method for implementing step S301 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0077] S302, cluster the first business data in each data group in the first data set to obtain one or more clusters for each data group.
[0078] In this embodiment of the disclosure, the method for implementing step S302 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0079] S303, obtain the centroid of each cluster in each data group, and based on the centroid of the cluster, classify and vote on the second business data in the second data set to obtain the cluster to which the second business data belongs.
[0080] In this embodiment of the disclosure, the method for implementing step S303 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0081] S304, obtain the total number of second business data in the second data set.
[0082] Optionally, the total number of all second business data in the second data set can be counted.
[0083] S305, iterate through the second business data in the second data set, and at the end of the iteration, count the number of second business data that meet the condition that the number of votes is greater than the first set threshold.
[0084] Optionally, the second business data in the second data set is traversed to determine the number of votes corresponding to each second business data. This number of votes is obtained from the candidate clusters corresponding to the second business data.
[0085] Furthermore, a vote that is greater than the first set threshold can be considered a successful vote. The number of second business data that meets the condition of having a vote greater than the first set threshold is counted, which is the number of second business data that are successfully voted.
[0086] S306, obtain the ratio of the number to the total number, as the voting consensus rate of the second data set.
[0087] Optionally, the ratio between the number of second business data that meets the requirement of having more than one set threshold of votes and the total number of all second business data in the second data set is calculated. This ratio represents the proportion of second business data that has successfully received votes in the second data set, and is used as the voting consistency rate of the second data set.
[0088] A higher vote consistency rate indicates a greater number of second business data sets that meet the first set threshold for vote count, meaning a greater number of second business data sets that have successfully voted. This makes the division of the target clusters of all second business data sets in the second data set more reliable.
[0089] S307, if the voting consensus rate is greater than or equal to the second set threshold, determine that the classification voting results of the second data set meet the set conditions.
[0090] Optionally, when the voting consensus rate is greater than or equal to the second set threshold, it indicates that there are a large number of second business data that have been successfully voted on, and therefore the classification voting results of the second data set are determined to meet the set conditions.
[0091] S308 summarizes the business data belonging to the same cluster to obtain all the business data included in the same cluster, and obtains the target clustering result of the full business data.
[0092] After determining that the classification voting results of the second data set meet the set conditions, the business data belonging to the same cluster are aggregated, that is, the first business data and the second business data belonging to the same cluster are aggregated to obtain all the business data included in each cluster; and then the target clustering result of the full business data is determined.
[0093] Optionally, if the voting consensus rate is less than the second set threshold, it indicates that the number of second business data that were successfully voted is small, and the result of this vote is not very reliable. The full set of business data can be re-divided, and the above steps can be performed on the re-divided first and second sets of data until the classification voting result of the second set of data meets the set conditions, and the target clustering result of the full set of business data is obtained.
[0094] In this embodiment, after determining the cluster to which the second business data belongs based on the number of votes, the total number of all second business data in the second data set and the number of second business data with a vote count greater than a first preset threshold are counted. The ratio of the number of second business data with a vote count greater than the first preset threshold to the total number of all second business data in the second data set is calculated as the voting consistency rate. The higher the voting consistency rate, the more second business data there are with a vote count greater than the first preset threshold, and the more reliable the cluster to which the second business data belongs based on the vote count. Therefore, the higher the voting consistency rate, the more reliable the classification of the second business data in the second data set. By judging whether the voting consistency rate is greater than or equal to the second preset threshold, the reliability of the classification result of the second data set is determined. The more reliable classification result of the second data set is then summarized and merged with the clustering result of the first data set, resulting in a more reliable and accurate target clustering result for all business data, ensuring the effectiveness of clustering.
[0095] Figure 4 A flowchart of another business data classification method provided in this disclosure embodiment is shown below. Figure 4 As shown, the method includes the following steps:
[0096] S401, group all business data to obtain a first data set and a second data set, wherein the first data set includes multiple data groups.
[0097] In this embodiment of the disclosure, the method for implementing step S401 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0098] S402, cluster the first business data in each data group in the first data set to obtain one or more clusters for each data group.
[0099] In this embodiment of the disclosure, the method for implementing step S402 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0100] S403, obtain the centroid of each cluster in each data group.
[0101] In this embodiment of the disclosure, the method for implementing step S403 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0102] S404, for each data group, obtain the distance between the second business data and the centroids of various clusters in the data group.
[0103] In this embodiment of the disclosure, the method for implementing step S404 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0104] S405, determine the cluster with the smallest distance within the data group, and use it as a candidate cluster for mapping the second business data to the data group.
[0105] In this embodiment of the disclosure, the method for implementing step S405 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0106] S406: Vote for the candidate clusters of all data groups, and obtain the target cluster to which the second business data belongs based on the voting results.
[0107] In this embodiment of the disclosure, the method for implementing step S406 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0108] S407, obtain the total number of second business data in the second data set.
[0109] In this embodiment of the disclosure, the method for implementing step S407 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0110] S408, traverse the second business data in the second data set, and at the end of the traversal, count the number of second business data that meet the condition that the number of votes is greater than the first set threshold.
[0111] In this embodiment of the disclosure, the method for implementing step S408 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0112] S409, obtain the ratio of the number to the total number, as the voting consensus rate of the second data set.
[0113] In this embodiment of the disclosure, the method for implementing step S409 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0114] S410, if the voting consensus rate is greater than or equal to the second set threshold, determine that the classification voting results of the second data set meet the set conditions.
[0115] In this embodiment of the disclosure, the method for implementing step S410 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0116] S411, summarize the business data belonging to the same cluster to obtain all the business data included in the same cluster, and obtain the target clustering result of the full business data.
[0117] In this embodiment of the disclosure, the method for implementing step S411 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.
[0118] In this embodiment, the full amount of business data is divided into a first data set and a second data set for analysis. Business data in each data group within the first data set is clustered, avoiding the problem of clustering algorithm failure under large data volumes. Furthermore, the centroid of each cluster in the data group is obtained. Based on the distance between the second business data and each centroid, candidate clusters for the second business data are determined in each data group, avoiding errors from analyzing a single data group. The number of votes for the same candidate cluster in each data group is counted; the larger the number of votes, the more likely the second business data belongs to that candidate cluster. Therefore, the voting data satisfies a first preset condition. The candidate clusters of the threshold are used as the target clusters for the second business data, resulting in more accurate classification of the second business data. After determining the target clusters for the second business data, the total number of all second business data in the second data set and the number of second business data with a vote count greater than the first set threshold are counted. The ratio of the number of second business data with a vote count greater than the first set threshold to the total number of all second business data in the second data set is calculated as the voting consistency rate. The higher the voting consistency rate, the more second business data there are with a vote count greater than the first set threshold, and the more reliable the classification of the second business data in the second data set. By judging whether the voting consistency rate is greater than or equal to the second set threshold, the reliability of the classification result of the second data set is determined. The more reliable classification result of the second data set is then summarized and merged with the clustering result of the first data set to obtain a more reliable and accurate target clustering result for the entire business data, ensuring the effectiveness of clustering.
[0119] Based on the above embodiments, this disclosure can be used by banks and other enterprises to determine the marketing capabilities of their employees. It involves dividing all employee-related business data into a first data set and a second data set, further dividing the business data in the first data set into multiple data groups, clustering the business data in each data group, and labeling each cluster based on the average value of the clustering results. For example, a numerical label can be used to represent the business data of that cluster as excellent, good, or poor. Using the centroid of each cluster in the first data set as a representative, the distance between each second business data point in the second data set and the centroid of each cluster is calculated. The smaller the distance, the more similar the second business data is to the first business data in the cluster containing that centroid. The cluster corresponding to the centroid with the smallest distance in each data group is selected as a candidate cluster for the second business data. The number of votes for candidate clusters under the same label is counted across all data groups. If the number of votes is greater than a first set threshold, the candidate cluster under that label is the target cluster for the second business data. If the number of votes is not greater than the first set threshold, the cluster corresponding to the centroid with the smallest distance in all data groups is the target cluster for the second business data.
[0120] Furthermore, the number of second business data items in the second dataset with a vote count greater than the first set threshold is counted. The ratio between the number of second business data items with a vote count greater than the first set threshold and the total number of second business data items in the second dataset is calculated as the voting consistency rate. The validity of the current classification of business data is determined based on the voting consistency rate. If the voting consistency rate is greater than or equal to the second set threshold, the classification is valid. The clustering results of the first dataset and the classification voting results of the second dataset are then summarized and merged to obtain the target clustering result. If the voting consistency rate is less than the second set threshold, the current classification is invalid. The entire set of business data is then re-divided into the first and second datasets, and the above operation is performed on the re-divided first and second datasets until the target clustering result is obtained. This ensures a good clustering effect, and the cluster to which each employee's business data belongs is determined based on the target clustering result. The marketing ability of employees is then determined based on the cluster labels, resulting in higher accuracy.
[0121] Figure 5 A structural block diagram of a business data classification device provided in this disclosure embodiment is shown below. Figure 5 As shown, the business data classification device 500 of this embodiment includes:
[0122] Grouping module 501 is used to group the full amount of business data to obtain a first data set and a second data set, wherein the first data set includes multiple data groups;
[0123] Clustering module 502 is used to cluster the first business data in each data group in the first data set to obtain one or more clusters for each data group.
[0124] The voting module 503 is used to obtain the centroid of each cluster in each data group, and based on the centroid of the cluster, to classify and vote on the second business data in the second data set to obtain the cluster to which the second business data belongs;
[0125] The classification module 504 is used to merge the clustering results corresponding to the first data set and the classification voting results corresponding to the second data set when the classification voting results of the second data set meet the set conditions, so as to obtain the target clustering result of the full business data.
[0126] In some implementations, each data group includes the same cluster of classes, where voting module 503 includes:
[0127] For each data group, obtain the distance between the second business data and the centroids of various clusters in the data group;
[0128] The cluster with the smallest distance within the data group is identified and used as a candidate cluster for mapping the second business data to the data group.
[0129] A vote is taken on the candidate clusters for all data groups, and the target cluster to which the second business data belongs is determined based on the voting results.
[0130] In some implementations, the voting module 503 includes:
[0131] For a candidate cluster i, obtain the number of votes mapped to candidate cluster i in all data sets, where i is a positive integer, 1≤i≤J, and J is the maximum number of clusters;
[0132] If the number of votes for candidate cluster i is greater than the first set threshold, the target cluster to which the second business data belongs is determined to be candidate cluster i, whose number of votes is greater than the first set threshold.
[0133] In some implementations, the voting module 503 also includes:
[0134] If there is no candidate cluster with more than the first set threshold, the distance between the centroids of the second business data and all clusters in all data groups is compared across groups to obtain the cluster with the smallest distance to the second business data among all centroids, which is taken as the target cluster to which the second business data belongs.
[0135] In some implementations, device 500 also includes:
[0136] Get the total number of second business data in the second data set;
[0137] The second business data in the second data set is traversed, and at the end of the traversal, the number of second business data that meet the requirement of having more than the first set threshold number of votes is counted.
[0138] The ratio of the obtained quantity to the total quantity is used as the voting consensus rate of the second data set;
[0139] If the voting consensus rate is greater than or equal to the second set threshold, the classification voting results of the second data set are determined to meet the set conditions.
[0140] In some implementations, the classification module 504 includes:
[0141] By aggregating business data belonging to the same cluster, we obtain all business data included in the same cluster, thus obtaining the target clustering result of the full set of business data.
[0142] In some implementations, after the classification module 504, the following also includes:
[0143] Generate tags for each business data based on the cluster to which it belongs;
[0144] Sample business data is generated based on business data and its tags.
[0145] In some implementations, prior to grouping module 501, the following also includes:
[0146] Preprocessing and feature engineering are performed on all initial business data to obtain full business data.
[0147] In this embodiment, the entire set of business data is grouped into two datasets for analysis. All business data in the first dataset are clustered to obtain multiple clusters, and the centroid of each cluster is obtained. Based on the centroids of each second business data item in the second dataset and each cluster, the cluster to which the second business data belongs is determined, thus obtaining the final target clustering result. In this embodiment, the clusters of the first business data in the first dataset are used as the classification basis to avoid the clustering algorithm failing with large amounts of data and reduce the computational load of the clustering process. Based on the centroids of each cluster, the second business data undergoes a classification vote to determine its corresponding cluster, resulting in better classification performance. Furthermore, the clustering results of the first dataset and the classification vote results of the second dataset are merged based on whether the classification vote results meet set conditions, ensuring the accuracy of the final target clustering result.
[0148] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Figure 6 As shown, the above-mentioned electronic device 600 includes:
[0149] The system includes a memory 610 and a processor 620, and a bus 630 connecting different components (including the memory 610 and the processor 620). The memory 610 stores a computer program, which, when executed by the processor 620, implements the business data classification method described in this embodiment of the present disclosure.
[0150] Bus 630 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.
[0151] Electronic device 600 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by electronic device 600, including volatile and non-volatile media, removable and non-removable media.
[0152] Memory 610 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 640 and / or cache memory 650. Electronic device 600 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 660 can be used to read and write non-removable, non-volatile magnetic media (… Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 630 via one or more data media interfaces. Memory 610 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.
[0153] A program / utility 680 having a set (at least one) of program modules 670 may be stored in, for example, memory 610. Such program modules 670 include—but are not limited to—an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 670 typically perform the functions and / or methods described in the embodiments of this disclosure.
[0154] Electronic device 600 can also communicate with one or more external devices 690 (e.g., keyboard, pointing device, display 691, etc.), and with one or more devices that enable a user to interact with the electronic device 600, and / or with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 692. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 693. Figure 6 As shown, network adapter 693 communicates with other modules of electronic device 600 via bus 630. It should be understood that, although... Figure 6 As not shown in the diagram, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0155] The processor 620 executes various functional applications and data processing by running programs stored in the memory 610.
[0156] It should be noted that the implementation process and technical principles of the electronic device in this embodiment are explained in the foregoing description of the classification method of business data in the embodiments of this disclosure, and will not be repeated here.
[0157] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium.
[0158] When the instructions in the computer-readable storage medium are executed by the processor of the business server, the business server is able to perform the business data classification method as described above. Optionally, the computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0159] To implement the above embodiments, this disclosure also provides a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the business data classification method as described above.
[0160] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims. It should be understood that this disclosure is not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A method for classifying business data, characterized in that, The method includes: The entire business data is grouped to obtain a first data set and a second data set, wherein the first data set includes multiple data groups; Cluster the first business data in each data group in the first data set to obtain one or more clusters for each data group; Obtain the centroid of each cluster in each data group. For each data group, obtain the distance between the second business data and the centroid of each cluster in the data group. Determine the cluster with the smallest distance in the data group as a candidate cluster to which the second business data is mapped. Vote for the candidate clusters in all data groups. Based on the voting results, obtain the target cluster to which the second business data belongs. When the classification voting results of the second data set meet the set conditions, the clustering results corresponding to the first data set and the classification voting results corresponding to the second data set are fused to obtain the target clustering result of the full business data.
2. The method according to claim 1, characterized in that, Each of the data groups includes the same clusters.
3. The method according to claim 2, characterized in that, The step of voting on the candidate clusters of all data groups and obtaining the target cluster to which the second business data belongs based on the voting results includes: For candidate clusters i Obtain all data groups mapped to the candidate clusters. i The number of votes, wherein the i It is a positive integer, 1≤ i ≤J, the J The maximum number of clusters; If the candidate cluster i If the number of votes is greater than a first preset threshold, the target cluster to which the second business data belongs is determined as a candidate cluster whose number of votes is greater than the first preset threshold. i .
4. The method according to claim 3, characterized in that, The step of voting on the candidate clusters of all data groups and obtaining the target cluster to which the second business data belongs based on the voting results further includes: If there is no candidate cluster whose number of votes is greater than the first set threshold, the distance between the centroids of the second business data and all data groups is compared across groups to obtain the cluster with the smallest distance to the second business data among all centroids, which is taken as the target cluster to which the second business data belongs.
5. The method according to claim 3, characterized in that, The method further includes: Obtain the total number of the second business data in the second data set; The second business data in the second data set is traversed, and at the end of the traversal, the number of second business data that satisfy the requirement that the number of votes is greater than the first set threshold is counted. The ratio of the stated quantity to the total quantity is used as the voting consensus rate of the second data set; If the voting consensus rate is greater than or equal to the second set threshold, it is determined that the classification voting results of the second data set meet the set conditions.
6. The method according to claim 5, characterized in that, The step of fusing the clustering results corresponding to the first data set and the classification voting results corresponding to the second data set to obtain the target clustering result of the full set of business data includes: By summarizing business data belonging to the same cluster, we obtain all business data included in the same cluster, and thus obtain the target clustering result of the full set of business data.
7. The method according to any one of claims 1-6, characterized in that, After obtaining the target clustering result of the full set of business data, the process further includes: Generate tags for the business data based on the cluster to which each business data belongs; Sample business data is generated based on the business data and the tags of the business data.
8. The method according to any one of claims 1-6, characterized in that, Before grouping all business data to obtain the first data set and the second data set, the process also includes: The full set of initial business data is preprocessed and feature-engineered to obtain the full set of business data.
9. A business data classification device, characterized in that, include: The grouping module is used to group the full amount of business data to obtain a first data set and a second data set, wherein the first data set includes multiple data groups; The clustering module is used to cluster the first business data in each data group in the first data set to obtain one or more clusters for each data group. The voting module is used to obtain the centroid of each cluster in each data group, obtain the distance between the second business data and the centroid of each cluster in each data group, determine the cluster with the smallest distance in the data group as a candidate cluster to which the second business data is mapped, and vote on the candidate clusters of all data groups to obtain the target cluster to which the second business data belongs based on the voting results. The classification module is used to merge the clustering results corresponding to the first data set and the classification voting results corresponding to the second data set when the classification voting results of the second data set meet the set conditions, so as to obtain the target clustering result of the full business data.
10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-8.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.
12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-8.