Data clustering method and device, equipment, storage medium and product

Abstract

The invention discloses a data clustering method and device, equipment, a storage medium and a product, and relates to the technical field of data processing, and the data clustering method comprises the steps: obtaining to-be-clustered initial data; dividing the initial data to obtain a plurality of first data groups; and for any first data group, dividing the first data group into at least one first cluster, and performing clustering processing on the first cluster to obtain a cluster optimization result. According to the invention, the processing efficiency of data clustering is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data clustering method, device, equipment, storage medium and computer program product. Background Art

[0002] In the field of data mining and analysis, it is often necessary to perform cluster analysis on large-scale data sets. Currently, mainstream clustering algorithms such as DBSCAN (density-based clustering algorithm) are often used to perform cluster analysis on data. However, due to the high time complexity of the algorithm, when processing large-scale data sets, the processing efficiency will be low.

[0003] Therefore, how to improve the processing efficiency of data clustering is a problem that needs to be solved urgently.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide an invention name, aiming to solve the technical problems corresponding to the background technology.

[0006] To achieve the above purpose, the present application proposes a data clustering method, which includes:

[0007] Obtain the initial data to be clustered;

[0008] Dividing the initial data to obtain a plurality of first data groups;

[0009] For any first data group, the first data group is divided into at least one first cluster, and clustering processing is performed on the first cluster to obtain an optimized cluster result.

[0010] In one embodiment, the initial data includes at least one initial group, and the step of dividing the initial data to obtain a plurality of first data groups includes:

[0011] Randomly extracting first data from the initial data, and using the first data as current processing data;

[0012] Determining a first distance between the currently processed data and each of the initial groups;

[0013] Based on the first distance, the currently processed data is divided into initial groups until the initial data is divided into a plurality of first data groups.

[0014] In one embodiment, the step of dividing the currently processed data into initial groups based on the first distance includes:

[0015] Comparing the first distance with a preset distance threshold to obtain a comparison result;

[0016] If the comparison result shows that there is no initial group whose first distance is less than or equal to the preset distance threshold, dividing the currently processed data into a first data group;

[0017] If the comparison result shows that there is an initial group whose first distance is less than or equal to the preset distance threshold, the currently processed data is divided into the initial group corresponding to the minimum value of the first distance.

[0018] In one embodiment, after the step of dividing the initial data to obtain a plurality of first data groups, the method further includes:

[0019] If the amount of data in the current first data group is less than a preset amount threshold, determining that the current first data group is a noise data group, and filtering the noise data group;

[0020] If the amount of data in the current first data group is greater than or equal to a preset amount threshold, calculating the mean of all data in the current first data group, and taking the mean as the center point of the current first data group;

[0021] Each of the first data groups is filtered based on a second distance between the central point and each data in each of the first data groups.

[0022] In one embodiment, the step of filtering each of the first data groups based on the second distance between the center point and each data in each of the first data groups further includes:

[0023] Calculate the second distance between each data and the center point;

[0024] Based on the second distance, a distance mean and a distance error are calculated;

[0025] Based on the distance mean and the distance error, a distance ratio is calculated;

[0026] If the distance ratio is less than a preset ratio threshold, the first data group corresponding to the distance ratio is determined to be a noise data group, and the noise data group is filtered; otherwise, the current first data group is not filtered.

[0027] In one embodiment, the step of dividing each of the first data packets into a plurality of first clusters comprises:

[0028] Calculating a weight value of each data in each of the first data groups;

[0029] Based on the arrangement order of the weight values, determining the second data of the number of elements and the preset cluster number threshold in order from large to small;

[0030] Integrate each of the second data into a first data set;

[0031] Determine connected data sets in the first data set, and merge the connected data sets to obtain a second data set;

[0032] For each of the first data groups, a plurality of first clusters are obtained based on the intersection of the first data set and the second data set.

[0033] In one embodiment, the step of performing clustering processing on the first cluster to obtain an optimized clustering result includes:

[0034] Calculate the mean of each data in the first cluster class to obtain the cluster center;

[0035] Calculate the third distance between the remaining second cluster classes and the cluster center;

[0036] If the third distance is less than a preset clustering radius threshold, merging the second cluster corresponding to the third distance with the current first cluster to obtain a merging result;

[0037] Based on the merging result, an optimized clustering result is determined.

[0038] In addition, to achieve the above purpose, the present application also proposes a data clustering device, which includes:

[0039] An acquisition module, used to obtain initial data to be clustered;

[0040] A division module, used for dividing the initial data to obtain a plurality of first data groups;

[0041] The cluster processing module is used to divide any first data group into at least one first cluster, and perform cluster processing on the first cluster to obtain an optimized cluster result.

[0042] In addition, to achieve the above objectives, the present application also proposes a data clustering device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data clustering method described above.

[0043] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data clustering method described above are implemented.

[0044] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the data clustering method described above are implemented.

[0045] The present application proposes a data clustering method, apparatus, device, storage medium and computer program product. The present application obtains initial data to be clustered and divides the initial data to obtain multiple first data groups, and divides each first data group into multiple first cluster classes, thereby clustering each first cluster class, and transforming the clustering processing of the first data groups into the clustering processing of the clusters. By reducing the calculation amount of the group to the calculation amount of the cluster, the time complexity is reduced, thereby improving the processing efficiency of clustering analysis of the data.

[0046] In summary, the present application divides the initial data to be clustered into multiple first data groups, divides each first data group into multiple first cluster classes, and then performs clustering processing on each first cluster class, thereby converting the data clustering method from clustering processing of groups to clustering processing of clusters, reducing the calculation amount of the group to the calculation amount of the cluster, thereby reducing the time complexity, and further improving the processing efficiency of clustering analysis of the data to solve the corresponding technical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 A schematic diagram of a process flow provided for the first embodiment of the data clustering method of this application;

[0050] Figure 2 This is a schematic diagram of the overall execution process involved in the data clustering method of this application;

[0051] Figure 3A brief flowchart of the data clustering method provided in Example 2 of the present application;

[0052] Figure 4 This is a schematic diagram of the module structure of the data clustering device according to an embodiment of the present application;

[0053] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the data clustering method in the embodiment of the present application.

[0054] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0055] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0056] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0057] The main solution of the embodiment of the present application is: obtaining initial data to be clustered;

[0058] Dividing the initial data to obtain a plurality of first data groups;

[0059] For any first data group, the first data group is divided into at least one first cluster, and clustering processing is performed on the first cluster to obtain an optimized cluster result.

[0060] In this embodiment, for the convenience of description, the following description is made by taking the data clustering device as the execution subject.

[0061] Although the clustering algorithms used in the prior art, such as K-means, are simple and easy to use, they may fall into local optimal solutions in some cases, resulting in low quality of clustering results.

[0062] The DBSCAN algorithm has been improved in some aspects, but there are still some problems when processing large-scale data sets. For example, the DBSCAN algorithm has a high time complexity and low processing efficiency for large data sets, which may not be applicable in some special application scenarios.

[0063] The present application provides a data clustering method, apparatus, device, storage medium and computer program product. The present application divides the initial data to be clustered into multiple clusters, reduces the calculation amount of the group to the calculation amount of the cluster, thereby reducing the time complexity, and further improving the processing efficiency of clustering analysis of the data.

[0064] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a data clustering device, etc. The following takes a data clustering device as an example to illustrate this embodiment and the following embodiments.

[0065] Based on this, the present application embodiment provides a data clustering method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the data clustering method of the present application.

[0066] In this embodiment, the data clustering method includes steps S10 to S30:

[0067] Step S10, obtaining initial data to be clustered;

[0068] It should be noted that the initial data is the data obtained without any processing. The initial data can be business data, financial data, communication data, etc., without specific limitation.

[0069] It should be noted that the purpose of clustering the initial data is to integrate data with similar or identical characteristics together, so as to facilitate in-depth mining of the characteristics of the initial data.

[0070] Step S20, dividing the initial data to obtain a plurality of first data groups;

[0071] It should be noted that when clustering is performed on the initial data, it is necessary to first divide the data into a plurality of first data groups, each of which contains one or more data or elements with similar characteristics.

[0072] It should be noted that the first data group includes multiple data, wherein for each first data group, the number of elements included may be the same or different, and is not specifically limited.

[0073] In a feasible implementation manner, the initial data includes at least one initial group, and the step S20 of dividing the initial data to obtain a plurality of first data groups includes:

[0074] Step S21, randomly extracting first data from the initial data, and using the first data as current processing data;

[0075] It should be noted that the first data is data randomly extracted from all the initial data, and the distance between the currently processed data and other initial groups is calculated based on the extracted first data.

[0076] It should be noted that the initial group may contain at least one element or data, but the data in each initial group is chaotic and disordered. In this embodiment, an initial group including a single data is used for explanation.

[0077] Step S22, determining a first distance between the currently processed data and each of the initial groups;

[0078] It should be noted that the first distance is the Euclidean distance between data, wherein the Euclidean distance may be calculated as follows:

[0079] Calculate the mean of all data in group i and use the mean as the center of group i.

[0080] 2) Calculate the Euclidean distance between each data in group i and the center

[0081] Where j is the identifier of the data in group i. is the Euclidean distance between the jth data in group i and the center.

[0082] If the center point is o i , then the Euclidean distance Among them, k is the data dimension, is the value of the kth dimension of the jth data in group i, is the value of the kth dimension of the center point of group i.

[0083] For example, multiple groups have been formed. For any group, if it currently includes multiple data, the distance between the currently processed data and each included data is calculated, and the distance here is the existing Euclidean distance. Then the average of the distances of the multiple data is used as the distance between the currently processed data and the group.

[0084] Step S23: dividing the currently processed data into initial groups based on the first distance, until the initial data is divided into a plurality of first data groups.

[0085] It should be noted that, by comparing the first distance with the preset distance threshold, it is determined according to the comparison result whether the currently processed data is divided into the initial group or as a new first data group.

[0086] It should be noted that after the current processing data is divided, data continues to be extracted from the remaining initial data as the current processing data until all the data to be clustered are executed, at which time multiple first data groups that have been divided can be obtained.

[0087] In a feasible implementation manner, the step S23 of dividing the currently processed data into initial groups based on the first distance includes:

[0088] Comparing the first distance with a preset distance threshold to obtain a comparison result;

[0089] It should be noted that the preset distance threshold may be the maximum effective value of the distance between two data or between a single data and the first data group, wherein the effective distance value for any group is σ is the mean of the sum of the distances between all the data in any group, and μ is the standard deviation of the sum of the distances between all the data in any group. The preset distance threshold may also be an empirical value set by the user, and is not specifically limited.

[0090] It should be noted that the comparison result may be that the first distance is less than, equal to, or greater than a preset distance threshold.

[0091] If the comparison result shows that there is no initial group whose first distance is less than or equal to the preset distance threshold, dividing the currently processed data into a first data group;

[0092] It should be noted that if there is no initial group whose first distance is less than or equal to the preset distance threshold, it means that there is no group with similar characteristics to the currently processed data in the existing initial groups, and the currently processed data is divided into separate first data groups.

[0093] If the comparison result shows that there is an initial group whose first distance is less than or equal to the preset distance threshold, the currently processed data is divided into the initial group corresponding to the minimum value of the first distance.

[0094] It should be noted that when there is an initial group whose first distance is less than or equal to the preset distance threshold, the initial group corresponding to the minimum value of the first distance is selected, and the currently processed data is divided into the initial group until all the initial data are traversed and the first data group is also divided.

[0095] Step S30: for any first data group, divide the first data group into at least one first cluster, and perform clustering processing on the first cluster to obtain an optimized clustering result.

[0096] It should be noted that the first data packet is divided into a plurality of first cluster classes, wherein each first cluster class represents a cluster, and each first data packet can be divided into one or more clusters.

[0097] In a feasible implementation manner, the step S30 of performing clustering processing on the first cluster to obtain an optimized clustering result includes steps S31 to S34:

[0098] Step S31, calculating the mean of each data in the current first cluster class to obtain the cluster center;

[0099] It should be noted that, for any cluster y in group i, the cluster center of the cluster y is calculated, wherein the cluster center can be obtained by calculating the mean of each data in the first cluster class.

[0100] Step S32, calculating the third distance between the remaining second clusters and the cluster center;

[0101] It should be noted that the second cluster class is the remaining cluster classes in the first data group except the first cluster class.

[0102] It should be noted that the third distance represents the distance between clusters, more precisely, the distance between cluster centers. The third distance can be represented by dqy.

[0103] Like other clusters q, the distance to cluster y is dqy = the distance between the center of cluster q and the center of cluster y - (the maximum distance between the center of cluster q and each data of cluster q + the maximum distance between the center of cluster y and each data of cluster y).

[0104] Step S33: if the third distance is less than a preset clustering radius threshold, merging the second cluster corresponding to the third distance with the current first cluster to obtain a merging result;

[0105] It should be noted that the preset clustering radius threshold is the value of the radius of the cluster. When the distance between two clusters is less than the radius threshold, it means that the two clusters intersect or overlap, that is, the two clusters have the possibility of being merged, and the corresponding second cluster class can be merged with the current first cluster class to obtain a merged cluster.

[0106] Step S34: determining an optimized clustering result based on the merging result.

[0107] It should be noted that the merging result is the merged cluster. This operation is performed for each cluster until each cluster is merged, and the clustering processing of the initial data is completed.

[0108] The present application proposes a data clustering method, apparatus, device, storage medium and computer program product. The present application obtains initial data to be clustered and divides the initial data to obtain multiple first data groups, and divides each first data group into multiple first cluster classes, thereby clustering each first cluster class, and transforming the clustering processing of the first data groups into the clustering processing of the clusters. By reducing the calculation amount of the group to the calculation amount of the cluster, the time complexity is reduced, thereby improving the processing efficiency of clustering analysis of the data.

[0109] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 In order to filter the noise data in the first data group, after step S20, the data clustering method further includes steps S201 to S203:

[0110] Step S201, if the amount of data in the current first data group is less than a preset amount threshold, determining that the current first data group is a noise data group, and filtering the noise data group;

[0111] It should be noted that when the amount of data in the first data group is less than the preset threshold, it means that the amount of data in the current first data group is too small, and it may be a noise data group containing interference data. This group needs to be filtered out to ensure the accuracy of the data clustering result.

[0112] It should be noted that the preset quantity threshold may be 5, 10, etc., and may be changed according to user needs without specific limitation.

[0113] Step S202, if the amount of data in the current first data group is greater than or equal to a preset amount threshold, then calculating the mean of all data in the current first data group, and taking the mean as the center point of the current first data group;

[0114] It should be noted that, when the amount of data in the first data group is greater than or equal to a preset amount threshold, the mean of all data in the current first data group is calculated, and the mean is used as the center point of the group.

[0115] Step S203: filtering each of the first data groups based on the second distance between the center point and each data in each of the first data groups.

[0116] It should be noted that, according to the second distance between the center point of the first data group and other data, it is determined whether the current first data group needs to be filtered as a noise data group.

[0117] It should be noted that when determining whether each first data group contains noise data, two determination methods are required. The first is to use the number of data in each data group. When the number of data is less than a preset value, the group can be identified as noise data. The second is to determine whether it is a noise data group by calculating the distance between the center point of the data group and other data. By filtering noise data groups in two ways, the accuracy of data clustering can be guaranteed.

[0118] In a feasible implementation manner, the step S203 of filtering each of the first data groups based on the second distance between the center point and each data in each of the first data groups further includes steps A1 to A4:

[0119] Step A1, calculating the second distance between each data and the center point;

[0120] Step A2, based on the second distance, calculating a distance mean and a distance error;

[0121] It should be noted that the second distance is the same as the Euclidean distance mentioned above. The specific steps are:

[0122] 1) Calculate the mean of all data in the first data group i, and use the mean as the center of group i.

[0123] 2) Calculate the Euclidean distance between each data in the first data group i and the center

[0124] Wherein j is the identifier of the data in the first data group i. is the Euclidean distance between the jth data in the first data group i and the center.

[0125] If the center point is o i , then the Euclidean distance Among them, k is the data dimension, is the value of the kth dimension of the jth data in the first data group i, is the value of the kth dimension of the center point of the first data group i.

[0126] 3) Calculate the mean distance and distance error

[0127] Wherein J is the total number of data in the first data group i.

[0128] Step A3, calculating a distance ratio based on the distance mean and the distance error;

[0129] It should be noted that the calculated distance ratio = distance less than or equal to d i *(1+c i ) / J.

[0130] Step A4: if the distance ratio is less than a preset ratio threshold, determining that the first data group corresponding to the distance ratio is a noise data group, and filtering the noise data group; otherwise, not filtering the current first data group.

[0131] It should be noted that if the distance ratio is less than the preset ratio threshold, it means that the data features in the first data group i are relatively scattered and are noise data, which is filtered out. Otherwise, it means that the data in the first data group i is normal and is retained as the final group.

[0132] In this embodiment, a noise data filtering operation is performed on the first data group after the division, thereby eliminating interference items in the first data group and ensuring the accuracy of subsequent clustering data.

[0133] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first and second embodiments can be referred to the above description and will not be described in detail later. On this basis, the step S30 of dividing each of the first data packets into a plurality of first clusters includes:

[0134] Step S301, selecting currently processed data from each initial data, and calculating the weight value of each data in each first data group;

[0135] It should be noted that the weight value can be calculated as follows:

[0136] (1) For any data J in the first data group, calculate the Euclidean distance between other data in the first data group and data J Among them, k is the identifier of other data in group i, is the Euclidean distance between data j and data k in group i.

[0137] (2) Calculate the number of data with a distance less than R

[0138] (3) Calculate the weight of data j

[0139] Step S302, based on the arrangement order of the weight values, determining second data whose data distance is less than a preset clustering radius threshold in order from large to small, wherein the data distance is used to represent the distance between the currently processed data and other data in the first data group;

[0140] It should be noted that the second data whose data distance is greater than the preset clustering radius threshold is determined according to the order of weight values ​​from large to small.

[0141] It should be noted that the data distance is generally Euclidean distance, Minkowski distance, etc., and is not specifically limited.

[0142] Step S303, when the number of corresponding second data is greater than or equal to a preset cluster number threshold, integrating each second data into a first data set;

[0143] It should be noted that the currently selected data Is it greater than the preset cluster number threshold? If not, select the next data and repeat the process until all data are selected.

[0144] 2) If If it is greater than the preset cluster number threshold, All corresponding data form a set A, where set A is the first data set.

[0145] That is to say, the data in set A are all data whose distance to data j is less than R, and the number of elements in set A is greater than the preset cluster number threshold.

[0146] Step S304, determining connected data sets in the first data set, and merging the connected data sets to obtain a second data set;

[0147] It should be noted that for each data v in set A, the connected data set C of the data is obtained. v , the connected data set C of all data in set A v Merge to get the final second data set.

[0148] (1) The data in group i whose distance to data v is less than min{R, X} form a set B.

[0149] Where X = (the maximum value of the distance from the center of group i to each data of group i - R) / 2.

[0150] (2) Calculate the distance between data v and other data in set B

[0151] (3) Determine the number of data with a distance less than R

[0152] (4) If the number determined in (3) is not greater than D (the preset cluster number threshold), execute the next data in set A. If the number is greater than D, form set C with the data in set B whose distance is less than R (the preset cluster radius threshold). v .

[0153] Step S305 : for each of the first data groups, divide the data into a plurality of first clusters based on the intersection of the first data set and the second data set.

[0154] It should be noted that the intersection of the first data set and the second data set is determined as a first cluster, and the next data is selected to repeat the above steps until all the data are selected.

[0155] In this embodiment, the weights between data points are used to determine the cluster division, and a weight clustering algorithm is used for cluster division. The weight clustering algorithm is a weight-based clustering algorithm. Its basic idea is to calculate the weight of each data point according to the distance between the data points, and then divide the data points into multiple clusters according to the weights.

[0156] For example, to help understand the implementation process of the data clustering method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 2 , Figure 2 A brief flow chart of a data clustering method is provided, specifically:

[0157] By initially dividing the data to be clustered, filtering all the divided groups to obtain the final groups, and clustering each final group, and then optimizing and adjusting each completed cluster, the optimized clustering result is obtained. The data after clustering is analyzed according to the optimized clustering result. The analysis here is the existing analysis, which is an analysis plan determined according to the purpose of data analysis. If you need to find the characteristics of the data, you can perform feature analysis on the clustering results.

[0158] In addition, since clusters are further divided in the grouping in this step, each cluster in the same group not only has the same intra-group characteristics, but also has its own cluster characteristics. The prior art only performs grouping, which results in high time complexity due to the large amount of data in the group. However, the present application further subdivides the grouping, thereby reducing the calculation amount of the group to the calculation amount of the cluster, reducing the time complexity and thus improving the efficiency of cluster analysis.

[0159] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the data clustering method of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0160] This application also provides a data clustering device, please refer to Figure 4 , the data clustering device comprises:

[0161] An acquisition module 10 is used to acquire initial data to be clustered;

[0162] A division module 20, configured to divide the initial data into multiple first data groups;

[0163] The clustering processing module 30 is used to divide any first data group into at least one first cluster, and perform clustering processing on the first cluster to obtain an optimized clustering result.

[0164] Optionally, the division module includes:

[0165] An extraction unit, used for randomly extracting first data from the initial data, and using the first data as current processing data;

[0166] A first determining unit, configured to determine a first distance between the currently processed data and each of the initial groups;

[0167] The first dividing unit is used to divide the currently processed data into initial groups based on the first distance until the initial data is divided into a plurality of first data groups.

[0168] Optionally, the dividing unit includes:

[0169] A comparison subunit, configured to compare the first distance with a preset distance threshold to obtain a comparison result;

[0170] A first division subunit is configured to divide the currently processed data into a first data group if the comparison result shows that there is no initial group whose first distance is less than or equal to a preset distance threshold;

[0171] The second division subunit is configured to divide the currently processed data into the initial group corresponding to the minimum value of the first distance if the comparison result shows that there is an initial group whose first distance is less than or equal to the preset distance threshold.

[0172] Optionally, the device further comprises:

[0173] A determination module, configured to determine that the current first data group is a noise data group if the amount of data in the current first data group is less than a preset amount threshold, and filter the noise data group;

[0174] a calculation module, configured to calculate a mean value of all data in the current first data group if the amount of data in the current first data group is greater than or equal to a preset amount threshold, and use the mean value as the center point of the current first data group;

[0175] A filtering module is used to filter each of the first data groups based on the second distance between the center point and each data in each of the first data groups.

[0176] Optionally, the filtering module includes:

[0177] A calculation unit, used for calculating a second distance between each data and the center point;

[0178] A second calculation unit, configured to calculate a distance mean and a distance error based on the second distance;

[0179] A third calculation unit, configured to calculate a distance ratio based on the distance mean and the distance error;

[0180] The second determining unit is configured to determine that the first data group corresponding to the distance ratio is a noise data group and filter the noise data group if the distance ratio is less than a preset ratio threshold, and otherwise, not filter the current first data group.

[0181] Optionally, the clustering processing module includes:

[0182] a fourth calculation unit, configured to select currently processed data from each initial data, and calculate a weight value of each data in each of the first data groups;

[0183] A determination unit, configured to determine, based on the arrangement order of the weight values, second data whose data distance is less than a preset clustering radius threshold in order from large to small, wherein the data distance is used to characterize the distance between the currently processed data and other data in the first data group;

[0184] An integration unit, configured to integrate each of the second data into a first data set when the number of corresponding second data is greater than or equal to a preset cluster number threshold;

[0185] A first merging unit, configured to determine connected data sets in the first data set, and merge the connected data sets to obtain a second data set;

[0186] The second division unit is used to divide each of the first data groups into a plurality of first clusters based on the intersection of the first data set and the second data set.

[0187] Optionally, the clustering processing module further includes:

[0188] A fifth calculation unit, used to calculate the mean of each data in the current first cluster class to obtain a cluster center;

[0189] a sixth calculation unit, configured to calculate a third distance between the remaining second clusters and the cluster center;

[0190] A second merging unit, configured to merge the second cluster corresponding to the third distance with the current first cluster to obtain a merging result if the third distance is less than a preset clustering radius threshold;

[0191] The third determining unit is used to determine an optimized clustering result based on the merging result.

[0192] The data clustering device provided by the present application adopts the data clustering method in the above embodiment to solve the technical problem of data clustering. Compared with the prior art, the beneficial effects of the data clustering device provided by the present application are the same as the beneficial effects of the data clustering method provided by the above embodiment, and other technical features in the data clustering device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0193] The present application provides a data clustering device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data clustering method in the above-mentioned embodiment one.

[0194] Reference below Figure 5 , which shows a schematic diagram of the structure of a data clustering device suitable for implementing the embodiment of the present application. The data clustering device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The data clustering device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0195] like Figure 5As shown, the data clustering device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the data clustering device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data clustering device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a data clustering device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0196] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0197] The data clustering device provided by the present application adopts the data clustering method in the above embodiment to solve the technical problem of data clustering. Compared with the prior art, the beneficial effects of the data clustering device provided by the present application are the same as the beneficial effects of the data clustering method provided by the above embodiment, and the other technical features in the data clustering device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0198] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0199] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0200] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the data clustering method in the above-mentioned embodiment.

[0201] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0202] The computer-readable storage medium may be included in the data clustering device; or may exist independently without being assembled into the data clustering device.

[0203] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the data clustering device, the data clustering device:

[0204] Obtain the initial data to be clustered;

[0205] Dividing the initial data to obtain a plurality of first data groups;

[0206] For any first data group, the first data group is divided into at least one first cluster, and clustering processing is performed on the first cluster to obtain an optimized cluster result.

[0207] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0208] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0209] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0210] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned data clustering method, and can solve the technical problem of data clustering. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the data clustering method provided in the above-mentioned embodiment, and will not be described in detail here.

[0211] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned data clustering method when executed by a processor.

[0212] The computer program product provided by the present application can solve the technical problem of data clustering. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the data clustering method provided by the above embodiment, which will not be described in detail here.

[0213] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A data clustering method, characterized in that: The data clustering method comprises: Obtain the initial data to be clustered; Dividing the initial data to obtain a plurality of first data groups; For any first data group, the first data group is divided into at least one first cluster, and clustering processing is performed on the first cluster to obtain an optimized cluster result.

2. The method according to claim 1, characterized in that The initial data includes at least one initial group, and the step of dividing the initial data to obtain a plurality of first data groups includes: Randomly extracting first data from the initial data, and using the first data as current processing data; Determining a first distance between the currently processed data and each of the initial groups; Based on the first distance, the currently processed data is divided into initial groups until the initial data is divided into a plurality of first data groups.

3. The method according to claim 2, characterized in that The step of dividing the currently processed data into initial groups based on the first distance comprises: Comparing the first distance with a preset distance threshold to obtain a comparison result; If the comparison result shows that there is no initial group whose first distance is less than or equal to the preset distance threshold, dividing the currently processed data into first data groups; If the comparison result shows that there is an initial group whose first distance is less than or equal to the preset distance threshold, the currently processed data is divided into the initial group corresponding to the minimum value of the first distance.

4. The method according to claim 1, characterized in that After the step of dividing the initial data to obtain a plurality of first data groups, the method further includes: If the amount of data in the current first data group is less than a preset amount threshold, determining that the current first data group is a noise data group, and filtering the noise data group; If the amount of data in the current first data group is greater than or equal to a preset amount threshold, calculating the mean of all data in the current first data group, and taking the mean as the center point of the current first data group; Each of the first data groups is filtered based on a second distance between the central point and each data in each of the first data groups.

5. The method according to claim 4, characterized in that The step of filtering each of the first data groups based on the second distance between the center point and each data in each of the first data groups further includes: Calculate the second distance between each data and the center point; Based on the second distance, a distance mean and a distance error are calculated; Based on the distance mean and the distance error, a distance ratio is calculated; If the distance ratio is less than a preset ratio threshold, the first data group corresponding to the distance ratio is determined to be a noise data group, and the noise data group is filtered; otherwise, the current first data group is not filtered.

6. The method according to claim 1, characterized in that The step of dividing each of the first data groups into a plurality of first clusters comprises: Selecting currently processed data from each initial data, and calculating a weight value of each data in each first data group; Based on the arrangement order of the weight values, second data having a data distance less than a preset clustering radius threshold is determined in order from large to small, wherein the data distance is used to characterize the distance between the currently processed data and other data in the first data group; When the number of corresponding second data is greater than or equal to a preset cluster number threshold, integrating each second data into a first data set; Determine connected data sets in the first data set, and merge the connected data sets to obtain a second data set; For each of the first data groups, a plurality of first clusters are obtained based on the intersection of the first data set and the second data set.

7. The method according to claim 1, characterized in that The step of performing clustering processing on the first cluster to obtain an optimized cluster result includes: Calculate the mean of each data in the first cluster class to obtain the cluster center; Calculate the third distance between the remaining second cluster classes and the cluster center; If the third distance is less than a preset clustering radius threshold, merging the second cluster corresponding to the third distance with the current first cluster to obtain a merging result; Based on the merging result, an optimized clustering result is determined.

8. A data clustering device, characterized in that: The device comprises: An acquisition module, used to obtain initial data to be clustered; A division module, used for dividing the initial data to obtain a plurality of first data groups; The cluster processing module is used to divide any first data group into at least one first cluster, and perform cluster processing on the first cluster to obtain an optimized cluster result.

9. A data clustering device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data clustering method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data clustering method according to any one of claims 1 to 7 are implemented.

11. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the data clustering method according to any one of claims 1 to 7 are implemented.

Images