A classification method, device, equipment and storage medium

Abstract

The present application discloses a classification method, which includes: obtaining data to be classified; processing the data to be classified based on the K-means clustering algorithm to obtain K first clusters; wherein K is an integer greater than 1; obtaining the cluster weight of each cluster in the K first clusters; wherein the cluster weight of each cluster is used to represent the probability that each cluster is a cluster center; and classifying the K first clusters based on the cluster weight of each cluster. The present application also discloses a classification device, equipment, and storage medium. The classification method provided by the present application can reduce the complexity of data calculation time and the requirements for computing device memory while improving classification accuracy.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a classification method, a classification device, a classification equipment and a computer-readable storage medium. Background Art

[0002] When using the Density Peaks Clustering Algorithm (DCPA) to cluster data sets, as the amount of data at the data level increases, the time consumption of the data processing by DCPA is likely to increase and the memory consumption of the computing device is too large, which can easily affect the speed of data clustering. In order to improve the above defects of DCPA in data clustering, a grid-based density peak clustering method is proposed in the relevant technology. This method divides the data set to be classified into non-intersecting grid units of the same size, and calculates each grid unit as a whole, and then merges the above calculation results through DCPA to complete data clustering. However, although the computational time complexity of this clustering method has been improved, the accuracy of its clustering results has shown obvious deviations. Summary of the invention

[0003] The present application provides a classification method, a classification device, a classification equipment and a computer-readable storage medium.

[0004] The classification method provided in this application can overcome the problems of excessive computational complexity, excessive time consumption, and high memory usage of computing devices generated by DCPA when processing classified data, and can also improve the accuracy of data classification.

[0005] The classification method provided in this application is implemented as follows:

[0006] A classification method, comprising:

[0007] Obtain data to be classified;

[0008] Processing the data to be classified based on the K-means clustering algorithm to obtain K first-class clusters; wherein K is an integer greater than 1;

[0009] Obtaining a cluster weight of each cluster in the K first clusters; wherein the cluster weight of each cluster is used to indicate the probability that each cluster is a cluster center;

[0010] The K first clusters are classified based on the cluster weight of each cluster.

[0011] In some embodiments, classifying the K first clusters based on the cluster weight of each cluster includes:

[0012] Sorting the cluster weights of each cluster to obtain a sorting result;

[0013] Based on the sorting result, the K first clusters are classified.

[0014] In some implementations, classifying the K first-class clusters based on the sorting result includes:

[0015] Determining inflection point information in the sorting result;

[0016] The K first clusters are classified based on the inflection point information.

[0017] In some implementations, classifying the K first clusters based on the inflection point information includes:

[0018] Based on the inflection point information, determining a target cluster center from the K first clusters;

[0019] Based on the target cluster center and DPCA, the K first clusters are classified.

[0020] In some implementations, obtaining the cluster weight of each of the K first clusters includes:

[0021] Obtaining distance information between any two first-category clusters among the K first-category clusters;

[0022] Based on the distance information, a cluster weight of each cluster in the K first clusters is obtained.

[0023] In some implementations, obtaining distance information between any two first-type clusters among the K first-type clusters includes:

[0024] Obtaining cluster center point information of each cluster in the K first clusters;

[0025] Based on the cluster center point information of each cluster, distance information between any two first clusters is obtained.

[0026] In some implementations, calculating the cluster weights of the K first clusters based on the distance information includes:

[0027] Obtaining density information of each of the K first clusters;

[0028] Based on the density information of each cluster and the distance information, cluster weights of the K first clusters are calculated.

[0029] The present application also provides a classification device, which includes: an acquisition module and a processing module; wherein:

[0030] The acquisition module is used to acquire the data to be classified;

[0031] The processing module is used to process the classified data based on the K-means clustering algorithm to obtain K first clusters; obtain the cluster weight of each cluster in the K first clusters; classify the K first clusters based on the cluster weight of each cluster; wherein K is an integer greater than 1; the cluster weight of each cluster is used to represent the probability that each cluster is the cluster center of the K first clusters.

[0032] The present application also provides a classification device, which includes a processor, a memory, and a communication bus, wherein the communication bus is used to realize a communication connection between the processor and the memory; the processor is used to execute a computer program stored in the memory to implement any of the classification methods described above.

[0033] The present application also provides a computer-readable storage medium, which can be executed by a processor to implement any of the above classification methods.

[0034] In the classification method provided in the present application, in the process of using the K-means density algorithm to process the data to be classified and obtain K first-class clusters, the time complexity generated by the K-means density algorithm is greatly improved compared to the use of DCPA, and since the subsequent classification processing is based on the K first-class clusters instead of each data to be classified in all the data to be classified, the amount and size of memory applied for caching intermediate data during the classification process are also improved compared to DCPA.

[0035] The classification method provided in the present application adopts the K-means clustering algorithm to process the data to be classified and obtains the cluster centers of the K first-class clusters. During the execution of the classification method, the cluster centers will be continuously adjusted each time the cluster weights of the first-class clusters are sorted to obtain the sorting results, and the K first-class clusters will be reclassified according to the inflection point information determined by the sorting results. Therefore, the classification method provided in the present application can perform multiple sorting and classification according to the characteristics of the data to be classified itself. Therefore, the clustering results of the data to be classified by the classification method provided in the present application can more objectively reflect the actual characteristics of the data to be classified. Therefore, in the classification method provided in the present application, the accuracy of clustering is also greatly improved compared with the grid-based DCPA.

[0036] From the above, it can be seen that the classification method provided by the present application first obtains the data to be classified, processes the data to be classified based on the K-means clustering algorithm, obtains K first-class clusters, and then obtains the cluster weights in the K first-class clusters for indicating the probability that each cluster is a cluster center, and classifies the K first-class clusters based on the cluster weights of each cluster. In this way, the classification method provided by the present application has a significant improvement in the time complexity of the DPCA clustering method in the process of classifying the data to be classified, reduces the memory requirements of the computing device, and its classification accuracy is also significantly improved compared to the grid-based DPCA. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1a A schematic diagram of data set distribution in a data space in the related art;

[0038] Figure 1b For related technologies Figure 1a The corresponding decision diagram is obtained by counting each data point in the data set based on DCPA;

[0039] Figure 2 A schematic diagram of the process of the first classification method provided for this application;

[0040] Figure 3 A schematic diagram of the process flow of the second classification method provided for this application;

[0041] Figure 4 It is a schematic diagram of the distribution of the data to be classified in the two-dimensional space in the related technology;

[0042] Figure 5 A flowchart for implementing the third classification method provided in this application;

[0043] Figure 6 A schematic diagram of the structure of the classification device provided for this application;

[0044] Figure 7 A schematic diagram of the structure of the classification device provided for this application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0046] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] The present application relates to the field of data processing technology, and in particular to a classification method, a classification device, a classification equipment and a computer-readable storage medium.

[0048] DCPA is an algorithm for density clustering proposed in a paper published in Science in 2014. DCPA clusters categorical data using the idea that the density of the cluster center is greater than the density of the surrounding points and the distance between the cluster center and the higher density center is greater. Therefore, when using DCPA to mine categorical data, two necessary parameters must be calculated first: the local density ρ and the distance δ from the high density point.

[0049] Figure 1a Schematic diagram of data set distribution in data space in related technology.

[0050] exist Figure 1a The data set distributed in the two-dimensional space shown in FIG. 1 includes 28 data points in total, 1 to 28. Figure 1a The first part of the data in the upper left corner contains 20 data points, namely 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 14, 16, 17, 18, 20, 21, 23, 24 and 25; the second part of the data in the lower right corner contains 5 data points, namely 13, 15, 10, 19 and 22; and there are 3 data points distributed in other areas, namely 26, 27 and 28.

[0051] exist Figure 1a In the figure, the first part of the data is concentrated together, and is centered around data point 1; the second part of the data is concentrated around data point 10; and the three data points 26, 27, and 28 are far away from the first and second parts of the data, and the distances between these data are also far away.

[0052] Figure 1b For related technologies Figure 1a The corresponding decision diagram is obtained by counting each data point in the data set based on DCPA. Figure 1b The decision graph shown is formed by DCPA based on the local density ρ and the distance δ to the high-density points.

[0053] exist Figure 1b In , the horizontal axis can be the local density ρ, and the vertical axis is the distance δ between high-density points. Figure 1b In , data point 1 and data point 10 have both relatively high distance and local density, so data point 1 and data point 10 can be used as cluster centers; Figure 1b Although data points 26, 27, and 28 have relatively high distances, their local densities are relatively small, and therefore, these data points are called outliers.

[0054] pass Figure 1bIt can be seen that through DCPA Figure 1a By processing the data in the cluster, the data other than the outliers can be classified into the category of the cluster center that has a higher density and is closest to them.

[0055] However, since the clustering method of DCPA needs to calculate the distances between all data points or data object points in the data set in advance; and as the amount of data in the data set to be classified continues to grow, the time consumption when using DCPA to calculate the distance matrix will increase exponentially, and the dimension and / or size of the calculated distance matrix will also increase significantly. Therefore, during the execution of the algorithm, it is necessary to apply for a larger memory to the computing device to cache intermediate variables including high-dimensional and / or large-size matrices, which will also seriously affect the data processing speed.

[0056] In response to the above-mentioned defects of DCPA, a grid-based density peak clustering method is proposed in the related art. This method uses a network clustering algorithm to first compress the data set to be classified, that is, the grid idea is used to divide the data set to be processed into multiple non-intersecting grid units of equal size; then each grid unit is taken as a whole to calculate the density value of the grid unit and the distance value between the grid unit and the high-density grid unit; finally, the density value of the grid unit and the distance value between the grid unit and the high-density grid unit are merged through DCPA to complete data clustering.

[0057] For massive data sets to be classified, the grid clustering method is used to compress the data sets to be classified, which to a certain extent achieves the purpose of reducing the time complexity of the DCPA algorithm.

[0058] However, the above scheme is only one-sided in order to reduce the time complexity, and the grid clustering algorithm is selected to compress the data set to be classified, that is, it is just a compression processing method that subjectively divides the data into equal but non-intersecting grids. The above operation process does not consider how to ensure the accuracy of data clustering while compressing the data. Therefore, although this method can reduce the time complexity of the algorithm, it also pays the price of causing the accuracy of data clustering to be seriously affected. Therefore, the above method in the related technology makes it easy for the final clustering result to have obvious deviations. In other words, from the essence of the algorithm implementation, the clustering accuracy of the above scheme is not enough, and it relies on the density threshold set by experience or professional and technical personnel, which leads to its poor ability to recognize edge data.

[0059] In view of the above problems, the embodiment of the present application provides a classification method. This classification method fully combines the advantages of high accuracy of DCPA clustering results and the idea of ​​gridding the data set to be classified in the grid-based DCPA method, which can not only reduce the time complexity caused by clustering the data to be processed and reduce the memory requirement rate of the computer device, but also improve the accuracy of clustering.

[0060] Figure 2 A flowchart of the first classification method provided in an embodiment of the present application.

[0061] Figure 2 The classification method shown can be implemented by a processor of a classification device.

[0062] Exemplarily, the processor of the classification device can be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor.

[0063] exist Figure 2 The classification method provided in the embodiment of the present application may include the following steps:

[0064] Step 201: Obtain data to be classified.

[0065] In one implementation, the data to be classified may be stored in a classification device.

[0066] In one implementation, the data to be classified may be obtained by the classification device from a network or other storage devices.

[0067] In one implementation, the data to be classified may be a single data set or may include multiple data sets. For example, the multiple data sets may include a first data set, a second data set, and a third data set. The sources of the multiple data sets may be different. For example, the first data set may be from the storage space of the classification device, the second data set may be obtained by the classification device from the network, and the third data set may be randomly generated by the classification device in real time through an algorithm.

[0068] In one implementation, the data to be classified may be at least one of the following data: numbers, text information, pictures, videos, web pages, etc.

[0069] In one implementation, the data to be classified may include data of at least one category.

[0070] In one implementation, the data to be classified may include basic types of data, such as integer data, floating point data, etc.

[0071] In one embodiment, the data to be classified may include object type data. For example, a data object used to describe the overall characteristics of a person may include various information such as the person's gender, age, height, weight, occupation, hobbies, etc. Accordingly, each data in the data to be classified may be referred to as a data object.

[0072] In one implementation, each data in the data to be classified may carry feature information of at least one dimension, wherein the feature information may include at least one of the following: whether a specific object is included, and features of a specific object.

[0073] In one implementation, the specific object may be specific text information or specific image information.

[0074] In one embodiment, the specific text information may include at least one of the following: specific numbers, characters, strings, and combination information. For example, the combination information may be a combination of at least two types of specific numbers, characters, and strings, and at least one information in each type.

[0075] In one implementation, the specific image information may include at least one of the following: a person image, an animal image, a natural image, and a social image.

[0076] In one implementation, the character image may represent at least one local feature or a combination of local features of the character, such as a combination of facial and hand features.

[0077] In one implementation, the character image may also represent the overall characteristics of the character.

[0078] In one implementation, the character image may also represent the characteristics of a group of characters or the characteristics of a single character.

[0079] In one implementation, the character images may further include images of at least one type of character, for example, character images differentiated by gender, country, age, etc.

[0080] In one embodiment, the animal image may represent at least one local feature or a combination of local features of the animal, such as a combination of facial and neck features.

[0081] In one implementation, the animal image may also represent the overall characteristics of a certain type of animal.

[0082] In one embodiment, the animal image may also represent the characteristics of a group of animals or the characteristics of a single animal.

[0083] In one embodiment, the animal images may further include images of at least one type of animal, such as images of animals differentiated by body shape, habits, distribution area, etc.

[0084] In one embodiment, the natural image may refer to an image of natural scenery. For example, the natural image may be at least one of the following: a river, a mountain, a canyon, a glacier, an ocean, a desert, a coastline, and the like.

[0085] In one embodiment, the social image may refer to an image constructed by human society. For example, the social image may include at least one of the following: streets, buildings, traffic conditions, furniture, electronic equipment, etc.

[0086] In one implementation, the feature of a specific object may represent at least one of the following: the number, color, size, height, density, spacing, etc. of the specific objects.

[0087] In one embodiment, the amount of data to be classified may be changed in real time. For example, the classification device may acquire new data to be classified in real time and add the new data to be classified to the existing data to be classified.

[0088] Step 202: Process the data to be classified based on the K-means clustering algorithm to obtain K first-class clusters.

[0089] Wherein, K is an integer greater than 1.

[0090] K-means clustering algorithm is a cluster analysis algorithm that is solved through iteration. The steps are: randomly select K objects as initial cluster centers in advance, then calculate the distance between each data point or data object in the data to be classified and each initial cluster center, and assign each data object to the initial cluster center closest to it. The initial cluster center and the data objects assigned to the initial cluster center can represent a cluster.

[0091] Furthermore, each time a sample is assigned, the initial cluster center of the cluster can be recalculated and determined based on all existing objects in the cluster.

[0092] The above process will be repeated until the conditions for terminating the repetition are met.

[0093] The condition for terminating the repetition can be any of the following:

[0094] No data objects or data points were reassigned to any cluster;

[0095] The cluster centers no longer change;

[0096] The sum of squared errors is locally minimized.

[0097] In one implementation, each of the K first clusters may include a plurality of to-be-classified data.

[0098] For example, the set p of K first-class clusters may be p={p 1 ,p 2 ,p 3 ,…,p k}, where p 1 ,p 2 ,p 3 ,…,p k They are used to represent the first, second, third, ... and Kth first-class clusters respectively.

[0099] Correspondingly, the data set p of the i-th first-class cluster i Can be: p i ={d a ,d a+1 ,d a+2 ,…d a+i}, where d a ,d a+1 ,d a+2 ,…d a+i They are used to represent the first, second, third, ... and i-th data point or data object in the i-th first-class cluster, respectively. Where a is an integer greater than 1.

[0100] Step 203: Obtain the cluster weight of each of the K first clusters.

[0101] Among them, the cluster weight of each cluster is used to indicate the probability that each cluster is the cluster center.

[0102] In one implementation, the cluster weight of each cluster may be obtained by performing statistics on data points or data objects in each of the K first clusters.

[0103] In one implementation, the cluster weight of each cluster may be obtained by calculating the cluster density of each cluster in the K first clusters.

[0104] In one implementation, the cluster weight of each cluster may be obtained by calculating the distance between each cluster in the K first clusters and the other K-1 clusters.

[0105] In one implementation, the cluster weight of each cluster may be obtained by processing each of the K first clusters based on a certain algorithm.

[0106] In one embodiment, the cluster weight of each cluster may be obtained by processing each of the K clusters based on DPCA. Exemplarily, each of the K clusters may be processed as a whole data object through DPCA.

[0107] Step 204: Classify the K first clusters based on the cluster weight of each cluster.

[0108] In one implementation, a higher cluster weight may indicate a greater probability that the K first clusters are cluster centers; correspondingly, a lower cluster weight may indicate a smaller probability that the K first clusters are cluster centers.

[0109] In one embodiment, after the cluster weight of each cluster is obtained, at least one cluster center can be determined from the K first clusters based on the cluster weight of each cluster, and the remaining first clusters can be classified based on the at least one cluster center.

[0110] In one implementation, the K first-category clusters may be classified by using a specific clustering algorithm, for example, DPCA.

[0111] In practical applications, for a data volume of n to be classified, the time and space complexity of simply using DPCA in the related technology to complete the clustering operation is ο(n 2 ), and if the K-means clustering algorithm is used to process the above data to be classified, the time and space complexity generated is ο(K 2 ). Wherein, K is the number of the first type of clusters, and n is an integer greater than 1. Exemplarily, n can be an integer much greater than 1.

[0112] According to the results of theory and actual practice, when K is infinitely close to When the K-means clustering algorithm is used to classify the classified data, the time complexity and clustering accuracy can achieve the optimal solution at the same time.

[0113] Therefore, in the classification method provided in the embodiment of the present application, in the process of using the K-means density algorithm to process the data to be classified and obtain K first-class clusters, the time and space complexity generated by the K-means density algorithm is greatly improved compared to the use of DCPA, and since the subsequent classification processing is based on the K first-class clusters rather than on n data to be classified, the amount and size of memory applied for caching intermediate data during the classification process are also improved compared to DCPA. Furthermore, in the process of using the K-means clustering algorithm to process the data to be classified, the initially set K clustering centers will be continuously adjusted during the execution of the algorithm as the K first-class clusters are reclassified according to the cluster weights of each cluster. Therefore, the accuracy of its clustering is also greatly improved compared to the grid-based DCPA.

[0114] From the above, it can be seen that the classification method provided in the embodiment of the present application first obtains the data to be classified, processes the data to be classified based on the K-means clustering algorithm to obtain K first-class clusters, and then obtains the cluster weights in the K first-class clusters for indicating the probability that each cluster is a cluster center, and classifies the K first-class clusters based on the cluster weights of each cluster. In this way, the classification method provided in the embodiment of the present application has a significant improvement in the time complexity of the DPCA clustering method in the process of classifying the data to be classified, reduces the memory requirements of the computing device, and its classification accuracy is also significantly improved compared to the grid-based DPCA.

[0115] Based on the foregoing embodiments, the embodiments of the present application provide a second classification method. Figure 3 A flow chart of a second classification method provided in an embodiment of the present application. The classification method may include the following steps:

[0116] Step 301: Obtain data to be classified.

[0117] Exemplarily, the data to be classified D can be as follows D={d 1 ,d 2 ,d 3 ,d 4 ,…,d n}. Among them, d 1 ,d 2 ,d 3 ,d 4 ,…,d n They are used to represent the first, second, third, fourth, ... and Nth data point or data object respectively. N is an integer greater than 4.

[0118] Step 302: Process the data to be classified based on the K-means clustering algorithm to obtain K first-class clusters.

[0119] Wherein, K is an integer greater than 1.

[0120] Step 303: Obtain the cluster weight of each cluster in the K first clusters.

[0121] Among them, the cluster weight of each cluster is used to indicate the probability that each cluster is the cluster center.

[0122] Exemplarily, step 303 can be implemented by steps A1 and A2:

[0123] Step A1: Obtain distance information between any two first-category clusters among the K first-category clusters.

[0124] In one implementation, the distance information between any two first-type clusters may be the distance between a first data point or data object in a first first-type cluster and a second data point or data object in a second first-type cluster of any two first-type clusters.

[0125] Exemplarily, the first data point or data object may be any of the following: a data point or data object in the first first cluster that is closest to the second first cluster, a cluster center of the first first cluster, or any data point or data object in the first first cluster. The cluster center of the first first cluster may be referred to as the first cluster center.

[0126] Accordingly, the second data point or data object can be any of the following: the data point or data object in the second first cluster that is closest to the first first cluster, the cluster center of the second first cluster, or any data point or data object in the second first cluster. The cluster center of the second first cluster can be called the second cluster center.

[0127] In one implementation, the distance information between any two first-type clusters may be the Euclidean distance between any two first-type clusters.

[0128] Exemplarily, when the data to be classified is distributed in a quantifiable space, the distance information between the first first-category cluster and the second first-category cluster can be any of the following distances between the first data point in the first first-category cluster and the second data point in the second first-category cluster: Manhattan distance, Chebyshev distance, angle cosine similarity (Cosine), Pearson correlation coefficient, Mahalanobis distance, Hamming distance, etc.

[0129] Figure 4 It is a schematic diagram of the distribution of the data to be classified in the two-dimensional space in the related technology. Figure 4In the two-dimensional coordinate system shown, the abscissa is the distance in the horizontal direction, and the unit may be meter or centimeter, and the ordinate is the distance in the vertical direction, and the unit may be meter or centimeter.

[0130] exist Figure 4 In the figure, the first cluster 401 can be any cluster among the K first clusters, and the cluster center 402 can be the center of the first cluster 401. Correspondingly, other first clusters can also include corresponding cluster centers. The distance between the first cluster 401 and other first clusters can be the distance between the cluster center 402 of the first cluster 401 and the cluster centers of other first clusters in two-dimensional space. Specifically, it can be calculated by the first coordinate (x1, y1) of the cluster center 402 and the second coordinate (x2, y2) of the cluster centers of other clusters.

[0131] Exemplarily, step A1 can be implemented through steps B1 and B2:

[0132] Step B1: Obtain cluster center point information of each of the K first clusters.

[0133] In one implementation, the cluster center point information may be the location information of the cluster center point.

[0134] In one implementation, the location information of the cluster center point may be location information relative to a reference point.

[0135] In one implementation, the location information of the cluster center point may be determined by the location of the cluster center point relative to the origin of the coordinate system after the coordinate system is selected.

[0136] In one implementation, the location information of the cluster center point may be location information relative to a specified cluster among the K cluster centers.

[0137] For example, the cluster center point information m of each cluster can be m={cm 1 ,cm 2 ,cm 3 ,…,cm k}, where cm 1 ,cm 2 ,cm 3 ,…,cm k They are respectively the center point information of the first, second, third, ... and Kth first-category clusters. Exemplarily, the center point information may be the position information of the center point.

[0138] Step B2: Based on the cluster center point information of each cluster, obtain the distance information between any two first clusters.

[0139] In one implementation, the distance information between any two first clusters may be determined by cluster center point information of any two first clusters.

[0140] Exemplarily, the cluster center point of the first first cluster can be the first cluster center, the center point of the second first cluster can be the second cluster center, and the distance information between the first first cluster and the second first cluster can be determined by the Euclidean distance between the first cluster center and the second cluster center.

[0141] For example, after obtaining the center point information of each first cluster, the cluster distance matrix dist can be constructed, and the distance between each cluster and the higher density cluster can be calculated, thereby forming a distance value set containing each first cluster: δ = {cδ 1 ,cδ 2 ,cδ 3 ,…,cδ k}, where cδ 1 ,cδ 2 ,cδ 3 ,…,cδ k They are the distance information of the first, second, third, ... and Kth first-class clusters relative to the higher-density first-class clusters.

[0142] For example, the distance information cδ between the i-th first-class cluster and the first-class cluster with higher density is i It can be obtained by formula (1):

[0143]

[0144] In formula (1), we can obtain cδ i The condition is cρ i The global maximum. In formula (1), cρ i Used to represent the cluster density information of the i-th first cluster; cρ j Used to represent the cluster density information of the jth first cluster; cp i Used to represent the cluster center point information of the i-th first cluster; cp j It is used to represent the cluster center point information of the jth first cluster. For example, the cluster center point information of the jth first cluster can be the cluster center point position of the jth first cluster. In formula (1), i and j are both integers greater than 1.

[0145] In formula (1), The cluster density cρ for the i-th first cluster i Less than the jth first-class cluster cρ jWhen , the minimum distance between the i-th first-class cluster and the j-th first-class cluster is obtained. Exemplarily, the cluster density is greater than cρ i The number of first-class clusters, there can be multiple.

[0146] In formula (1), Used to obtain the cluster center position cp of the i-th first cluster i and the cluster center position cp of the jth first cluster j The maximum distance between .

[0147] Step A2: Based on the distance information, obtain the cluster weight of each cluster in the K first clusters.

[0148] In one implementation, the cluster weight of each cluster in the K first clusters may be obtained by sorting and calculating the distance information.

[0149] Exemplarily, step A2 may also be implemented through steps C1 and C2:

[0150] Step C1: Obtain density information of each of the K first-class clusters.

[0151] In one implementation, the density information of each of the K first clusters can be used to represent the number of data points contained in each cluster, that is, the density value. Figure 4 As shown, the density information of the first cluster 401 may be the number of data points in the circular area corresponding to the first cluster 401 .

[0152] For example, the density value cρ of the i-th first-class cluster among the K first-class clusters is i , can be realized by formula (2):

[0153]

[0154] In formula (2), x j is the ith data point or data object, C i is the i-th first-class cluster, num() is used to determine the data point or data object x j Is there a first-class cluster C? i If the data point or data object x j Exists in the first cluster C i If num() is incremented by 1, otherwise, num() is incremented by 0. After traversing all data points, the density value of each first-class cluster can be obtained.

[0155] Accordingly, the distance information between any two first clusters may be the distance between the ith first cluster and a higher density first cluster. Exemplarily, the higher density first cluster may be a first cluster having a density value higher than that of the ith first cluster.

[0156] Exemplarily, after obtaining the center point information and the distance information of each first-type cluster, it can be obtained according to the center point information and the distance information.

[0157] Step C2: based on the density information and distance information of each cluster, obtain the cluster weight of each cluster in the K first clusters.

[0158] In one implementation, the cluster weight of each cluster in the K first clusters may be obtained by performing a balanced calculation on the density information and the distance information of each cluster.

[0159] In one embodiment, the cluster weight of each cluster in the K first clusters can be obtained by statistically analyzing the density information and distance information of each cluster, determining a processing method for the density information and distance information of each cluster based on the statistical results, and processing the density information and distance information according to the processing method.

[0160] In one implementation, the cluster weight of each of the K first clusters may be obtained by processing density information and distance information of each cluster in a clustering algorithm.

[0161] In one implementation, the cluster weight of each of the K first clusters may be obtained by processing the density information and distance information of each cluster according to DPCA. i The cluster weight can be obtained by formula (3).

[0162] w i =cρ i ×cδ i (3)

[0163] In formula (3), cρ i is the density information of the i-th first-class cluster, that is, the density value of the i-th first-class cluster; cδ i is the distance between the i-th first cluster and the higher density cluster.

[0164] Step 304: sort the cluster weights of each cluster to obtain a sorting result.

[0165] In one implementation, the sorting result may be obtained by sorting the cluster weights corresponding to each cluster according to the number of each cluster.

[0166] In one implementation, the ranking result may be obtained by ranking the cluster weights corresponding to each cluster according to the cluster density information of each cluster.

[0167] In one implementation, the ranking result may be obtained by ranking the cluster weights according to a change rule of the cluster weights of each cluster.

[0168] In one implementation, the ranking result may be obtained by arranging the cluster weight of each cluster in descending order or ascending order.

[0169] Step 305: Classify the K first-category clusters based on the sorting result.

[0170] Exemplarily, step 305 can be implemented by steps D1 and D2:

[0171] Step D1, determining the inflection point information in the sorting result.

[0172] In one embodiment, the sorting result may be obtained by arranging the weight of each cluster in descending order. Accordingly, the inflection point information may be obtained by arranging the weight of each cluster in descending order and then statistically analyzing the slope of the curve information corresponding to the sorting result.

[0173] In one implementation, the inflection point information may be the point with the largest slope in the curve information corresponding to the sorting result.

[0174] In one implementation, the inflection point information may be a cluster deviation trend graph drawn according to the sorting result, and a critical point with the maximum deviation trend obtained from the graph.

[0175] Exemplarily, the inflection point information Tp can be obtained by formula (4).

[0176]

[0177] In formula (4), w i and w i+1 are used to represent the weight information of the i-th and i+1-th first-class clusters, respectively, w 1 Used to represent the weight information of the first cluster. It is used to find the critical point with the largest deviation trend among the K weight information. In formula (4), i is an integer greater than 1.

[0178] Step D2: classify the K first-category clusters based on the inflection point information.

[0179] Exemplarily, classifying the K first clusters based on the inflection point information may be to classify the first clusters whose weight information is less than the inflection point information into the second clusters, and to classify the first clusters whose weight information is greater than the inflection point information into the third clusters.

[0180] In one implementation, the K first clusters are classified based on the inflection point information, and the K first clusters may be classified according to the weight information, the inflection point information, and the density information and the distance information of each first cluster.

[0181] Exemplarily, step D2 may be implemented by steps E1 and E2:

[0182] Step E1: Based on the inflection point information, obtain the target cluster center from the K first-class clusters.

[0183] Exemplarily, the target cluster center may be the first cluster corresponding to the cluster weight information corresponding to the inflection point information.

[0184] In one implementation, the number of target cluster centers may be at least one.

[0185] Step E2: classify the K first-category clusters based on the target cluster center and DPCA.

[0186] In one implementation, the target cluster center can be used as the cluster center, and a density peak algorithm can be used to classify K-1 first-class clusters among the K first-class clusters, so that the first-class cluster closest to the target cluster center is merged into the target cluster center, thereby forming a final clustering result.

[0187] In one embodiment, after merging at least one first-category cluster into the first-category cluster corresponding to the target cluster center, the classification method provided in the embodiment of the present application can be used to classify the remaining data to be classified corresponding to several unmerged first-category clusters again, so as to more accurately classify each data to be classified.

[0188] Therefore, the classification method provided in the embodiment of the present application first divides and clusters the large data set to form several smaller cluster sets, and then calculates the number of data samples in each smaller cluster set as the cluster density value of each smaller cluster set, and determines the maximum cluster density value from the cluster density values, and then calculates the distance between other smaller cluster density values ​​and the maximum cluster density value to form a cluster distance matrix, and finally processes the cluster distance matrix through DPCA to obtain the final clustering result.

[0189] Moreover, the similarity of the data objects in the K first-category clusters obtained each time is relatively high, which also improves the accuracy of the classification method from another aspect.

[0190] Through the above steps, the classification method provided in the embodiment of the present application adopts the K-means clustering algorithm to process the data to be classified to obtain the cluster centers of the K first-class clusters. During the execution of the classification method, the cluster centers will be continuously adjusted as the cluster weights of the first-class clusters are sorted each time to obtain the sorting results, and the K first-class clusters are reclassified according to the inflection point information determined by the sorting results. Therefore, the classification method provided in the embodiment of the present application can perform multiple sorting and classification according to the characteristics of the data to be classified itself. Therefore, the clustering results of the data to be classified by the classification method provided by the present application can more objectively reflect the actual characteristics of the data to be classified.

[0191] Figure 5 This is a flowchart for implementing the third classification method provided in the embodiment of the present application. Figure 5 The classification method provided in the embodiment of the present application may include the following steps:

[0192] Step 501: Input data to be classified.

[0193] Step 502: pre-process the data to be classified.

[0194] Exemplarily, the data to be classified are preprocessed to normalize the data to be classified by a max-min method or a z-score normalization method, that is, the data to be classified are normalized to remove noise data in the data to be classified, thereby obtaining vectorized data to be classified.

[0195] Step 503: Obtain K first-category clusters.

[0196] Exemplarily, step 503 may be implemented through steps 5031 to 5034:

[0197] Step 5031: Randomly select K data objects as initial cluster centers.

[0198] In the embodiment of the present application, the data to be classified, and the data to be classified after preprocessing, may also be data objects.

[0199] Step 5032: Calculate the distance between each data point and the initial cluster center, and assign it to the cluster with the closest distance.

[0200] Exemplarily, after the data points are assigned to the clusters closest to them, the initial cluster centers may be updated according to the initial cluster centers and the data points assigned to the initial cluster centers.

[0201] Step 5033: Determine whether the stop condition is met.

[0202] If not, execute step 5032 again; if so, execute step 5034.

[0203] Exemplarily, the stop condition may be the condition for terminating repetition as described in the above embodiment, which will not be described in detail here.

[0204] Step 5034: output the K first-category clusters finally formed.

[0205] Step 504: Obtain the density value and cluster center value of each first cluster.

[0206] The cluster center value in step 504 may be the cluster center point information as described in the above-mentioned embodiment. For example, the cluster center value may also be continuously adjusted according to the execution process of the classification method.

[0207] Illustratively, the density value of each type of cluster in step 504 may be the density information of each type of cluster as described in the above-mentioned embodiment.

[0208] Step 505: Calculate the distance between each first cluster and a higher density first cluster using the cluster center value.

[0209] Step 506: Calculate the weight information of each cluster in the K first clusters, sort the weight information, obtain the inflection point information, and determine the target cluster center.

[0210] Step 507: merge at least one of the K first clusters with the target cluster center.

[0211] For example, in actual execution, the clusters in the K first clusters may be merged into the first cluster represented by the target cluster center closest to them.

[0212] Step 508: Output the final clustering result.

[0213] From the above, it can be seen that the classification method provided in the embodiment of the present application, after obtaining the data to be classified and processing the data to be classified based on the K-means clustering algorithm to obtain K first-class clusters, can also obtain the weight information of each cluster in the K first-class clusters, and sort the weight information of each cluster to obtain the sorting result, and then classify the K first-class clusters according to the sorting result.

[0214] In this way, the classification method provided in the embodiment of the present application, after obtaining the data to be classified, uses the K-means clustering algorithm to process the data to be classified to obtain K first-class clusters, thereby achieving data compression of the data to be classified, thereby reducing the amount of computation of subsequent data calculation and classification, and further reducing the time complexity of computing the data to be classified; on the other hand, when the subsequent classification operation is performed on the data to be classified, it is performed in units of K first-class clusters, rather than based on each data in the data to be classified, thereby reducing the amount of data calculation and the memory consumption caused by caching intermediate variables on the computing device; and, in the classification method provided in the embodiment of the present application, the K first-class clusters obtained by the K-means clustering algorithm and the cluster weights obtained subsequently can more truly reflect the distribution characteristics of the data to be classified itself, thereby overcoming the disadvantage of low accuracy of the grid-based DPCA clustering method.

[0215] When the amount of data to be classified is large, the higher accuracy, lower time complexity and lower computational complexity of the classification method provided by the embodiment of the present application are more obvious.

[0216] Based on the foregoing embodiments, an embodiment of the present application provides a classification device 6. Figure 6 A schematic diagram of the structure of the classification device 6 provided in an embodiment of the present application.

[0217] like Figure 6 As shown, the data classification device 6 may include: an acquisition module 601 and a processing module 602; wherein:

[0218] An acquisition module 601 is used to acquire data to be classified;

[0219] Processing module 602 is used to process the classified data based on the K-means clustering algorithm to obtain K first-class clusters; obtain the cluster weight of each cluster in the K first-class clusters; classify the K first-class clusters based on the cluster weight of each cluster; wherein K is an integer greater than 1; the cluster weight of each cluster is used to represent the probability that each cluster is the cluster center of the K first-class clusters.

[0220] In some implementations, the processing module 602 is used to sort the cluster weights of each cluster to obtain a sorting result; and classify the K first clusters based on the sorting result.

[0221] In some implementations, the processing module 602 is used to determine inflection point information in the sorting result; and classify the K first clusters based on the inflection point information.

[0222] In some implementations, the processing module 602 is configured to determine a target cluster center from the K first-category clusters based on the inflection point information; and classify the K first-category clusters based on the target cluster center and DPCA.

[0223] In some implementations, the acquisition module 601 is used to acquire distance information between any two first-type clusters among the K first-type clusters.

[0224] The processing module 602 is used to obtain the cluster weight of each cluster in the K first clusters based on the distance information.

[0225] In some implementations, the acquisition module 601 is used to acquire cluster center point information of each of the K first clusters.

[0226] The processing module 602 is used to obtain distance information between any two first clusters based on the cluster center point information of each cluster.

[0227] In some implementations, the acquisition module 601 is used to acquire density information of each of the K first clusters.

[0228] The processing module 602 is used to calculate the cluster weights of the K first clusters based on the density information and distance information of each cluster.

[0229] It should be noted that the acquisition module 601 and the processing module 602 can be at least one of a special application integrated circuit ASIC, a DSP, a DSPD, a PLD, an FPGA, a CPU, a controller, a microcontroller, and a microprocessor. It can be understood that the electronic device used to implement the above-mentioned processor function can also be other, and the embodiment of the present application does not specifically limit it.

[0230] Therefore, in the classification device provided in the embodiment of the present application, when the K-means density algorithm is used to process the data to be classified and the K first-class clusters are obtained, the time complexity generated is greatly improved compared with the use of DCPA, and since the subsequent classification processing is based on the K first-class clusters rather than on n data to be classified, the amount and size of memory applied for caching intermediate data during the classification process are also improved compared with DCPA. Furthermore, since the K-means clustering algorithm is used to process the data to be classified, the initially set K cluster centers will be continuously adjusted as the algorithm progresses, so the accuracy of its clustering is also greatly improved compared with the grid-based DCPA.

[0231] As can be seen from the above, the classification device 6 provided in the embodiment of the present application first obtains the data to be classified, processes the data to be classified based on the K-means clustering algorithm to obtain K first clusters, and then obtains the cluster weights in the K first clusters for indicating the probability that each cluster is a cluster center, and classifies the K first clusters based on the cluster weights of each cluster. In this way, the classification device 6 provided in the embodiment of the present application has a significant improvement in the time complexity of the DPCA clustering method in the process of classifying the data to be classified, reduces the memory requirements of the computing device, and has a significant improvement in classification accuracy compared to the grid-based DPCA.

[0232] Based on the above embodiments, the present application embodiment provides a classification device 7, Figure 7 This is a schematic diagram of the structure of the classification device 7 provided in the embodiment of the present application. Figure 7 As shown, the classification device 7 may include a processor 701, a memory 702 and a communication bus, wherein the communication bus is used to realize the communication connection between the processor 701 and the memory 702, and the processor 701 is used to execute the computer program stored in the memory 702 to realize the classification method as described in any of the aforementioned embodiments.

[0233] It should be noted that the processor 701 may be at least one of an application specific integrated circuit ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. It is understandable that the electronic device used to implement the functions of the processor may also be other, which is not specifically limited in the embodiments of the present application.

[0234] The memory 702 may be a volatile memory, such as a RAM; or a non-volatile memory, such as a ROM, a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above types of memories, and provide instructions and data to the processor.

[0235] Based on the foregoing embodiments, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the classification method as described in any of the preceding items is implemented.

[0236] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0237] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0238] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0239] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0240] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0241] It should be noted that the above-mentioned computer-readable storage medium can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM) and other memories; it can also be various electronic devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0242] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0243] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0244] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0245] The present application is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0246] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0247] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0248] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A classification method, characterized in that: The method comprises: Acquire data to be classified; wherein the data to be classified includes at least one of text information, pictures and videos; Processing the data to be classified based on the K-means clustering algorithm to obtain K first-class clusters; wherein K is an integer greater than 1; Obtaining a cluster weight of each cluster in the K first clusters; wherein the cluster weight of each cluster is used to indicate the probability that each cluster is a cluster center; the cluster weight of each cluster is obtained by processing each cluster in the K first clusters as a whole data object through DPCA; the cluster weight of each cluster is obtained by statistically analyzing the data points or data objects contained in each cluster in the K first clusters; The K first clusters are classified based on the cluster weight of each cluster.

2. The method according to claim 1, characterized in that The classifying the K first clusters based on the cluster weight of each cluster includes: Sorting the cluster weights of each cluster to obtain a sorting result; Based on the sorting result, the K first clusters are classified.

3. The method according to claim 2, characterized in that The classifying the K first-class clusters based on the sorting result includes: Determining inflection point information in the sorting result; The K first clusters are classified based on the inflection point information.

4. The method according to claim 3, characterized in that The classifying the K first clusters based on the inflection point information includes: Based on the inflection point information, determining a target cluster center from the K first clusters; Based on the target cluster center and the density peak algorithm DPCA, the K first clusters are classified.

5. The method according to claim 1, characterized in that The obtaining of the cluster weight of each cluster in the K first clusters includes: Obtaining distance information between any two first-category clusters among the K first-category clusters; Based on the distance information, a cluster weight of each cluster in the K first clusters is obtained.

6. The method according to claim 5, characterized in that The obtaining of distance information between any two first-category clusters among the K first-category clusters includes: Obtaining cluster center point information of each cluster in the K first clusters; Based on the cluster center point information of each cluster, distance information between any two first clusters is obtained.

7. The method according to claim 5, characterized in that The calculating the cluster weights of the K first clusters based on the distance information includes: Obtaining density information of each of the K first clusters; Based on the density information of each cluster and the distance information, cluster weights of the K first clusters are calculated.

8. A classification device, characterized in that: The classification device comprises: an acquisition module and a processing module; wherein: The acquisition module is used to acquire the data to be classified; wherein the data to be classified includes at least one of text information, pictures and videos; The processing module is used to process the classified data based on the K-means clustering algorithm to obtain K first clusters; obtain the cluster weight of each cluster in the K first clusters; classify the K first clusters based on the cluster weight of each cluster; wherein K is an integer greater than 1; the cluster weight of each cluster is used to indicate the probability that each cluster is the cluster center of the K first clusters; the cluster weight of each cluster is obtained by processing each cluster in the K first clusters as a whole data object through DPCA; the cluster weight of each cluster is obtained by statistically analyzing the data points or data objects contained in each cluster in the K first clusters.

9. A sorting device, characterized in that: The classification device includes a processor, a memory and a communication bus, wherein the communication bus is used to realize the communication connection between the processor and the memory; the processor is used to execute the computer program stored in the memory to realize the classification method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The readable storage medium can be executed by a processor to implement the classification method as described in any one of claims 1-7.

Images