Text classification method and device based on vertical clustering
Through multiple rounds of vertical clustering methods, including initial k-mean clustering, resampling and adaptive k-mean clustering, the problem of insufficient iterations of existing clustering algorithms is solved, and a deeper and accurate clustering structure is constructed, which improves the quality and efficiency of text classification.
Patent Information
- Application Number
- CN202411794894.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-09
- Publication Date
- 2025-04-25
AI Technical Summary
Existing clustering algorithms, such as k-mean clustering and hierarchical clustering, usually only perform single-round iterations, limiting the improvement of clustering depth and accuracy.
A text classification method based on vertical clustering is proposed. Through multiple rounds of vertical clustering, including initial k-mean clustering, resampling and adaptive k-mean clustering, iterating multiple times until the termination condition is met, and a hierarchical clustering structure is constructed.
Through multiple rounds of vertical clustering, deeper and more precise clustering structures can be built, the quality and efficiency of text classification can be improved, category exploration can be automated, and data set representativeness and diversity can be enhanced.
Smart Images
Figure CN119884381A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to a text classification method. Background Art
[0002] Clustering refers to dividing a data set into different clusters according to a specific standard (such as distance), so that the similarity of data objects in the same cluster is as large as possible, and the difference of data objects in different clusters is as large as possible. After clustering, data of the same category are gathered together as much as possible, and data of different categories are separated as much as possible.
[0003] Common clustering algorithms include k-means clustering, hierarchical clustering, etc.
[0004] K-means clustering is an iterative clustering algorithm that divides data into K groups in advance. K is a fixed value. K data are randomly selected as the initial cluster centers. Then the distance between each data and each cluster center is calculated, and each data is assigned to the cluster center closest to it. The cluster centers and the data assigned to them represent a cluster. Each time a data is assigned, the cluster center will be recalculated based on the existing data. This process will be repeated until the set termination condition is met.
[0005] Hierarchical clustering creates a hierarchical nested clustering tree by calculating the similarity between different categories of data. In the clustering tree, the original data point is the lowest level of the tree, and the top level of the tree is the root node with only one cluster. There are two strategies for hierarchical clustering: one is the agglomerative method of continuous merging from bottom to top, and the number of clusters gradually decreases until only one cluster remains or the set termination condition is met; the other is the divisive method of continuous splitting from top to bottom, and the number of clusters gradually increases until each cluster can no longer be divided (contains only one data) or the set termination condition is met.
[0006] Existing clustering algorithms have the following disadvantages. Both k-means clustering and hierarchical clustering involve an iterative process, but usually only one round of iteration is performed, and the iteration is terminated after the set termination condition is met. This single round of iteration limits the improvement of clustering depth and accuracy. Summary of the invention
[0007] The technical problem to be solved by this application is how to improve the depth and accuracy of clustering.
[0008] In order to solve the above technical problems, the present application proposes a text classification method based on vertical clustering, comprising the following steps. Step S1: Collect various types of data as data to be screened. Step S2: Use sentence embedding to calculate the embedded representation of the data to be screened in the feature space. Step S3: Use the embedded representation of the data to be screened to perform multiple rounds of vertical clustering on the data to be screened to obtain cluster data after multiple rounds of vertical clustering. Step S4: Use a large model to extract category labels from the cluster data obtained by multiple rounds of vertical clustering, and use the category label of each cluster data extracted as the category label of each original data point in the corresponding cluster; the original data point is the data to be screened. Step S5: Construct a classification vector database based on the embedded representation and category labels of all data to be screened. Step S6: Use the classification vector database to classify new input data to obtain predicted category labels for the new input data.
[0009] Furthermore, the step S3 further includes the following steps. Step S31: initial k-means clustering; the data to be screened is called the original data point, and all the original data points are divided into K clusters according to the position of the embedded representation of all the original data points in the feature space, where K is a preset value, and a cluster refers to a set of original data points belonging to the same cluster center, so as to minimize the sum of the squares of the distances from the original data points in each cluster to the center of each cluster; initial k-means clustering obtains K clusters, and generates a cluster center for each cluster. Step S32: resample the clusters obtained by the last clustering; for each cluster obtained by the last clustering, select multiple original data points from the cluster to form a subset of the cluster. Step S33: use the subset of each cluster obtained by the last clustering as the new input data, and perform adaptive k-means clustering; adaptive k-means clustering obtains multiple clusters, and generates a cluster center for each cluster. Repeat steps S32 to S33 until the set termination condition is met. Step S34: resampling the clusters obtained from the last clustering to obtain cluster data; for each cluster obtained from the last adaptive k-means clustering, multiple original data points are selected from the cluster to form a subset of the cluster, which is called cluster data.
[0010] Preferably, in step S32, for each cluster obtained by the last clustering, a plurality of original data points close to the cluster center are selected from the cluster to form a subset of the cluster.
[0011] Furthermore, the termination condition is that a preset number of iterations is reached, or the sum of the Euclidean distances of all data points in the cluster to the cluster center changes below a threshold.
[0012] Furthermore, after multiple rounds of iterations of step S32 and step S33 are terminated, a multi-level tree cluster structure is obtained, in which the clusters obtained by the last clustering serve as leaf nodes, and each layer of nodes inside represents the clusters obtained by each previous clustering.
[0013] Preferably, in step S34, for each cluster obtained by the last adaptive k-means clustering, multiple original data points are selected from the cluster by uniform sampling to form a subset of the cluster, which is called cluster data.
[0014] Furthermore, in step S4, the large model is first used to generate a content summary for each cluster data after multiple rounds of vertical clustering; then the large model is used to extract a category label from the content summary generated for each cluster data to represent the content of the cluster data; then, each original data point in the cluster corresponding to each cluster data is marked with the extracted category label of the cluster data.
[0015] Furthermore, in step S5, a nearest neighbor search algorithm is used to construct a classification vector database.
[0016] Furthermore, in step S6, the sentence embedding model is first used to calculate the new input data to obtain the embedded representation of the new input data in the feature space, which is called the query vector; then the nearest neighbor search algorithm is used to find the top P embedded representations and their corresponding category labels that are most similar to the query vector in the classification vector database; then the category labels corresponding to the top P embedded representations are voted, and the category label with the most occurrences is used as the predicted category label of the new input data.
[0017] The present application also proposes a text classification device based on vertical clustering, including a data collection unit, a vector representation unit, a multi-round vertical clustering unit, a category extraction unit, a database construction unit, and a classification prediction unit. The data collection unit is used to collect various types of data as data to be screened. The vector representation unit is used to calculate the embedded representation of the data to be screened in the feature space using a sentence embedding model. The multi-round vertical clustering unit is used to perform multi-round vertical clustering on the data to be screened using the embedded representation of the data to be screened, and obtain cluster data after multi-round vertical clustering. The category extraction unit is used to extract category labels from the cluster data obtained by multi-round vertical clustering using a large model, and use the category label of each extracted cluster data as the category label of each original data point in the corresponding cluster; the original data point is the data to be screened. The database construction unit is used to build a classification vector database based on the embedded representation and category labels of all data to be screened. The classification prediction unit is used to classify new input data using the classification vector database to obtain predicted category labels for the new input data.
[0018] The technical effect achieved by this application is to perform text classification through multiple rounds of vertical clustering, which can build a hierarchical clustering structure, automate category discovery, improve the representativeness and diversity of the data set, and adopt technical means such as efficient nearest neighbor search to improve the quality and efficiency of text classification. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1It is a flowchart of the text classification method based on vertical clustering proposed in this application.
[0020] Figure 2 It is a schematic diagram of the process of the multi-round vertical clustering method in step S3.
[0021] Figure 3 It is a structural diagram of a text classification device based on vertical clustering proposed in this application.
[0022] Explanation of the reference numerals in the figure: data acquisition unit 1, vector representation unit 2, multi-round vertical clustering unit 3, category extraction unit 4, database building unit 5, classification prediction unit 6. DETAILED DESCRIPTION
[0023] See also Figure 1 ,The text classification method based on vertical clustering proposed in this application includes the following steps.
[0024] Step S1: Collect various types of data as data to be screened, such as web page type data, book type data, paper type data, etc.
[0025] Step S2: Use a sentence embedding model to calculate the data to be screened, and obtain the embedding representation of the data to be screened in the feature space, which is recorded as <data, embedding representation>.
[0026] Step S3: Using the embedded representation of the data to be screened, perform multiple rounds of vertical clustering on the data to be screened to obtain cluster data after multiple rounds of vertical clustering.
[0027] See also Figure 2 The specific method of multiple rounds of vertical clustering in step S3 is as follows.
[0028] Step S31: Initial (first) k-means clustering. The data to be screened are called original data points. According to the positions of the embedded representations of all original data points in the feature space, all original data points are divided into K clusters, where K is a preset value. A cluster refers to a collection of original data points belonging to the same cluster center, so as to minimize the sum of the squares of the distances from the original data points in each cluster to the centers of each cluster. The initial k-means clustering includes one round of iterative process, and the clustering result is K clusters. A cluster center is generated for each cluster, and these cluster centers are the basis for the next clustering.
[0029] Step S32: Resample the clusters obtained from the last clustering. The last clustering results in multiple clusters, each cluster has a cluster center, and each cluster contains multiple original data points belonging to the same cluster center. For each cluster obtained from the last clustering, a certain number of original data points are selected from the cluster (original data points close to the cluster center are preferred) to form a cluster subset, and the next clustering is performed based on this cluster subset. The resampling step is introduced in this step to ensure a more uniform distribution of the original data points, thereby improving the quality and balance of clustering and optimizing the clustering results.
[0030] Step S33: Adopt the subset of each cluster obtained from the last clustering as new input data to perform adaptive k-means clustering. In the traditional k-means clustering algorithm, K is a fixed value. In the adaptive k-means clustering algorithm, the value of K is not fixed, and the number of clusters K can be automatically adjusted according to the distribution of data. This is a prior art. Performing an adaptive k-means clustering also includes a round of iterative process. The clustering result is to obtain multiple clusters, and a cluster center is generated for each cluster. These cluster centers are the basis for the next clustering.
[0031] Repeat steps S32 to S33 until the set termination condition is met. The termination condition is, for example, reaching a preset number of iterations, or the sum of the Euclidean distances of all data points in the cluster to the cluster center changes below a certain threshold (indicating that the cluster has become relatively stable). Multiple rounds of iterations are used to achieve in-depth mining and clustering of the original data points to obtain more accurate cluster divisions. After the iteration is terminated, a multi-level tree cluster structure is obtained, with the cluster obtained by the last clustering as a leaf node, and each layer of nodes inside represents the cluster obtained by each previous clustering. This multi-level tree cluster structure indicates that the original data points can be organized into clusters (categories) of different levels. Specifically, this means that the original data points do not exist in isolation, but are interconnected in a hierarchical manner. Certain data points or combinations of data points can be regarded as part of a larger cluster, and this larger cluster can be further subdivided into multiple smaller clusters.
[0032] Step S34: Resample the clusters obtained from the last clustering to obtain cluster data. For each cluster obtained from the last adaptive k-means clustering, a certain number of original data points are selected from the cluster (uniform sampling is preferred) to form a subset of the cluster, called cluster data, which serves as input data for subsequent category recognition. This step of resampling ensures the representativeness and diversity of the cluster data, providing high-quality data for subsequent classification and recognition.
[0033] The multiple rounds of vertical clustering in step S3 are significantly different from existing clustering methods. First, existing clustering methods, such as k-means clustering and hierarchical clustering, only perform one round of iterative operations. The multiple rounds of vertical clustering in the present application perform multiple rounds of iterative operations, that is, iterative nested iterations. The external iterative operation is: steps S32 to S33 are executed multiple times. The nested internal iterative operation is: the adaptive k-means clustering method adopted in step S33 includes one round of iterative process each time it is executed. Second, in the existing k-means clustering method, the number of clusters K is a fixed preset value. In the multiple rounds of vertical clustering of the present application, the number of clusters in each round of adaptive k-means clustering (i.e., step S33) is not fixed, and will be dynamically adjusted during the clustering process. Third, the multi-round vertical clustering of the present application improves the quality and balance of clustering through multiple iterations of adaptive k-means clustering (i.e., step S33) combined with resampling (i.e., step S22), and resamples the clusters obtained from the previous clustering as new input data points for the next clustering, thereby improving the depth and accuracy of clustering.
[0034] Step S4: Use a large model to extract category labels from the cluster data obtained from multiple rounds of vertical clustering, and use the category label of each cluster data extracted as the category label of each original data point (i.e., the data to be screened) in the corresponding cluster. A large model refers to a machine learning model with ultra-large-scale parameters and complex computing structure.
[0035] This step first uses the big model to generate a content summary for each cluster data after multiple rounds of vertical clustering; then uses the big model to extract the category label from the content summary generated for each cluster data to represent the content of the cluster data; then, each original data point in the cluster corresponding to each cluster data is marked with the extracted category label of the cluster data.
[0036] Step S5: Using the embedding representation <data, embedding representation> of each data to be screened and the category label of each data to be screened, the <embedded representation, category label> of each data to be screened is obtained, and then a classification vector database is constructed based on the <embedded representation, category label> of all data to be screened.
[0037] In this step, for example, a nearest neighbor search algorithm is used to construct a classification vector database, preferably a HNSW (Hierarchical Navigable Small World) algorithm is used to construct a classification vector database. HNSW is an efficient Approximate Nearest Neighbor (ANN) search algorithm, which is particularly suitable for large-scale high-dimensional vector data. By inserting the <embedded representation, category label> tuples of all the data to be screened into the HNSW index, a classification vector database that can be quickly retrieved can be constructed.
[0038] Step S6: Classify the new input data using the classification vector database to obtain the predicted category label of the new input data.
[0039] In this step, the sentence embedding model is first used to calculate the new input data to obtain the embedded representation of the new input data in the feature space, which is called the query vector. The HNSW algorithm is then used to find the top P embedded representations and their corresponding category labels that are most similar to the query vector in the classification vector database. The category labels corresponding to the top P embedded representations are then voted on, and the category label that appears the most times is used as the predicted category label for the new input data.
[0040] See also Figure 3 The text classification device based on vertical clustering proposed in this application includes a data acquisition unit 1, a vector representation unit 2, a multi-round vertical clustering unit 3, a category extraction unit 4, a library building unit 5, and a classification prediction unit 6. Figure 3 The device shown corresponds to Figure 1 The method shown.
[0041] The data collection unit 1 is used to collect various types of data as data to be screened.
[0042] The vector representation unit 2 is used to calculate the embedding representation of the data to be screened in the feature space by using a sentence embedding model.
[0043] The multi-round vertical clustering unit 3 is used to perform multi-round vertical clustering on the data to be screened by using the embedded representation of the data to be screened, and obtain cluster data after the multi-round vertical clustering.
[0044] The category extraction unit 4 is used to extract category labels from cluster data obtained by multiple rounds of vertical clustering using a large model, and use the extracted category label of each cluster data as the category label of each original data point (ie, data to be screened) in the corresponding cluster.
[0045] The database building unit 5 is used to build a classification vector database based on the embedded representation and category labels of all the data to be screened.
[0046] The classification prediction unit 6 is used to classify new input data using the classification vector database to obtain a predicted category label of the new input data.
[0047] Compared with the prior art, the present application has the following beneficial effects.
[0048] First, this application uses multiple rounds of vertical clustering methods to deeply explore the hierarchical structure of the data, which can more finely identify and classify complex patterns in the data compared to traditional single-round clustering methods. Suppose there is a large amount of news article data. In the first round of clustering, this application may classify articles by topic (such as politics, sports, and technology), and in the second round of clustering, each topic can be further subdivided, such as dividing articles under the theme of technology into sub-topics such as artificial intelligence, software development, and hardware evaluation. This hierarchical classification reveals more detailed patterns and structures in the data.
[0049] Second, the present application iteratively uses the adaptive k-means clustering algorithm (i.e., step S33) in the multiple rounds of vertical clustering process in step S3 to dynamically adjust the number of clusters instead of fixing the number of clusters. This flexibility enables the clustering results to more accurately reflect the actual distribution of the data. The present application also improves the representativeness and diversity of the final data set through uniform sampling (i.e., step S34) in the multiple rounds of vertical clustering process in step S3, which is crucial to improving the generalization ability of the classifier.
[0050] Third, traditional text classification methods require pre-defined categories or in-depth understanding of data content during category discovery, which increases manual intervention and reliance on prior knowledge. The present application can automatically extract category labels from cluster data in step S4 without pre-defining categories or in-depth understanding of data content, reducing manual intervention and reliance on prior knowledge.
[0051] Fourth, the present application adopts an efficient nearest neighbor search algorithm in step S5 to construct a classification vector database, which is particularly suitable for fast retrieval of high-dimensional data and improves the efficiency and accuracy of data retrieval.
[0052] Fifth, the present application can quickly predict the category labels of new data in step S6, especially when processing large-scale data, demonstrating a rapid response capability that is difficult to match with existing technologies.
[0053] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should all be included in the protection scope of the present application.
Claims
1. A text classification method based on vertical clustering, characterized in that: The method comprises the following steps: Step S1: Collect various types of data as data to be screened; Step S2: Use sentence embedding to calculate the embedding representation of the data to be screened in the feature space; Step S3: using the embedded representation of the data to be screened, perform multiple rounds of vertical clustering on the data to be screened, and obtain cluster data after multiple rounds of vertical clustering; Step S4: using a large model to extract category labels from cluster data obtained by multiple rounds of vertical clustering, and using the category label of each cluster data extracted as the category label of each original data point in the corresponding cluster; the original data point is the data to be screened; Step S5: construct a classification vector database based on the embedded representation and category labels of all the data to be screened; Step S6: Classify the new input data using the classification vector database to obtain the predicted category label of the new input data.
2. The text classification method based on vertical clustering according to claim 1, characterized in that: The step S3 further comprises the following steps: Step S31: initial k-means clustering; the data to be screened are called original data points, and all original data points are divided into K clusters according to the positions of the embedded representations of all original data points in the feature space, where K is a preset value, and a cluster refers to a collection of original data points belonging to the same cluster center, so as to minimize the sum of the squares of the distances from the original data points in each cluster to the center of each cluster; initial k-means clustering obtains K clusters, and generates a cluster center for each cluster; Step S32: resampling the clusters obtained by the last clustering; for each cluster obtained by the last clustering, selecting a plurality of original data points from the cluster to form a subset of the cluster; Step S33: using a subset of each cluster obtained by the previous clustering as new input data, and performing adaptive k-means clustering; adaptive k-means clustering obtains multiple clusters, and generates a cluster center for each cluster; Repeat steps S32 to S33 until the set termination condition is met; Step S34: resampling the clusters obtained from the last clustering to obtain cluster data; for each cluster obtained from the last adaptive k-means clustering, multiple original data points are selected from the cluster to form a subset of the cluster, which is called cluster data.
3. The text classification method based on vertical clustering according to claim 2, characterized in that: In step S32, for each cluster obtained by the last clustering, a plurality of original data points close to the cluster center are selected from the cluster to form a subset of the cluster.
4. The text classification method based on vertical clustering according to claim 2 is characterized in that: The termination condition is that a preset number of iterations is reached, or the sum of the Euclidean distances of all data points in the cluster to the cluster center changes below a threshold.
5. The text classification method based on vertical clustering according to claim 2, characterized in that: After multiple rounds of iterations of step S32 and step S33 are terminated, a multi-level tree cluster structure is obtained, in which the clusters obtained by the last clustering serve as leaf nodes, and each layer of nodes inside represents the clusters obtained by each previous clustering.
6. The text classification method based on vertical clustering according to claim 2 is characterized in that: In step S34, for each cluster obtained by the last adaptive k-means clustering, multiple original data points are selected from the cluster by uniform sampling to form a subset of the cluster, which is called cluster data.
7. The text classification method based on vertical clustering according to claim 1 is characterized in that: In step S4, the large model is first used to generate a content summary for each cluster data after multiple rounds of vertical clustering; then the large model is used to extract a category label from the content summary generated for each cluster data to represent the content of the cluster data; Then, for each original data point in the cluster corresponding to each cluster data, the category label of the extracted cluster data is marked.
8. The text classification method based on vertical clustering according to claim 1, characterized in that: In step S5, a nearest neighbor search algorithm is used to construct a classification vector database.
9. The text classification method based on vertical clustering according to claim 1, characterized in that: In step S6, the sentence embedding model is first used to calculate the new input data to obtain the embedding representation of the new input data in the feature space, which is called the query vector; then the nearest neighbor search algorithm is used to find the top P embedding representations and their corresponding category labels that are most similar to the query vector in the classification vector database; Then, the category labels corresponding to the top P embedding representations are voted, and the category label with the most occurrences is used as the predicted category label for the new input data.
10. A text classification device based on vertical clustering, characterized in that: It includes data collection unit, vector representation unit, multi-round vertical clustering unit, category extraction unit, database building unit, and classification prediction unit; The data collection unit is used to collect various types of data as data to be screened; The vector representation unit is used to calculate the embedding representation of the data to be screened in the feature space by using a sentence embedding model; The multi-round vertical clustering unit is used to perform multi-round vertical clustering on the data to be screened using the embedded representation of the data to be screened, and obtain cluster data after the multi-round vertical clustering; The category extraction unit is used to extract category labels from cluster data obtained by multiple rounds of vertical clustering using a large model, and use the extracted category label of each cluster data as the category label of each original data point in the corresponding cluster; the original data point is the data to be screened; The database building unit is used to build a classification vector database based on the embedded representation and category labels of all the data to be screened; The classification prediction unit is used to classify new input data using the classification vector database to obtain a predicted category label of the new input data.