Text clustering method and device, computer readable medium and computer equipment

Abstract

The embodiment of the invention provides a text clustering method and device, a computer readable medium and computer equipment. The text clustering method comprises the steps of obtaining a to-be-clustered text and clustering keywords respectively corresponding to at least two clustering categories; matching the text to be clustered with the clustering keywords respectively corresponding to the at least two clustering categories to obtain a matching degree between the text to be clustered and each clustering category; according to the matching degree between the to-be-clustered text and each clustering category, dividing the to-be-clustered text into the at least two clustering categories; and extracting a feature vector corresponding to the to-be-clustered text in each clustering category, and performing clustering processing on the to-be-clustered text in each clustering category according to the feature vector corresponding to the to-be-clustered text to obtain a clustering result of the to-be-clustered text. According to the technical scheme provided by the embodiment of the invention, the text clustering accuracy and the text clustering efficiency are effectively improved.

Description

Technical Field

[0001] The present application relates to the field of computer and communication technology, and in particular to a text clustering method, apparatus, computer-readable medium, and computer equipment. Background Art

[0002] In today's information explosion, data volumes are growing exponentially, particularly in scenarios like social media, news commentary, and video comments, generating massive amounts of duplicate or similar text. This redundant information not only consumes significant storage resources but also significantly wastes the time and effort of manual review. To improve review efficiency and optimize resource allocation, there is an urgent need for a technology that can efficiently and accurately cluster similar text. However, existing text clustering schemes suffer from poor clustering accuracy and high computational complexity. Summary of the Invention

[0003] The embodiments of the present application provide a text clustering method, apparatus, computer-readable medium, and computer equipment, which can first perform rapid coarse classification through a domain keyword table, and then vectorize the text to be clustered within each cluster category and perform fine-grained clustering division, effectively improving the accuracy and efficiency of text clustering.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by practice of the present application.

[0005] According to one aspect of an embodiment of the present application, a text clustering method is provided, comprising: obtaining text to be clustered, and clustering keywords corresponding to at least two clustering categories; matching the text to be clustered with the clustering keywords corresponding to the at least two clustering categories to obtain a matching degree between the text to be clustered and each of the clustering categories; dividing the text to be clustered into the at least two clustering categories according to the matching degree between the text to be clustered and each of the clustering categories; extracting a feature vector corresponding to the text to be clustered in each of the clustering categories, and clustering the text to be clustered in each of the clustering categories according to the feature vector corresponding to the text to be clustered to obtain a clustering result of the text to be clustered.

[0006] According to one aspect of an embodiment of the present application, a text clustering device is provided, comprising: an acquisition unit configured to acquire a text to be clustered and clustering keywords corresponding to at least two clustering categories; a matching unit configured to match the text to be clustered with the clustering keywords corresponding to the at least two clustering categories, to obtain a matching degree between the text to be clustered and each of the clustering categories; a division unit configured to divide the text to be clustered into the at least two clustering categories according to the matching degree between the text to be clustered and each of the clustering categories; a processing unit configured to extract a feature vector corresponding to the text to be clustered in each of the clustering categories, and perform clustering processing on the text to be clustered in each of the clustering categories according to the feature vector corresponding to the text to be clustered, to obtain a clustering result of the text to be clustered.

[0007] In some embodiments of the present application, based on the aforementioned scheme, the matching unit is configured to: intercept text characters from the text to be clustered according to a sliding window length of a set size, and match the intercepted text characters with the clustering keywords corresponding to each of the clustering categories to obtain a matching result between the text characters and the clustering keywords corresponding to each of the clustering categories; determine the matching degree between the text to be clustered and each of the clustering categories based on the matching result between the text characters and the clustering keywords corresponding to each of the clustering categories.

[0008] In some embodiments of the present application, based on the aforementioned scheme, the matching unit is configured to: determine at least one matching degree between the text to be clustered and each of the clustering categories based on the matching content contained in the matching result; if multiple matching degrees between the text to be clustered and each of the clustering categories are determined, then perform weighted summation processing on the multiple matching degrees to obtain the final matching degree between the text to be clustered and each of the clustering categories.

[0009] In some embodiments of the present application, based on the aforementioned scheme, at least one matching degree between the text to be clustered and each of the clustering categories is determined according to the matching content contained in the matching results, including: determining the character length in the text to be clustered that matches the clustering keyword corresponding to each of the clustering categories according to the matching results; and determining the first matching degree between the text to be clustered and each of the clustering categories according to the ratio between the matched character length and the character length of the text to be clustered.

[0010] In some embodiments of the present application, based on the aforementioned scheme, at least one matching degree between the text to be clustered and each of the clustering categories is determined according to the matching content contained in the matching results, including: determining the edit distance between the text characters and the clustering keywords corresponding to each of the clustering categories according to the matching results; normalizing the edit distance, and determining the second matching degree between the text to be clustered and each of the clustering categories according to the result of the normalization.

[0011] In some embodiments of the present application, based on the aforementioned scheme, the edit distance is normalized, including: determining the maximum value of the character length based on the character length of the text characters and the character length of the clustering keywords corresponding to each of the clustering categories; calculating the ratio between the edit distance and the maximum value of the character length to obtain the result of the normalization processing.

[0012] In some embodiments of the present application, based on the aforementioned scheme, at least one matching degree between the text to be clustered and each of the clustering categories is determined according to the matching content contained in the matching results, including: determining the word frequency and inverse document frequency of the clustering keywords corresponding to each of the clustering categories in the text to be clustered according to the matching results; and determining a third matching degree between the text to be clustered and each of the clustering categories according to the product of the word frequency and the inverse document frequency.

[0013] In some embodiments of the present application, based on the aforementioned solution, the division unit is configured to: divide the text to be clustered into the cluster type with the highest matching degree according to the matching degree between the text to be clustered and each of the cluster categories.

[0014] In some embodiments of the present application, based on the aforementioned scheme, the processing unit is configured to: extract the feature vector corresponding to the text to be clustered in each clustering category through a pre-trained student model; wherein the text clustering device also includes: a model training unit, configured to: obtain a sample text for training the student model, input the sample text into the student model, and obtain a first feature vector output by the student model for the sample text; input the sample text and prompt information for summarizing the content of the sample text into the teacher model, and obtain a second feature vector output by the teacher model; calculate the model loss corresponding to the student model based on the first feature vector and the second feature vector, and adjust the model parameters of the student model based on the model loss.

[0015] In some embodiments of the present application, based on the aforementioned scheme, the model training unit is configured to: perform normalization on the first eigenvector and the second eigenvector respectively to obtain the first normalized eigenvector and the second normalized eigenvector corresponding to the sample text; calculate the first similarity matrix between the first normalized eigenvectors corresponding to a batch of sample texts, and the second similarity matrix between the second normalized eigenvectors corresponding to a batch of sample texts; calculate the relative entropy model loss corresponding to the student model based on the first similarity matrix and the second similarity matrix.

[0016] In some embodiments of the present application, based on the aforementioned scheme, the processing unit is configured to: calculate the similarity between the texts to be clustered in each clustering category according to the feature vector corresponding to the texts to be clustered in each clustering category; construct an undirected graph corresponding to each clustering category according to the similarity between the texts to be clustered in each clustering category, the nodes in the undirected graph represent the texts to be clustered, and the edges in the undirected graph represent that the similarity between the texts to be clustered is greater than or equal to a set threshold; select a target node as the clustering center according to the degree of each node in the undirected graph, and cluster the target node and the nodes directly connected to the target node into one category to obtain at least one node cluster; determine the clustering result of the texts to be clustered according to the at least one node cluster and the texts to be clustered represented by the nodes in the node cluster.

[0017] In some embodiments of the present application, based on the aforementioned scheme, the processing unit is configured to: select the node with the largest degree as a target node according to the degree of each node in the undirected graph; after clustering the selected target node and the nodes directly connected to the target node into one category, select the node with the largest degree again from the remaining nodes of the undirected graph as a target node, and perform clustering processing on the remaining nodes until the node traversal in the undirected graph is completed.

[0018] In some embodiments of the present application, based on the aforementioned scheme, the processing unit is further configured to: after obtaining the clustering results of the text to be clustered, extract the keywords in each of the clustering results; and update the clustering keywords of the clustering category corresponding to each of the clustering results based on the keywords extracted from each of the clustering results.

[0019] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the text clustering method as described in the above embodiment is implemented.

[0020] According to one aspect of an embodiment of the present application, a computer device is provided, comprising: one or more processors; and a storage device for storing one or more computer programs, wherein when the one or more computer programs are executed by the one or more processors, the computer device implements the text clustering method as described in the above embodiment.

[0021] According to one aspect of an embodiment of the present application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads and executes the computer program from the computer-readable storage medium, causing the computer device to perform the text clustering method provided in the various optional embodiments described above.

[0022] In the technical solutions provided in some embodiments of the present application, after obtaining the text to be clustered, the text to be clustered can be matched with clustering keywords corresponding to at least two clustering categories respectively to obtain the matching degree between the text to be clustered and each clustering category. Then, according to the matching degree between the text to be clustered and each clustering category, the text to be clustered can be divided into the corresponding clustering category. This method can use clustering keywords to quickly and roughly classify massive texts to be clustered, thereby quickly reducing the amount of data to be processed for each clustering category, greatly reducing the subsequent computational burden of fine-grained clustering of the text to be clustered in each clustering category, improving the data processing speed, and reducing the demand for computing resources. By extracting the feature vectors corresponding to the text to be clustered in each clustering category, and clustering the text to be clustered in each clustering category according to the feature vectors corresponding to the text to be clustered, the clustering results of the text to be clustered are obtained, so that the deep semantic associations between the texts to be clustered can be captured, thereby achieving more accurate clustering results. It can be seen that the technical solution of the embodiment of the present application first performs rapid coarse classification through the domain keyword table, and then vectorizes the text to be clustered in each cluster category and performs fine-grained clustering division, which solves the problems of high resource consumption and insufficient fine-grained accuracy in related technologies, and effectively improves the accuracy and efficiency of text clustering.

[0023] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied;

[0025] Figure 2 A flowchart of a text clustering method according to an embodiment of the present application is shown;

[0026] Figure 3 A flow chart showing a method for training a student model according to an embodiment of the present application is shown;

[0027] Figure 4 A flowchart of a text clustering method according to an embodiment of the present application is shown;

[0028] Figure 5 A schematic diagram of the architecture of a lightweight model obtained by training according to an embodiment of the present application is shown;

[0029] Figure 6 A schematic diagram of a two-layer clustering architecture according to an embodiment of the present application is shown;

[0030] Figure 7 A flowchart of a fine-grained clustering process according to an embodiment of the present application is shown;

[0031] Figure 8 A block diagram of a text clustering device according to an embodiment of the present application is shown;

[0032] Figure 9 A schematic diagram of the structure of a computer system suitable for implementing a computer device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0033] Example embodiments will now be described in a more complete manner with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to these examples; rather, these embodiments are provided to make this application more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art.

[0034] In addition, the features, structures or characteristics described in the present application may be combined in one or more embodiments in any suitable manner. In the following description, there are many specific details so that the embodiments of the present application can be fully understood. However, it will be appreciated by those skilled in the art that when implementing the technical solution of the present application, it is not necessary to use all the detailed features in the embodiments, one or more specific details may be omitted, or other methods, elements, devices, steps, etc. may be adopted.

[0035] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0036] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0037] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0038] It should be noted that the term "plurality" used in this document refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0039] It is understandable that this application can display a prompt interface or pop-up window before collecting relevant user data (such as comment data, barrage data, and other text data that need to be reviewed) and during the process of collecting relevant user data. The prompt interface or pop-up window is used to prompt the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining user-related data after obtaining the user's confirmation operation on the prompt interface or pop-up window. Otherwise (that is, when the user's confirmation operation on the prompt interface or pop-up window is not obtained), the relevant steps of obtaining user-related data are terminated, that is, the user's relevant data is not obtained. In other words, all user data collected by this application are collected with the user's consent and authorization, and the collection, use and processing of relevant user data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0040] The technical solution of the embodiment of this application mainly involves the field of natural language processing, and is particularly suitable for feature extraction and hierarchical clustering scenarios of massive texts (such as user comments, bullet comments, etc.), aiming to solve problems such as waste of audit resources, insufficient semantic understanding, and low computational efficiency. Specifically, due to the massive amount of repeated or similar texts, a large amount of manpower is inefficiently dissipated in processing texts to be audited with highly similar content. Therefore, by accurately clustering similar texts and conducting centralized audits, the audit efficiency can be significantly improved and resource allocation can be optimized.

[0041] In related technologies, text clustering schemes usually first preset the number of clusters K to be clustered, and then divide the text to be clustered into K clusters through iterative optimization to maximize the text similarity within each cluster. Although the clustering calculation speed of this scheme is faster, the clustering effect mainly depends on the selection of K. In actual business, K is difficult to preset. Therefore, in actual business, the clustering effect is unstable and it is easy to miss or mis-cluster.

[0042] At the same time, when clustering text, it is necessary to extract the text's feature vectors to calculate the similarity between texts. Related technologies typically use pre-trained semantic vector models to generate vector representations of text. For example, the models used are context-sensitive models such as BERT and text2vec. Although this solution is lightweight and efficient, pre-trained semantic vector models lack deep semantic understanding and are generally only suitable for basic semantic tasks. This is because the semantic understanding capabilities of pre-trained semantic vector models are inherently limited by the pre-training tasks and training data. When faced with highly implicit semantics, rhetorical techniques, domain-specific terminology, or emerging internet jargon, the vectors they generate struggle to fully capture deep semantic connections, resulting in limited accuracy in fine-grained clustering results (for example, the inability to distinguish between an ironic "good" and a genuine "good").

[0043] Alternatively, a large language model (LLM) can be used to extract feature vectors of text. LLM is an artificial intelligence model trained on massive amounts of text data with powerful natural language understanding and generation capabilities. The core of its vector extraction lies in dynamically generating semantic representations using the model's internal hidden states. In specific implementation, the LLM is generally guided to output a hidden layer state vector through statements such as "extract the semantic vector of this sentence", and then the decoder's last hidden state is taken as the text representation. The vector features based on LLM have strong perception of deep semantics and can significantly improve the performance of complex tasks such as polysemy and coreference resolution. However, the main limitation of LLM is its high deployment and computing costs. This is because the LLM model has a large number of parameters and extremely high inference computational complexity. Directly using it to process the vectorization and large-scale similarity calculations of massive amounts of text requires a very large amount of graphics processing unit (GPU) computing power, making it unsuitable for online high-concurrency scenarios that are sensitive to both timeliness and resources.

[0044] Based on the above technical problems, a new text clustering scheme is proposed in the embodiment of this application. It can first perform rapid coarse classification through the domain keyword table, and then vectorize the text to be clustered in each cluster category and perform fine-grained clustering division, effectively improving the accuracy and efficiency of text clustering. At the same time, through knowledge distillation, the LLM can be migrated to a lightweight pre-trained vector model with a size of about one twentieth, thereby significantly reducing computing resource consumption while retaining the model's deep semantic understanding ability, further improving the accuracy of text clustering.

[0045] Specifically, if Figure 1 As shown, in a system architecture 100 applying the technical solution of an embodiment of the present application, a terminal device 110, a network 120 and a server 130 may be included. The terminal device 110 may include a smart phone, a tablet computer, a laptop computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, and the like. The server 130 may be a server that provides various services. It may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Among them, the network 120 may be a medium that provides a communication connection between the terminal device 110 and the server 130, such as a wired communication link or a wireless communication link.

[0046] According to implementation requirements, the system architecture in the embodiment of the present application may have any number of terminal devices 110, networks 120, and servers 130. For example, the server 130 may be a server group composed of multiple server devices.

[0047] In one embodiment of the present application, a user can use a terminal device 110 to publish news comments, video barrages and other text information through a network 120. These text information are obtained by a server 130 as text to be clustered. After obtaining these texts to be clustered, the server 130 can match the text to be clustered with clustering keywords corresponding to at least two pre-set clustering categories to obtain the matching degree between the text to be clustered and each clustering category. Then, according to the matching degree between the text to be clustered and each clustering category, the text to be clustered can be divided into corresponding clustering categories. This method can use clustering keywords to quickly and roughly classify massive texts to be clustered, thereby quickly reducing the amount of data that needs to be processed in each clustering category, greatly reducing the computational burden of subsequent fine-grained clustering of the text to be clustered in each clustering category. For example, in the review scenario of video comments or video barrage, the clustering categories set can be "praise" and "complaint". The clustering keywords corresponding to the "praise" category can be "good-looking", "wonderful", "great special effects", "logical and rigorous", etc.; the clustering keywords corresponding to the "complaint" category can be "ugly", "not good-looking", "the special effects are too fake", "the plot has flaws", etc.

[0048] After the texts to be clustered are divided into corresponding cluster categories, the server 130 can extract the feature vectors corresponding to the texts to be clustered in each cluster category for each cluster category, and cluster the texts to be clustered in each cluster category according to the feature vectors corresponding to the texts to be clustered, and obtain the clustering results of the texts to be clustered. This method can capture the deep semantic associations between the texts to be clustered, thereby achieving more accurate clustering results. For example, in the above-mentioned video comment or video barrage review scenario, for the texts to be clustered in the "Tucao" category, further clustering can be performed based on the feature vectors to determine whether the user is complaining about the plot or the special effects, and then obtain refined clustering results: "Tucao plot" category and "Tucao special effects" category.

[0049] In addition to the review scenarios of video comments or video barrages listed above, the technical solutions of the embodiments of the present application can also be applied to the review scenarios of user-generated content, such as in the comment area of ​​self-media articles, which are full of advertising traffic-generating content (such as "Add my contact information xxx for a private chat"), and the variations are varied (such as symbol replacement, use of homophonic characters, etc.). When adopting the technical solutions of the embodiments of the present application, coarse-grained clustering categories can be first performed by matching the traffic-generating keyword table. For example, the traffic-generating keyword table can include: + contact information, add my contact information, add contact information, click on my avatar, etc. After the coarse-grained clustering category division, fine-grained clustering processing can be performed based on the feature vector of the text, such as identifying "+ contact information xxx" and "add my contact information xxx" as the same cluster. This is because the semantics between the two texts are consistent. This method can effectively improve the clustering accuracy of advertising traffic-generating texts and effectively improve the review efficiency of user-generated content.

[0050] In one application scenario of the present application, the technical solution of the embodiment of the present application can also be applied to comment sentiment analysis. For example, for the comments on news articles, the comments can be divided into coarse-grained cluster categories according to the theme through preset clustering keywords, such as "artificial intelligence", "intangible cultural heritage inheritance", etc. After the coarse-grained clustering category division, fine-grained clustering processing can be performed based on the feature vector of the text. For example, for the comments on "artificial intelligence", clusters of supporters can be obtained (such as artificial intelligence promotes work efficiency and improves the convenience of life, etc.), clusters of opponents (such as artificial intelligence cannot think like humans, has limitations, uncontrollable, etc.) and clusters of neutral parties (such as the effect still needs to be observed).

[0051] In an application scenario of the present application, the technical solution of the embodiment of the present application can also be applied to the classification of content labels. For example, for film and television dramas, the plot summary or user comments can be coarse-grained clustered categories by pre-set clustering keywords to obtain coarse-grained labels for the film and television drama. Specifically, for example, the keywords of cluster 1 may include: reversal, brain-burning, etc., then the film and television drama can be labeled as "suspense", and the keywords of cluster 2 may include: sweetness, sugar-sprinkling, etc., then the film and television drama can be labeled as "love". After the coarse-grained clustering category division, fine-grained clustering processing can be performed based on the text feature vectors of the plot summary or user comments. For example, for film and television dramas labeled as "suspense", fine-grained content labels such as "ancient costume suspense", "modern suspense", etc. can be obtained through fine-grained clustering processing of user comments.

[0052] In an application scenario of the present application, the technical solution of the embodiment of the present application can also be applied to product experience feedback. For example, for an application being developed, user feedback can be coarse-grained clustered categories using preset clustering keywords, such as using the names of the functional modules of the application as clustering keywords (such as "payment", "interface", etc.). After performing coarse-grained clustering categories, the functional modules of the user feedback can be located, and then fine-grained clustering processing can be performed based on the text feature vectors of the user feedback. For example, for user feedback on "payment", cluster 1 (such as payment keeps spinning in circles, etc., the problem of this cluster may be a server failure) and cluster 2 (no response after scanning the code, etc., the problem of this cluster may be a front-end compatibility problem) can be obtained. In this way, the core problems of the application can be quickly located, and the technical team can be encouraged to prioritize repairing high-frequency faults.

[0053] The following is a detailed description of the implementation details of the technical solution of the embodiment of the present application:

[0054] Figure 2 The flowchart of the text clustering method according to one embodiment of the present application is shown. The text clustering method can be executed by a computer device, which can be a server, a terminal device, or other devices with computing and processing functions. Figure 2 As shown, the text clustering method includes at least S210 to S240, which are described in detail as follows:

[0055] In S210 , text to be clustered and clustering keywords corresponding to at least two clustering categories are obtained.

[0056] In some optional embodiments, obtaining texts to be clustered refers to collecting or preparing a series of text data that need to be clustered. These texts can come from various channels, such as comments on social media, messages under news articles, and comments on video platforms. The goal of obtaining texts to be clustered is to analyze and group these texts for better understanding and management.

[0057] In some optional embodiments, the cluster keywords corresponding to at least two cluster categories may be preset. For example, a set of representative keywords may be determined for at least two different cluster categories. Since each cluster category has its own specific theme or characteristics, these keywords are used to identify and distinguish different categories. For example, if there are two cluster categories, namely "sports" and "entertainment", then the keywords corresponding to the "sports" category may include "game", "athlete", "score", etc.; and the keywords corresponding to the "entertainment" category may include "movie", "star", "concert", etc.

[0058] In S220 , the text to be clustered is matched with clustering keywords corresponding to at least two clustering categories, and a matching degree between the text to be clustered and each clustering category is obtained.

[0059] In some optional embodiments, matching the text to be clustered with the clustering keywords corresponding to at least two clustering categories refers to comparing and analyzing each text to be clustered with the clustering keywords corresponding to each clustering category. For example, if there are two clustering categories, namely "sports" and "entertainment", then each text to be clustered needs to be matched with the keywords corresponding to the "sports" category (such as "game", "athlete", etc.) and the keywords corresponding to the "entertainment" category (such as "movie", "star", etc.). Among them, the matching degree between the text to be clustered and each clustering category refers to the similarity or correlation score between the text to be clustered and each clustering category, which reflects the degree of fit between the text to be clustered and a specific clustering category. Optionally, the matching degree can be calculated based on a variety of methods, such as through word frequency statistics, edit distance, term frequency-inverse document frequency (TF-IDF) weight, or semantic similarity algorithm.

[0060] In some optional embodiments, text characters can be intercepted from the text to be clustered according to a set sliding window length, and the intercepted text characters can be matched with the clustering keywords corresponding to each clustering category to obtain the matching results between the text characters and the clustering keywords corresponding to each clustering category. Then, based on the matching results between the text characters and the clustering keywords corresponding to each clustering category, the matching degree between the text to be clustered and each clustering category is determined.

[0061] It should be noted that the concepts related to the sliding window include the size and step size of the sliding window, wherein the size of the sliding window refers to the length of the data segment intercepted from the text to be clustered each time, and the step size refers to the interval between the starting position of the data between two adjacent interception operations. The step size can be equal to the window size (no overlap) or less than the window size (with overlap). The sliding window solution specifically refers to "sliding" a window of a set size on a text to be clustered to extract characters from the text. This method can effectively identify local features of the text. For example, if the size of the sliding window is 3 characters and the step size is 1 character, then for the text to be clustered "this video is very good", the text characters intercepted for the first time are "this video", the text characters intercepted for the second time are "a video", and the text characters intercepted for the third time are "the video is very", and so on (the characters in this example are illustrated as if they do not contain double quotes).

[0062] In some optional embodiments, the matching results between text characters and clustering keywords corresponding to each clustering category may include at least one of the following information: the length of characters in the text to be clustered that match the clustering keywords corresponding to each clustering category, the edit distance between the text characters and the clustering keywords corresponding to each clustering category, the word frequency and inverse document frequency of the clustering keywords corresponding to each clustering category in the text to be clustered, etc.

[0063] In some optional embodiments, the degree of matching between the text to be clustered and each cluster category is determined based on the matching results between the text characters and the clustering keywords corresponding to each cluster category. Specifically, at least one degree of matching between the text to be clustered and each cluster category is determined based on the matching content included in the matching results. If a single degree of matching between the text to be clustered and each cluster category is determined, this single degree of matching can be directly used as the final degree of matching between the text to be clustered and each cluster category. If multiple degrees of matching between the text to be clustered and each cluster category are determined, the multiple degrees of matching can be weighted and summed to obtain the final degree of matching between the text to be clustered and each cluster category.

[0064] In some optional embodiments, at least one matching degree between the text to be clustered and each clustering category may include one or more of the first matching degree, the second matching degree, and the third matching degree introduced in the following embodiments. The calculation processes of the first matching degree, the second matching degree, and the third matching degree are introduced below respectively.

[0065] In some optional embodiments, when calculating the first degree of matching, the length of characters that match the clustering keywords corresponding to each clustering category in the text to be clustered can be determined based on the matching results between the text characters and the clustering keywords corresponding to each clustering category, and then the first degree of matching between the text to be clustered and each clustering category can be determined based on the ratio between the matched character length and the character length of the text to be clustered. Optionally, the ratio can be used directly as the first degree of matching, or the ratio can be adjusted (such as increasing a certain value, decreasing a certain value, multiplying by a set multiple, etc.) and then used as the first degree of matching. For example, if a text to be clustered is "This video is good-looking", and the clustering keywords corresponding to a certain clustering category include "good-looking", then the character length that matches the text to be clustered with the clustering category is 2, and since the character length of the text to be clustered is 7, the first degree of matching between the text to be clustered and the clustering category can be expressed as 2 / 7.

[0066] In some optional embodiments, when calculating the second matching degree, the edit distance between the text characters and the clustering keywords corresponding to each clustering category can be determined based on the matching results between the text characters and the clustering keywords corresponding to each clustering category, and then the edit distance is normalized, and the second matching degree between the text to be clustered and each clustering category is determined based on the result of the normalization. For example, if a text to be clustered is "This video is very good", the text characters intercepted by the sliding window are "The video is very good", and the clustering keywords corresponding to a certain clustering category include "The video is very good", then the edit distance between the text characters and the clustering keywords is 1.

[0067] Optionally, when normalizing the edit distance, the maximum value of the character length can be determined based on the character length of the text characters and the character length of the clustering keywords corresponding to each clustering category, and then the ratio between the edit distance and the maximum value of the character length is calculated to obtain the normalized result. Continuing with the above example, the character length of the intercepted text characters is 4 and the character length of the clustering keywords is also 4, so the maximum value of the character length is 4, then the ratio between the edit distance and the maximum value of the character length is 1 / 4. In this embodiment, by normalizing the edit distance, the calculation results of the matching degree can be standardized to ensure the rationality of the matching degree comparison.

[0068] Optionally, the second matching degree between the text to be clustered and each cluster category is determined based on the result of the normalization process, which may be the difference between 1 and the result of the normalization process as the second matching degree, or the difference may be adjusted (such as increasing a certain value, decreasing a certain value, multiplying by a set multiple, etc.) as the second matching degree.

[0069] In some optional embodiments, when calculating the third matching degree, the word frequency and inverse document frequency of the clustering keywords corresponding to each clustering category in the text to be clustered can be determined based on the matching results between the text characters and the clustering keywords corresponding to each clustering category, and then the third matching degree between the text to be clustered and each clustering category can be determined based on the product of the word frequency and the inverse document frequency.

[0070] It should be noted that the word frequency of the clustering keyword corresponding to each clustering category in the text to be clustered refers to the frequency of the clustering keyword appearing in the text to be clustered. This value reflects the general importance of the clustering keyword in the text to be clustered. The inverse document frequency means that if a clustering keyword appears in many texts to be clustered, then it may not be particularly discriminatory. By combining the word frequency and the inverse document frequency, the importance weight of a clustering keyword to the text to be clustered can be obtained. Optionally, the product between the word frequency and the inverse document frequency can be used as the third matching degree, or the product can be adjusted (such as increasing a certain value, decreasing a certain value, multiplying by a set multiple, etc.) and used as the third matching degree.

[0071] Continue to refer to Figure 2 As shown, in S230, the text to be clustered is divided into at least two cluster categories according to the matching degree between the text to be clustered and each cluster category.

[0072] In some optional embodiments, each text to be clustered may be divided into a cluster type with the highest matching degree according to the matching degree between the text to be clustered and each cluster category.

[0073] In S240 , the feature vectors corresponding to the texts to be clustered in each cluster category are extracted, and clustering processing is performed on the texts to be clustered in each cluster category according to the feature vectors corresponding to the texts to be clustered, to obtain a clustering result of the texts to be clustered.

[0074] In some optional embodiments, a pre-trained model can be used to extract feature vectors corresponding to the text to be clustered in each cluster category. The feature vector corresponding to the text to be clustered refers to a numerical vector that represents the semantic information or other important features of the text to be clustered. This vector typically has a fixed dimension, and the value of each dimension reflects the characteristics of the text to be clustered in a specific aspect.

[0075] Alternatively, the pre-trained model can be a pre-trained student model, which can be obtained by migrating the teacher model through knowledge distillation. This can significantly reduce computing resource consumption while retaining the teacher model's deep semantic understanding capabilities, further improving the accuracy of text clustering. Alternatively, the teacher model can be a large language model.

[0076] In some optional embodiments, the training method of the student model can be as follows Figure 3 As shown, the following steps are included:

[0077] S310, obtaining a sample text for training a student model, inputting the sample text into the student model, and obtaining a first feature vector output by the student model for the sample text.

[0078] Optionally, the sample text can be real data collected from actual application scenarios or part of a processed dataset, such as collected comment data, barrage data, etc. After the sample text is input into the lightweight student model, the student model generates a feature vector representation (also known as the first eigenvector) for each sample text. This feature vector is a mathematical description of the semantics of the sample text and usually has a fixed dimension (e.g., 768 dimensions).

[0079] S320: Input the sample text and the prompt information for summarizing the content of the sample text into the teacher model to obtain a second feature vector output by the teacher model.

[0080] It should be noted that the teacher model, due to its large number of parameters and complex structure, can capture deeper semantic information than the student model. In an embodiment of the present application, for the teacher model, not only does the same sample text need to be input into the teacher model, but it also needs to be fed into the teacher model together with a specific prompt information. The role of the prompt information is to guide the teacher model to focus on extracting the core meaning or summary content of the sample text, and then the teacher model generates a more refined and in-depth feature representation (i.e., the second eigenvector) based on the input sample text and prompt information. The eigenvector is a mathematical representation of the summarized content.

[0081] S330, calculating a model loss corresponding to the student model according to the first eigenvector and the second eigenvector, and adjusting model parameters of the student model according to the model loss.

[0082] It should be noted that in an embodiment of the present application, in order to enable the student model to learn the capabilities of the teacher model, the difference between the first eigenvector generated by the student model and the second eigenvector generated by the teacher model (i.e., model loss) can be compared, and then the error is back-propagated based on the calculated model loss to adjust the parameters of the student model so that its output is as close as possible to the output of the teacher model, thereby achieving knowledge transfer.

[0083] In a specific example, suppose a lightweight feature extraction model (student model) is being trained to perform well in resource-constrained environments. A large pre-trained language model (teacher model) can be leveraged to help it better understand text. Specifically, for a sample text entry, "The food at this restaurant is delicious," the student model might generate a 768-dimensional first feature vector. Simultaneously, this sample text, along with a prompt: "Summarize the review in one sentence," is fed into the teacher model. Based on this input, the teacher model generates a more detailed second feature vector. Next, these two feature vectors are compared, the difference between them calculated (model loss), and the student model's weights are updated using an optimization algorithm (such as gradient descent). This ensures that the student model produces results closer to those of the teacher model the next time it encounters similar input. This approach significantly reduces computing resource requirements while maintaining high performance, making it ideal for text processing tasks in high-concurrency online scenarios.

[0084] In some optional embodiments, the process of calculating the model loss corresponding to the student model based on the first eigenvector and the second eigenvector in S330 may specifically include: normalizing the first eigenvector and the second eigenvector respectively to obtain the first normalized eigenvector and the second normalized eigenvector corresponding to the sample text, and then calculating the first similarity matrix between the first normalized eigenvectors corresponding to a batch of sample texts, and the second similarity matrix between the second normalized eigenvectors corresponding to a batch of sample texts, and then calculating the relative entropy model loss corresponding to the student model based on the first similarity matrix and the second similarity matrix.

[0085] Optionally, the first eigenvector and the second eigenvector are normalized separately, and L2 normalization, also known as unit vector normalization, is used to make the length of each eigenvector 1. This process helps to eliminate the differences in the scales of different vectors and ensures more accurate similarity calculations. For the sample texts in a batch, based on their corresponding first normalized eigenvectors, a similarity matrix can be formed by calculating the cosine similarity between the eigenvectors of each pair of sample texts (or other similarity calculation methods). For example, if the batch size is 16, a 16×16 first similarity matrix will be obtained, and each element in the first similarity matrix represents the degree of similarity between the two sample texts from the perspective of the student model.

[0086] Similarly, for sample texts within the same batch, based on their corresponding second normalized feature vectors, the similarity between each pair of second normalized feature vectors can be calculated to form a second similarity matrix. For example, if the batch size is 16, a 16×16 second similarity matrix will be obtained, reflecting the similarity relationship between texts from the perspective of the teacher model.

[0087] It should be noted that relative entropy can also be called KL divergence (Kullback-Leibler Divergence), which is used to measure the difference between two probability distributions. In an embodiment of the present application, the first similarity matrix and the second similarity matrix can be regarded as two different probability distributions, and the KL divergence value between them is calculated as the model loss. The goal of training the student model is to minimize this model loss, that is, to make the first similarity matrix as close as possible to the second similarity matrix, so that the student model can learn the semantic representation ability of the teacher model. Optionally, during the training of the student model, the model parameters of the teacher model can remain unchanged.

[0088] In some optional embodiments, such as Figure 4 As shown, the process of clustering the texts to be clustered in each cluster category according to the feature vectors corresponding to the texts to be clustered may include the following steps:

[0089] S410 , calculating the similarity between the texts to be clustered in each cluster category according to the feature vectors corresponding to the texts to be clustered in each cluster category.

[0090] In some optional embodiments, the cosine similarity or Euclidean distance between the feature vectors of the texts to be clustered in each clustering category can be calculated as the similarity between the texts to be clustered. If the similarity between two texts to be clustered is high, it means that the two texts to be clustered are more similar and are more suitable to be classified into one category.

[0091] S420: construct an undirected graph corresponding to each clustering category based on the similarity between the texts to be clustered in each clustering category, wherein the nodes in the undirected graph represent the texts to be clustered, and the edges in the undirected graph represent that the similarity between the texts to be clustered is greater than or equal to a set threshold.

[0092] In this step, the similarity information between the texts to be clustered can be converted into a graph structure to facilitate clustering using a graph algorithm, where each node in the constructed undirected graph represents a text to be clustered. If the similarity between two texts to be clustered is greater than or equal to a preset threshold (for example, 0.8), then an edge can be added between the corresponding two nodes; if the similarity between two texts to be clustered is less than the preset threshold (for example, 0.8), then there is no edge between the two nodes, thus forming a graph structure reflecting the similarity relationship between texts.

[0093] S430 , selecting a target node as a cluster center according to the degree of each node in the undirected graph, and clustering the target node and the nodes directly connected to the target node into one category to obtain at least one node cluster.

[0094] It should be noted that in an undirected graph, the number of edges a node has is called the degree of the node. In other words, the degree of each node is the number of edges directly connected to the node (that is, how many other texts have a high degree of similarity with this text). If a node's degree is higher, it means that there are more other texts similar to the text represented by the node. In this case, the node can be used as a cluster center, and the nodes directly connected to the node can be clustered into one category.

[0095] In specific implementation, based on the degrees of the nodes in the undirected graph, the node with the largest degree is first selected as a target node. The selected target node and the nodes directly connected to it are then clustered into one category. Afterwards, the node with the largest degree is again selected from the remaining nodes in the undirected graph as a target node. The remaining nodes are clustered according to the above strategy, and this process is repeated until all nodes in the undirected graph are traversed.

[0096] S440 : Determine a clustering result of the text to be clustered based on at least one node cluster and the text to be clustered represented by the nodes in the node cluster.

[0097] It should be noted that since the nodes in each node cluster represent a group of highly similar texts to be clustered, the final clustering results of the texts to be clustered can be determined based on these node clusters. For example, if a node cluster contains multiple reviews about "sports events", then all the texts in the node cluster can be considered to belong to the cluster category "sports".

[0098] In some optional embodiments, after obtaining clustering results for the text to be clustered, keywords from each clustering result can be extracted, and then the clustering keywords for the clustering category corresponding to each clustering result can be updated based on the keywords extracted from each clustering result. In this way, the accuracy and adaptability of the clustering process can be enhanced, ensuring that the clustering keywords can effectively respond to the ever-changing language environment and the development of emerging fields.

[0099] Optionally, when extracting keywords from each cluster, all text within each cluster can be preprocessed, such as removing stop words and punctuation. Appropriate algorithms or tools (such as TF-IDF and TextRank) can then be used to extract keywords from the text that reflect the core theme or main content of the cluster. For example, if a cluster contains multiple reviews about "smartphones," the extracted keywords might include "mobile phone," "photography," and "battery life."

[0100] When updating the cluster keywords of the cluster category corresponding to each cluster result based on the keywords extracted from each cluster result, the keywords extracted from all the cluster results within each cluster category can be summarized, and then the summarized keywords can be filtered and merged to remove duplicates and sorted according to the importance of the keywords (such as frequency, weight, etc.). The filtered and merged keyword list is then added to the corresponding cluster category to update the original cluster keyword set. For example, suppose the original "technology" category keywords include "computer" and "Internet". After this update, new keywords such as "smartphone" and "5G technology" have been added, making the cluster keywords more comprehensive.

[0101] The following takes a specific application scenario as an example, combined with Figures 5 to 7 The implementation details of the technical solution of the embodiment of this application are elaborated in detail:

[0102] The technical solution of the embodiment of the present application mainly involves a method for calculating text similarity and a hierarchical clustering method based thereon. Among them, when calculating text similarity, a lightweight model trained by knowledge distillation can be used to extract the feature vector of the text. Specifically, the large language model is guided to perform deep semantic compression tasks through instructional prompt information, and the "summary word" embedding vector predicted by the model is used as a high-density text representation; at the same time, feature space-oriented knowledge distillation is used to transfer the semantic compression capability of the large model to the lightweight model, realizing efficient online deployment, thereby generating a universal and high-precision text vector comparable to the large model in resource-constrained scenarios, solving the dual dilemma of weak semantic understanding of traditional vector models and high LLM deployment costs.

[0103] The hierarchical clustering method proposed in the embodiment of this application is mainly a two-layer clustering architecture, which mainly adopts a vocabulary-guided coarse-grained classification and a vector-optimized fine-grained clustering collaborative mechanism: first, a predefined domain keyword table is used to quickly and roughly classify massive texts, greatly reducing the amount of subsequent processing; then, the text within each coarse category is vectorized and fine-grained clustering is performed based on similarity. This method can effectively reduce computational overhead, while taking into account the efficiency of large-scale text processing and fine-grained semantic accuracy through hierarchical optimization, and supports adapting to new terminology scenarios by dynamically updating the vocabulary.

[0104] Specifically, if Figure 5 The figure shows the architecture diagram of the lightweight model trained based on prompt information and knowledge distillation, which mainly includes the input layer, model layer and knowledge alignment layer.

[0105] In the embodiment of the present application, the content of the input layer of the teacher model and the student model is different due to the difference in the architecture of the teacher model and the student model. For the student model, the input is the original text of the feature vector to be extracted in the data set, such as Figure 5 As shown in the figure, "Add me at: xxx"; for the teacher model, considering the effect of knowledge distillation, in the embodiment of the present application, a dynamic prompting project is used to guide the teacher model to compress the semantics of a long text into a single token. Specifically, the teacher model can be an LLM model. Based on the autoregressive generation characteristics of the LLM, a specific prompt information is superimposed on the original text to guide the teacher model to compress the information of the original text into the hidden state of the last token. The prompt information can be "summarize the above content in one word" or other information used to summarize the input original text.

[0106] Optionally, in the model layer, the embodiment of the present application adopts a contrastive learning method, which specifically requires obtaining the similarity matrix between the student model and the teacher model in each batch.

[0107] Specifically, the last hidden layer of the student model is pooled to obtain the compressed vector h student , the vector size corresponding to a sample text in each batch can be [1,768]; then the vector h student Perform L2 normalization to obtain Finally, calculate the similarity matrix between sample texts in each batch Among them, the similarity matrix S student Elements in B represents the size of a batch, that is, the number of sample texts contained in a batch.

[0108] For the teacher model, the hidden state vector of the last Token can be taken in this embodiment of the application. Among them, d t It can be 4098, that is, the vector size is 4098 dimensions; the same can be done for h teacher Perform L2 normalization and calculate the similarity matrix Among them, the similarity matrix S teacher Elements in

[0109] In actual training, if the batch size is 16 and the vector size is 4098 dimensions, the corresponding output vector size is a similarity matrix of [16, 16].

[0110] Continue to refer to Figure 5 As shown, in the knowledge alignment stage, the student model needs to fit the output of the teacher model as closely as possible. By aligning the similarity matrix between the teacher model and the student model, the student model learns the semantic metric space of the teacher model, thereby completing the knowledge alignment process. In this embodiment of the application, KL divergence can be used as a loss function to measure the difference in the similarity matrix between the student model and the teacher model. Optionally, the loss function It can be expressed as follows:

[0111]

[0112] The core of the knowledge alignment phase is to minimize The output distribution of the student model is made close to the teacher model. During the whole process, the parameters of the teacher model are fixed, that is, no fine-tuning is performed, and the parameters of the student model can participate in the training and be optimized through back propagation. Finally, the knowledge transfer is completed. Optionally, the teacher model can choose the qwen3 7B model, and the student model can choose the text2vec vector model.

[0113] The two-layer clustering architecture proposed in the embodiment of this application is as follows Figure 6 As shown in the figure, it mainly includes coarse-grained classification process and fine-grained clustering process. Among them, the coarse-grained classification process is mainly based on the domain keyword table to quickly classify massive texts; the fine-grained clustering process is mainly based on the text vector representation to perform accurate semantic clustering processing. Overall, through the divide-and-conquer strategy, the time complexity is reduced from O(N 2 ) is reduced to O(M×K 2 ), where N is the total sample size, M is the number of coarse classifications, and K is the maximum sample size within a class (K< <N)。

[0114] In an embodiment of the present application, the coarse-grained classification process requires a predefined keyword table to quickly classify massive amounts of text into several predefined categories, significantly reducing the amount of text that needs to be processed in each fine-grained clustering process. Specifically, a list of frequently occurring keywords can be summarized for each online business based on the actual business situation. The overall table structure will be classified according to the categories actually needed. For example, for the "traffic diversion" category, the corresponding keyword table may include keywords such as [add me, + contact information].

[0115] For each input text, a sliding window matching method can be used to scan the text, attempting to match defined keywords at each character position. Ultimately, the text is assigned to a predefined coarse category. If the text matches multiple categories, only the category with the highest match is retained. In the embodiment of the present application, indicators such as character length, edit distance, and TF-IDF can be matched, and the comprehensive match between the text and the category is calculated using a weighted formula.

[0116] Optionally, for matching character lengths, a character length matching degree may be calculated, which may be specifically expressed as: character length matching degree = matched character length / total character length of the text.

[0117] For edit distance, we can calculate the edit distance between the text and the keyword (i.e., the number of insertion, deletion, and replacement operations), and then calculate the edit distance match. Specifically, it can be expressed as: edit distance match = 1-edit distance / max(total length of text characters, length of keyword characters). For example, if the text is "video is good" and the keyword is "video is good", then the edit distance is 1 (i.e., "watch" is replaced by "like"), and the edit distance match = 1-1 / 4 = 0.75.

[0118] Alternatively, for TF-IDF, the matching weight between the text and the keyword is calculated by combining the term frequency (TF) and the inverse document frequency (IDF), which is recorded as the TF-IDF matching degree and expressed as:

[0119]

[0120] For example, if the text is "video membership activity" and the keyword is "video", then the TF-IDF value of "video" is high, which means that the TF-IDF matching degree is significantly improved.

[0121] In some optional embodiments, the comprehensive matching degree between text and category can be expressed as: α·character length matching degree + β·edit distance matching degree + γ·TF-IDF matching degree. Optionally, α+β+γ=1. In an exemplary embodiment, α can be 0.2, β can be 0.2, and γ can be 0.6.

[0122] In the embodiment of the present application, the fine-grained clustering process mainly involves performing finer clustering of text within each coarse classification, such as Figure 6 In the example shown, for coarse category 1, fine-grained clustering can be used to obtain fine categories 1.1 and 1.2; for coarse category 2, fine-grained clustering can be used to obtain fine category 2.1; for coarse category N, fine-grained clustering can be used to obtain fine category Nx, and the obtained fine categories can be used as clustering results. This allows each coarse category to be further divided into one or more fine-grained clusters, and the text within each fine-grained cluster has a high degree of semantic similarity.

[0123] In some optional embodiments, when performing fine-grained clustering processing on texts within a coarse category, the student model obtained by distillation in the aforementioned embodiment can be used to generate text vectors for each text in the coarse category, and then the similarity between each text in the coarse category can be calculated based on the text vectors. For example, cosine similarity can be used to calculate the similarity between vectors. After calculating the similarity between texts, a graph path-based clustering method can be used for fine-grained clustering processing, that is, based on the similarity between all texts in the coarse category and setting a similarity threshold (0.8), an undirected graph is constructed. The nodes in the undirected graph represent texts, and the edges between nodes represent that the similarity between the corresponding texts is greater than or equal to the similarity threshold. Then, according to Figure 7 The process shown in the figure performs fine-grained clustering processing, which includes the following steps:

[0124] S701, calculating the degrees of all nodes in the undirected graph, that is, the number of edges directly connected to each node.

[0125] S702 , sorting the nodes in descending order of degree.

[0126] S703, determine whether all nodes in the undirected graph have been visited, if so, output the clustering result; otherwise, execute S704.

[0127] S704: Select the unvisited node with the highest degree as the current cluster center, and then add all nodes directly connected to the cluster center into the cluster.

[0128] S705: Mark the node in the cluster as visited, and then return to S703. This means the search continues for the next unvisited node with the highest degree. If the node has already been clustered, it is skipped. Otherwise, it is used as the new cluster center, and all connected unclustered nodes are added to the cluster until all nodes have been visited.

[0129] Continue to refer to Figure 6As shown, in an embodiment of the present application, in order to adapt to new terms and changes in fields, the keyword table can be dynamically updated. Specifically, after fine-grained clustering, each fine-grained cluster can be subjected to topic analysis to extract representative keywords, and then keywords that do not appear in the original vocabulary can be added to the keyword table of the corresponding field.

[0130] The technical solution of the above-mentioned embodiment of the present application first performs rapid coarse classification through the domain keyword table, and then vectorizes the text to be clustered in each cluster category and performs fine-grained clustering division, which solves the problems of large resource consumption and insufficient fine-grained accuracy in related technologies, and effectively improves the accuracy and efficiency of text clustering.

[0131] The following describes an embodiment of the apparatus of the present application, which can be used to execute the text clustering method in the above embodiment of the present application. For details not disclosed in the embodiment of the apparatus of the present application, please refer to the embodiment of the text clustering method in the above embodiment of the present application.

[0132] Figure 8 A block diagram of a text clustering apparatus according to an embodiment of the present application is shown. The text clustering apparatus can be applied to a computer device, which can be a server, a terminal device, or other device with computing and processing functions.

[0133] Reference Figure 8 As shown, a text clustering device 800 according to an embodiment of the present application includes: an acquisition unit 802 , a matching unit 804 , a division unit 806 and a processing unit 808 .

[0134] Among them, the acquisition unit 802 is configured to acquire the text to be clustered and the clustering keywords corresponding to at least two clustering categories; the matching unit 804 is configured to match the text to be clustered with the clustering keywords corresponding to the at least two clustering categories, and obtain the matching degree between the text to be clustered and each of the clustering categories; the division unit 806 is configured to divide the text to be clustered into the at least two clustering categories according to the matching degree between the text to be clustered and each of the clustering categories; the processing unit 808 is configured to extract the feature vector corresponding to the text to be clustered in each of the clustering categories, and cluster the text to be clustered in each of the clustering categories according to the feature vector corresponding to the text to be clustered, and obtain the clustering result of the text to be clustered.

[0135] In some embodiments of the present application, based on the aforementioned scheme, the matching unit 804 is configured to: intercept text characters from the text to be clustered according to a sliding window length of a set size, and match the intercepted text characters with the clustering keywords corresponding to each of the clustering categories to obtain a matching result between the text characters and the clustering keywords corresponding to each of the clustering categories; determine the matching degree between the text to be clustered and each of the clustering categories based on the matching result between the text characters and the clustering keywords corresponding to each of the clustering categories.

[0136] In some embodiments of the present application, based on the aforementioned scheme, the matching unit 804 is configured to: determine at least one matching degree between the text to be clustered and each of the clustering categories based on the matching content contained in the matching result; if multiple matching degrees between the text to be clustered and each of the clustering categories are determined, then perform weighted summation processing on the multiple matching degrees to obtain the final matching degree between the text to be clustered and each of the clustering categories.

[0137] In some embodiments of the present application, based on the aforementioned scheme, at least one matching degree between the text to be clustered and each of the clustering categories is determined according to the matching content contained in the matching results, including: determining the character length in the text to be clustered that matches the clustering keyword corresponding to each of the clustering categories according to the matching results; and determining the first matching degree between the text to be clustered and each of the clustering categories according to the ratio between the matched character length and the character length of the text to be clustered.

[0138] In some embodiments of the present application, based on the aforementioned scheme, at least one matching degree between the text to be clustered and each of the clustering categories is determined according to the matching content contained in the matching results, including: determining the edit distance between the text characters and the clustering keywords corresponding to each of the clustering categories according to the matching results; normalizing the edit distance, and determining the second matching degree between the text to be clustered and each of the clustering categories according to the result of the normalization.

[0139] In some embodiments of the present application, based on the aforementioned scheme, the edit distance is normalized, including: determining the maximum value of the character length based on the character length of the text characters and the character length of the clustering keywords corresponding to each of the clustering categories; calculating the ratio between the edit distance and the maximum value of the character length to obtain the result of the normalization processing.

[0140] In some embodiments of the present application, based on the aforementioned scheme, at least one matching degree between the text to be clustered and each of the clustering categories is determined according to the matching content contained in the matching results, including: determining the word frequency and inverse document frequency of the clustering keywords corresponding to each of the clustering categories in the text to be clustered according to the matching results; and determining a third matching degree between the text to be clustered and each of the clustering categories according to the product of the word frequency and the inverse document frequency.

[0141] In some embodiments of the present application, based on the aforementioned solution, the division unit 806 is configured to: divide the text to be clustered into the cluster type with the highest matching degree according to the matching degree between the text to be clustered and each of the cluster categories.

[0142] In some embodiments of the present application, based on the aforementioned scheme, the processing unit 808 is configured to: extract the feature vector corresponding to the text to be clustered in each clustering category through a pre-trained student model; wherein the text clustering device 800 also includes: a model training unit, configured to: obtain a sample text for training the student model, input the sample text into the student model, and obtain a first feature vector output by the student model for the sample text; input the sample text and prompt information for summarizing the content of the sample text into the teacher model, and obtain a second feature vector output by the teacher model; calculate the model loss corresponding to the student model based on the first feature vector and the second feature vector, and adjust the model parameters of the student model based on the model loss.

[0143] In some embodiments of the present application, based on the aforementioned scheme, the model training unit is configured to: perform normalization on the first eigenvector and the second eigenvector respectively to obtain the first normalized eigenvector and the second normalized eigenvector corresponding to the sample text; calculate the first similarity matrix between the first normalized eigenvectors corresponding to a batch of sample texts, and the second similarity matrix between the second normalized eigenvectors corresponding to a batch of sample texts; calculate the relative entropy model loss corresponding to the student model based on the first similarity matrix and the second similarity matrix.

[0144] In some embodiments of the present application, based on the aforementioned scheme, the processing unit 808 is configured to: calculate the similarity between the texts to be clustered in each clustering category according to the feature vector corresponding to the texts to be clustered in each clustering category; construct an undirected graph corresponding to each clustering category according to the similarity between the texts to be clustered in each clustering category, wherein the nodes in the undirected graph represent the texts to be clustered, and the edges in the undirected graph represent that the similarity between the texts to be clustered is greater than or equal to a set threshold; select a target node as the clustering center according to the degree of each node in the undirected graph, and cluster the target node and the nodes directly connected to the target node into one category to obtain at least one node cluster; determine the clustering result of the texts to be clustered according to the at least one node cluster and the texts to be clustered represented by the nodes in the node cluster.

[0145] In some embodiments of the present application, based on the aforementioned scheme, the processing unit 808 is configured to: select the node with the largest degree as a target node according to the degree of each node in the undirected graph; after clustering the selected target node and the nodes directly connected to the target node into one category, select the node with the largest degree again from the remaining nodes of the undirected graph as a target node, and perform clustering processing on the remaining nodes until the node traversal in the undirected graph is completed.

[0146] In some embodiments of the present application, based on the aforementioned scheme, the processing unit 808 is further configured to: after obtaining the clustering results of the text to be clustered, extract the keywords in each of the clustering results; and update the clustering keywords of the clustering category corresponding to each of the clustering results based on the keywords extracted from each of the clustering results.

[0147] Figure 9 A schematic structural diagram of a computer system of a computer device suitable for implementing an embodiment of the present application is shown. The computer device may be the text clustering apparatus in the aforementioned embodiment.

[0148] It should be noted that Figure 9 The computer system 900 of the computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0149] like Figure 9As shown, the computer system 900 may include a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 902 or the program loaded from the storage part 908 into the random access memory (RAM) 903, such as executing the method described in the above embodiment. Various programs and data required for system operation are also stored in the RAM 903. The CPU 901, ROM 902 and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0150] The following components can be connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 908 including a hard disk; and a communication section 909 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. Removable media 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed in the drive 910 as needed so that computer programs read therefrom can be installed into the storage section 908 as needed.

[0151] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program is used to perform the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from a removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the various functions defined in the system of the present application are performed.

[0152] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a computer program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable computer program. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0153] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and a computer program.

[0154] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. In some cases, the names of these units do not constitute limitations on the units themselves.

[0155] As another aspect, the present application further provides a computer-readable medium, which may be included in the computer device described in the above embodiments, or may exist independently and not be incorporated into the computer device. The computer-readable medium carries one or more computer programs, and when the one or more computer programs are executed by the computer device, the computer device implements the method described in the above embodiments.

[0156] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the application, the features and functions of two or more modules or units described above can be concretized in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0157] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on a network, and includes several instructions to enable a computer device to execute the method according to the embodiments of the present application.

[0158] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of this application and include common knowledge or customary techniques in the art that are not disclosed herein.

[0159] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A text clustering method, characterized in that: include: Obtain the text to be clustered and the clustering keywords corresponding to at least two clustering categories; Matching the text to be clustered with clustering keywords corresponding to the at least two clustering categories, to obtain a matching degree between the text to be clustered and each of the clustering categories; Dividing the text to be clustered into the at least two cluster categories according to the matching degree between the text to be clustered and each of the cluster categories; The feature vectors corresponding to the texts to be clustered in each cluster category are extracted, and clustering processing is performed on the texts to be clustered in each cluster category according to the feature vectors corresponding to the texts to be clustered, so as to obtain clustering results of the texts to be clustered.

2. The text clustering method according to claim 1, characterized in that Matching the text to be clustered with clustering keywords corresponding to the at least two clustering categories to obtain a matching degree between the text to be clustered and each of the clustering categories includes: intercepting text characters from the text to be clustered according to a set sliding window length, and matching the intercepted text characters with clustering keywords corresponding to each clustering category to obtain matching results between the text characters and the clustering keywords corresponding to each clustering category; The matching degree between the text to be clustered and each of the cluster categories is determined according to the matching result between the text characters and the cluster keywords corresponding to each of the cluster categories.

3. The text clustering method according to claim 2, characterized in that Determining the degree of matching between the text to be clustered and each cluster category according to a matching result between the text characters and the cluster keywords corresponding to each cluster category includes: Determining at least one matching degree between the text to be clustered and each of the cluster categories according to matching content included in the matching result; If multiple matching degrees between the text to be clustered and each of the cluster categories are determined, a weighted summation process is performed on the multiple matching degrees to obtain a final matching degree between the text to be clustered and each of the cluster categories.

4. The text clustering method according to claim 3, characterized in that Determining at least one matching degree between the text to be clustered and each of the cluster categories according to the matching content included in the matching result includes: Determining, according to the matching results, the character lengths of the texts to be clustered that match the clustering keywords corresponding to each clustering category; A first matching degree between the text to be clustered and each of the clustering categories is determined according to a ratio between the length of the matched characters and the length of the characters of the text to be clustered.

5. The text clustering method according to claim 3, characterized in that: Determining at least one matching degree between the text to be clustered and each of the cluster categories according to the matching content included in the matching result includes: Determining, according to the matching results, an edit distance between the text character and the clustering keyword corresponding to each clustering category; The edit distance is normalized, and a second matching degree between the text to be clustered and each of the cluster categories is determined according to a result of the normalization.

6. The text clustering method according to claim 5, characterized in that Normalizing the edit distance includes: Determining a maximum value of character length according to the character length of the text characters and the character length of the clustering keywords corresponding to each of the clustering categories; The ratio between the edit distance and the maximum value of the character length is calculated to obtain the result of the normalization processing.

7. The text clustering method according to claim 3, characterized in that Determining at least one matching degree between the text to be clustered and each of the cluster categories according to the matching content included in the matching result includes: Determining the word frequency and inverse document frequency of the clustering keyword corresponding to each clustering category in the text to be clustered according to the matching results; A third matching degree between the text to be clustered and each of the cluster categories is determined according to the product of the word frequency and the inverse document frequency.

8. The text clustering method according to claim 1, characterized in that: Dividing the text to be clustered into the at least two cluster categories according to the matching degree between the text to be clustered and each of the cluster categories includes: According to the matching degree between the text to be clustered and each of the cluster categories, the text to be clustered is divided into the cluster type with the highest matching degree.

9. The text clustering method according to claim 1, characterized in that: Extracting the feature vector corresponding to the text to be clustered in each cluster category, including: extracting the feature vector corresponding to the text to be clustered in each cluster category by using a pre-trained student model; Wherein, the text clustering method further includes: Obtaining a sample text for training the student model, inputting the sample text into the student model, and obtaining a first feature vector output by the student model for the sample text; Inputting the sample text and prompt information for summarizing the content of the sample text into the teacher model to obtain a second feature vector output by the teacher model; A model loss corresponding to the student model is calculated according to the first eigenvector and the second eigenvector, and a model parameter of the student model is adjusted according to the model loss.

10. The text clustering method according to claim 9, characterized in that: Calculating a model loss corresponding to the student model according to the first eigenvector and the second eigenvector includes: Normalizing the first feature vector and the second feature vector respectively to obtain a first normalized feature vector and a second normalized feature vector corresponding to the sample text; Calculating a first similarity matrix between the first normalized feature vectors corresponding to a batch of sample texts, and a second similarity matrix between the second normalized feature vectors corresponding to a batch of sample texts; Calculate the relative entropy model loss corresponding to the student model according to the first similarity matrix and the second similarity matrix.

11. The text clustering method according to claim 1, characterized in that: Clustering the texts to be clustered in each cluster category according to the feature vectors corresponding to the texts to be clustered includes: Calculating the similarity between the texts to be clustered in each cluster category according to the feature vectors corresponding to the texts to be clustered in each cluster category; According to the similarity between the texts to be clustered in each cluster category, an undirected graph corresponding to each cluster category is constructed, wherein the nodes in the undirected graph represent the texts to be clustered, and the edges in the undirected graph represent that the similarity between the texts to be clustered is greater than or equal to a set threshold; Selecting a target node as a cluster center according to the degree of each node in the undirected graph, and clustering the target node and the nodes directly connected to the target node into one category to obtain at least one node cluster; A clustering result of the text to be clustered is determined based on the at least one node cluster and the text to be clustered represented by the nodes in the node cluster.

12. The text clustering method according to claim 11, characterized in that: According to the degree of each node in the undirected graph, a target node is selected as the cluster center, and the target node and the nodes directly connected to the target node are clustered into one category, including: According to the degrees of the nodes in the undirected graph, selecting the node with the largest degree as a target node; After the selected target node and the nodes directly connected to the target node are clustered into one category, the node with the largest degree is again selected from the remaining nodes of the undirected graph as a target node, and the remaining nodes are clustered until the node traversal in the undirected graph is completed.

13. The text clustering method according to any one of claims 1 to 12, characterized in that: The text clustering method further includes: After obtaining the clustering results of the text to be clustered, extracting keywords from each of the clustering results; According to the keywords extracted from each clustering result, the clustering keywords of the clustering category corresponding to each clustering result are updated.

14. A text clustering device, characterized in that: include: An acquisition unit configured to acquire text to be clustered and clustering keywords corresponding to at least two clustering categories; a matching unit configured to match the text to be clustered with clustering keywords corresponding to the at least two clustering categories, and obtain a matching degree between the text to be clustered and each of the clustering categories; a dividing unit configured to divide the text to be clustered into the at least two cluster categories according to a matching degree between the text to be clustered and each of the cluster categories; The processing unit is configured to extract the feature vector corresponding to the text to be clustered in each cluster category, and perform clustering processing on the text to be clustered in each cluster category according to the feature vector corresponding to the text to be clustered, so as to obtain a clustering result of the text to be clustered.

15. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the text clustering method according to any one of claims 1 to 13 is implemented.

16. A computer device, characterized in that: include: one or more processors; A memory for storing one or more computer programs, which, when executed by the one or more processors, enables the computer device to implement the text clustering method according to any one of claims 1 to 13.

17. A computer program product, characterized in that The computer program product includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads and executes the computer program from the computer-readable storage medium, so that the computer device performs the text clustering method according to any one of claims 1 to 13.

Images