Document clustering method and system, computer program product and electronic equipment

Abstract

The invention discloses a document clustering method and system, a computer program product and electronic equipment. The method comprises the following steps: determining respective target abstract vectors of a plurality of target documents to be classified; clustering the plurality of target documents based on the target abstract vector to obtain a plurality of clusters, and determining a topic type corresponding to each cluster; for each cluster, determining a clustering distance between each target document in the cluster and other clusters, and taking the target document of which the clustering distance is lower than a preset clustering distance threshold as a to-be-corrected document; and re-classifying each to-be-corrected document by utilizing the first large language model, and updating each cluster according to an obtained classification result. According to the method and the device, the technical problem of relatively low clustering accuracy caused by clustering large-scale documents only according to a single feature in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular, to a document clustering method and system, a computer program product, and an electronic device. Background Art

[0002] With the continuous development of technology, the degree of document digitization is getting higher and higher, and more and more organizations and individuals adopt digital tools and solutions to manage, store, and share documents. After document digitization, new requirements for document search have emerged. Usually, it is necessary to focus on certain categories of documents, and with the increase in the order of magnitude of documents, manual classification has become very difficult. Therefore, how to organize and classify these digital documents becomes particularly important.

[0003] Currently, the document classification methods mainly include the following several types: First, document classification based on text similarity: By inputting a document into a pre-trained similarity score prediction model, obtaining the similarity score of the document, and clustering the bins according to the similarity scores of each document. Second, document clustering based on a knowledge graph: On the basis of keyword technology, a knowledge graph based on keywords is generated, and the knowledge graph is divided into several sub-graphs, and the clustering algorithm is used to cluster the documents into the events of the knowledge graph. Third, two-way clustering based on the term frequency-inverse document matrix: Directly perform document vectorization and achieve clustering by calculating the distance between text vectors. However, since these methods only perform clustering based on a single feature, the document clustering effect is poor.

[0004] In response to the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of the present application provide a document clustering method and system, a computer program product, and an electronic device, so as to at least solve the technical problem that in the related art, clustering of large-scale documents is performed only based on a single feature, resulting in poor clustering accuracy.

[0006] According to one aspect of the embodiments of the present application, a document clustering method is provided, including: determining respective target summary vectors of a plurality of target documents to be classified; clustering the plurality of target documents based on the target summary vectors to obtain a plurality of clustering clusters, and determining a corresponding theme type for each clustering cluster; for each clustering cluster, determining a clustering distance between each target document in the clustering cluster and other clustering clusters, and using the target documents with a clustering distance lower than a preset clustering distance threshold as documents to be corrected, where the clustering distance is determined by the distance between each other document in the same clustering cluster and the distance between each other document in other clustering clusters; re-classifying each document to be corrected by using a first large language model, and updating each clustering cluster according to the obtained classification results.

[0007] Optionally, determining the target summary vectors of multiple target documents to be classified includes: obtaining the multiple target documents to be classified; for each target document, using a second large language model to extract the content of the target document to obtain the target summary vector of the target document.

[0008] Optionally, using a second large language model to extract the content of the target document to obtain the target summary vector of the target document includes: dividing the target document into multiple paragraphs according to a preset division rule, where the division rule includes at least one of the following: dividing based on the number of sentences, dividing based on the paragraph structure, dividing based on keywords; using the second large language model to extract key content from each paragraph to obtain the paragraph summary vector of each paragraph; splicing the paragraph summary vectors of each paragraph in the order of their appearance in the document to obtain the initial summary vector of the document; using the second large language model to extract key content from the initial summary vector to obtain the target summary vector of the target document.

[0009] Optionally, clustering the multiple target documents according to the target summary vectors to obtain multiple clusters, and determining the topic type corresponding to each cluster includes: calculating the first distance between the target summary vectors of any two target documents in the multiple target documents, and dividing the target documents with the first distance less than a preset first distance threshold into the same cluster to obtain multiple clusters; for each cluster, determining the cluster center point of the cluster according to the target summary vectors of all target documents in the cluster; determining the second distance between the cluster center point of the cluster and the target summary vectors of each target document in the cluster, and taking the target documents with the second distance less than a preset second distance threshold as the central documents in the cluster, where the second distance threshold is less than the first distance threshold; determining the topic type corresponding to the cluster according to the target summary vectors corresponding to each central document.

[0010] Optionally, determining the topic type corresponding to the cluster according to the target summary vectors corresponding to each central document includes: using a first large language model to analyze the target summary vectors corresponding to each central document to obtain the document topic type of each central document; in the case where the document topic types of all central documents in the cluster are the same, taking the document topic type as the topic type of the cluster; in the case where the document topic types of all central documents in the cluster are different, determining the number of central documents included in each document topic type, and taking the document topic type with the largest number as the topic type of the cluster.

[0011] Optionally, determine the clustering distance between each target document in the clustering cluster and other clustering clusters, including: for each target document in the clustering cluster, determine the first average distance between the target document and the target summary vectors of each other target document in the same clustering cluster, and the second average distance between the target summary vector of the target document and the target summary vectors of each other target document in different other clustering clusters; according to the first average distance and the second average distance, determine the clustering distance between the target document and other clustering clusters according to the following formula:

[0012]

[0013] In the formula, s i represents the clustering quality of the i-th target document in the clustering cluster, d i represents the first average distance, c i represents the second average distance, and i ∈ [1, n], where n represents the total number of target documents included in the clustering cluster.

[0014] Optionally, use the first large language model to reclassify each document to be corrected, and update each clustering cluster according to the obtained classification results, including: for each document to be corrected in each clustering cluster, determine the second average distance between the target summary vector of the target document and the target summary vectors of each other target document in different other clustering clusters; sort the second average distances corresponding to each other clustering cluster in ascending order, and sequentially determine a preset number of second target average distances from the beginning of the obtained sorting result, and use the other clustering clusters corresponding to each second target average clustering as the candidate clustering clusters of the document to be corrected; input the document to be corrected, each candidate clustering cluster, and the topic type corresponding to each candidate clustering cluster into the first large language model to obtain the classification result of the document to be corrected output by the first large language model; update each clustering cluster according to the classification results of each document to be corrected in each clustering cluster.

[0015] According to another aspect of the embodiments of the present application, there is also provided a document clustering system, including: a first determination module, configured to determine the target summary vectors of multiple target documents to be classified; a clustering module, configured to cluster the multiple target documents based on the target summary vectors to obtain multiple clustering clusters, and determine the corresponding topic types of each clustering cluster; a second determination module, configured to, for each clustering cluster, determine the clustering distances between each target document in the clustering cluster and other clustering clusters, and use the target documents with clustering distances lower than a preset clustering distance threshold as documents to be corrected, where the clustering distance is determined by the distances between each other document in the same clustering cluster and the distances between each other document in other clustering clusters; a secondary clustering module, configured to reclassify each document to be corrected by using a first large language model, and update each clustering cluster according to the obtained classification results.

[0016] According to another aspect of the embodiments of the present application, there is also provided a computer program product, which includes: a computer program, where when the computer program is executed by a processor, the above-mentioned document clustering method is implemented.

[0017] According to another aspect of the embodiments of the present application, there is also provided an electronic device, which includes: a memory and a processor, where a computer program is stored in the memory, and the processor is configured to execute the above-mentioned document clustering method through the computer program.

[0018] In the embodiments of the present application, first, the target summary vectors of each target document to be classified are determined; then, the multiple target documents are clustered based on the target summary vectors to obtain multiple clustering clusters, and the corresponding topic types of each clustering cluster are determined; then, for each clustering cluster, the clustering distances between each target document in the clustering cluster and other clustering clusters are determined, and the target documents with clustering distances lower than a preset clustering distance threshold are used as documents to be corrected, where the clustering distance is determined by the distances between each other document in the same clustering cluster and the distances between each other document in other clustering clusters. Since there is a risk of misclassification for these documents to be corrected, therefore, a first large language model can be used to reclassify each document to be corrected, and each clustering cluster can be updated according to the obtained classification results. Thus, the technical effect of efficiently and accurately classifying large-scale documents is achieved, the purpose of ensuring the accuracy and robustness of the clustering results is achieved, and further, the technical problem that the related art only clusters large-scale documents based on a single feature, resulting in poor clustering accuracy, is solved. Description of the Drawings

[0019] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation to the present application. In the drawings:

[0020] Figure 1 It is a schematic flowchart of an optional document clustering method according to an embodiment of the present application;

[0021] Figure 2 It is a schematic structural diagram of an optional document clustering system according to an embodiment of the present application;

[0022] Figure 3 It is a schematic structural diagram of an optional electronic device according to an embodiment of the present application. Detailed implementation manners

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0025] To better understand the embodiments of the present application, some nouns or terms that appear in the description process of the embodiments of the present application are translated and explained as follows:

[0026] Large Language Model (LLM): It refers to a deep learning model trained with a large amount of text data, enabling the model to generate natural language text or understand the meaning of language text. These models can be trained on a large dataset to provide in-depth knowledge and language generation on various topics. The core idea is to learn the patterns and structures of natural language through large-scale unsupervised training, and to simulate the human language cognition and generation process to a certain extent.

[0027] Clustering: It is a process of dividing a set of physical or abstract objects into multiple classes composed of similar objects.

[0028] Example 1

[0029] According to an embodiment of the present application, a document clustering method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0030] Figure 1 is a schematic flowchart of a document clustering method provided according to an embodiment of the present application, as Figure 1 shown, the method includes the following steps:

[0031] Step S102, determine the target summary vector of each of the multiple target documents to be classified.

[0032] In the technical solution provided in the above step S102, since the original target documents are often high-dimensional and complex, therefore, in order to facilitate subsequent clustering operations, each target document to be classified can be traversed, and after deep summarization and vectorization processing of the target document, the corresponding target summary vector is obtained. Among them, the target summary vector can convert complex document text information into a numerical format that is easy for a computer to process, and can accurately capture the semantic features of the target document, so as to facilitate mathematical operations and pattern recognition.

[0033] Step S104, cluster the multiple target documents based on the target summary vectors to obtain multiple clustering clusters, and determine the topic type corresponding to each clustering cluster.

[0034] In the technical solution provided in the above step S104, clustering operations are performed on these target documents based on the target summary vector of each target document to divide them into multiple clustering clusters, so that the target summary vectors of the target documents within the same cluster have a high similarity, while the similarity between the target summary vectors of the target documents in different clustering clusters is low. In the embodiment of the present application, clustering operations are performed according to the target summary vector, which can reduce the amount of calculation during clustering, thereby reducing the consumed computing resources and improving the clustering efficiency.

[0035] Immediately afterwards, by analyzing the target documents in each initial clustering cluster, the core words or phrases representing these target documents are refined, and the refined core words or phrases are used as the topic type of the initial clustering cluster. Among them, the determination of the topic type helps to understand and manage the documents, and provides great convenience for subsequent document retrieval, document classification, and document recommendation.

[0036] Step S106: For each clustering cluster, determine the clustering distance between each target document within the cluster and other clustering clusters, and use the target documents with a clustering distance lower than the preset clustering distance threshold as the documents to be corrected.

[0037] In the technical solution provided in the above step S106, traverse each clustering cluster and calculate the clustering quality between each target document within the cluster and other clustering clusters. Among them, the clustering quality is determined by the distances between each other document within the same clustering cluster and the distances between each other document in different clustering clusters; judge the size relationship between the clustering quality between each target document and other clustering clusters and the preset clustering distance threshold, and when the clustering quality of any target document is lower than the threshold, use this target document as the document to be corrected within this clustering cluster. That is to say, the position of this target document within the current clustering cluster is not stable enough, and it is more suitable for other clustering clusters or this target document itself contains the characteristics of cross-cluster topics.

[0038] Step S108: Use the first large language model to reclassify each document to be corrected, and update each clustering cluster according to the obtained classification results.

[0039] In the technical solution provided in the above step S108, for all the identified documents to be corrected, a secondary clustering correction process can be executed. This can use the first large language model to reclassify each document to be corrected to determine the best belonging of each document to be corrected, and update each clustering cluster according to the determined classification results, so as to perform targeted correction and improve the accuracy and robustness of the overall clustering. This method not only improves the accuracy of the clustering results, but also increases the flexibility and can adapt to complex data distributions and the inherent diversity of documents.

[0040] Based on the solution defined in the above steps S102 to S108, it can be known that in the embodiment of the present application, first determine the target summary vector of each target document to be classified; then cluster multiple target documents based on the target summary vector to obtain multiple clustering clusters, and determine the corresponding theme type of each clustering cluster; then, for each clustering cluster, determine the clustering quality between each target document within the cluster and other clustering clusters, and use the target documents with a clustering distance lower than the preset clustering distance threshold as the documents to be corrected. Since there is a risk of misclassification for these documents to be corrected, the first large language model can be used to reclassify each document to be corrected, and each clustering cluster can be updated according to the obtained classification results. The above clustering solution can effectively correct the clustering results and ensure that each document to be classified is classified into the cluster that best matches its content. Therefore, through the embodiment of the present application, not only the automatic classification of documents is realized, but also the high quality of the classification results is ensured, overcoming the problems of low efficiency and poor accuracy of traditional clustering methods when dealing with large-scale and diverse document data.

[0041] The following describes each step of the document clustering method in combination with a specific implementation process.

[0042] As an alternative implementation, in the technical solution provided in step S102 above, the system can determine the target summary vector of each target document according to the following method, including:

[0043] Step S1021: Obtain multiple target documents to be classified.

[0044] Among them, these target documents can be sourced from various information sources, including but not limited to news, academic papers, social media posts, etc.

[0045] Step S1022: For each target document, use the second large language model to extract the content of the target document to obtain the target summary vector of the target document.

[0046] Among them, the above-mentioned second large language model is trained in advance on a large amount of text data to understand and capture the semantic features in the document, so as to learn the mapping relationship between the text content and the semantic representation. Therefore, the system can obtain the target summary vector of the target document by directly calling the pre-trained second large language model and using the corresponding prompt words to extract the content of the paragraph. Among them, the prompt word can be in the form of "Please extract the core content of the following content for summary, with no more than 500 words, and the content is as follows:".

[0047] Optionally, in the technical solution provided in step S1022 above, the second large language model can extract the content of the target document according to the following steps, including:

[0048] The first step: Divide the target document into multiple paragraphs according to a preset division rule. Among them, the above-mentioned division rules include but not limited to: dividing based on the number of sentences, dividing based on the paragraph structure, dividing based on keywords. Through these division rules, a long document or a document with a complex structure can be decomposed into smaller and easier-to-process units in advance, facilitating subsequent in-depth content extraction.

[0049] The second step: Use the second large language model to extract the key content of each paragraph to obtain the paragraph summary vector of each paragraph. That is to say, traverse each paragraph obtained by the division, use the second large language model to deeply understand the text of the paragraph, identify the core viewpoints, keywords, etc. in the paragraph, and generate a semantic feature, that is, the paragraph summary vector.

[0050] The third step: Concatenate the paragraph summary vectors of each paragraph in the order of their appearance in the document to obtain the initial summary vector of the document.

[0051] Step 4: Use the second large language model to extract key content from the initial summary vector to obtain the target summary vector of the target document. In this step, when the system calls the pre-trained second large language model for summary extraction, it can extract the document summary in combination with the corresponding prompt words, and the target summary vector of the target document can be obtained. Among them, the prompt words can be in the form of "Please generate the summary information of the following article based on the summary of each paragraph of the article, not exceeding 500 words. The summary content is as follows". This re-extraction process is based on the model's understanding of the semantic coherence of the entire document, ensuring that the generated summary vector not only contains the paragraph summary information of each paragraph, but also reflects the overall structure of the target document.

[0052] Therefore, through the above solution, the key content of documents with long texts and complex structures can be accurately extracted and converted into semantic vectors, and much more accurate and representative summary vectors can be obtained. This provides high-quality input for subsequent document classification and significantly improves the accuracy and efficiency of document clustering.

[0053] As an alternative implementation, in the technical solution provided in the above step S104, the system can cluster multiple target documents according to the following method, including:

[0054] Step S1041, calculate the first distance between the target summary vectors of any two target documents in the multiple target documents, and divide the target documents with the first distance less than the preset first distance threshold into the same clustering cluster to obtain multiple clustering clusters.

[0055] Among them, the above first distance can be calculated by measurement methods such as cosine distance, Euclidean distance, Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm, etc. The above step evaluates the similarity degree between the semantic features of each target document and constructs clustering clusters according to the similarity degree, ensuring that the target documents in the same cluster have high consistency in content.

[0056] Step S1042, for each clustering cluster, perform the following steps:

[0057] The first step: Determine the clustering center point of the clustering cluster according to the target summary vectors of all target documents in the clustering cluster.

[0058] Among them, the average vector of the target summary vectors of all target documents within the clustering cluster can be used as the clustering center point of the clustering cluster. That is to say, the clustering center point is the average value of the eigenvalue of all target documents within the clustering cluster in multiple dimensions; alternatively, the density peak point within the clustering cluster can be used as the clustering center point of the clustering cluster, where the density peak point is the target document with the highest density within the clustering cluster, that is, the target summary vector with the most neighbors in the surrounding neighborhood of the target document; or, the most central point within the clustering cluster can be used as the clustering center point of the clustering cluster, where the most central point is the point with the smallest average distance from other documents within the same clustering cluster.

[0059] In addition to the several clustering center point determination schemes listed above, those skilled in the art can also determine the clustering center point of the clustering cluster through other technical solutions. For example, using the median vector within the clustering cluster as the clustering center point, using the centroid vector within the clustering cluster as the clustering center point, etc., should also be within the protection scope of the present invention.

[0060] Step 2: Determine the second distance between the clustering center point of the clustering cluster and the target summary vectors of each target document within the clustering cluster, and use the target documents with the second distance less than the preset second distance threshold as the central documents within the clustering cluster. Among them, the second distance threshold is less than the first distance threshold.

[0061] Step 3: Determine the topic type corresponding to the clustering cluster according to the target summary vectors corresponding to each central document. Since each central document determined through the above Step 2 is semantically closer to the center of the clustering cluster, it can often better represent the characteristics of the clustering cluster. Therefore, the embodiments of the present application can determine the topic type corresponding to the clustering cluster according to the target summary vectors corresponding to these central documents.

[0062] Optionally, in the technical solution provided in the above Step 3, the system can determine the topic type corresponding to the clustering cluster according to the following method, including:

[0063] First, use the first large language model to analyze the target summary vectors corresponding to each central document to obtain the document topic type of each central document.

[0064] Among them, the above first large language model deeply analyzes information such as semantic patterns, keywords, and topic tendencies in the summary vectors corresponding to a large number of texts, identifies and summarizes the topic types of each text, so as to learn the mapping relationship between the document content and the corresponding topic types. Therefore, the system can call the pre-trained first large language model, relying on its powerful text understanding and generation capabilities, to analyze the semantic information behind the target summary vectors corresponding to each central document, and identify and summarize the document topic type of the central document.

[0065] After identifying the document theme types of all central documents, check the uniformity of the document theme types of each central document within the cluster, where:

[0066] When the document theme types of all central documents within the cluster are the same, use the document theme type as the theme type of the cluster. This is because central documents are selected as the most representative part within the cluster. If their theme types are the same, then this theme can accurately reflect the core content of the entire cluster.

[0067] When the document theme types of all central documents within the cluster are not the same, adopt the majority voting principle to determine the final theme type. Specifically, determine the number of central documents included in each document theme type, and use the document theme type with the largest number as the theme type of the cluster. This method takes into account the theme diversity within the cluster, but determines the dominant theme through numerical superiority, ensuring the consistency and representativeness of the clustering results.

[0068] Through the above theme determination steps, it is possible to automatically extract the theme type of each cluster while maintaining the accuracy and robustness of the document clustering results, providing a clear classification basis for subsequent document management and information retrieval.

[0069] It should be noted that the unique document identifier, target document abstract, and target abstract vector of the above target document can also be combined into a triple for subsequent clustering analysis, which can avoid duplication and confusion of documents during the clustering process and facilitate subsequent retrieval, analysis, and update operations.

[0070] As an alternative implementation, in the technical solution provided in step S106 above, the system can first traverse each target document within the cluster and determine the clustering distance between each target document and other clusters according to the following steps, including:

[0071] For each target document within the cluster, determine the first average distance between the target abstract vectors of the target document and each other target document within the same cluster, and the second average distance between the target abstract vector of the target document and each other target document within different other clusters; according to the first average distance and the second average distance, determine the clustering distance between the target document and other clusters according to the following formula:

[0072]

[0073] In the formula, s i represents the clustering quality of the i-th target document within the cluster, d i represents the first average distance, c idenotes the second average distance, and \(i\in[1,n]\), where \(n\) represents the total number of target documents included in the clustering cluster. Additionally, \(s\) i \(\in[-1,1]\), where when \(s\) i is close to 1, it indicates that the clustering result of the \(i\)-th target document is reasonable; conversely, when \(s\) i is close to -1, it indicates that the clustering result of the \(i\)-th target document is unreasonable.

[0074] Furthermore, determine the magnitude relationship between the clustering distance between each target document in the clustering cluster and other clustering clusters and a preset clustering distance threshold. In the embodiments of the present application, preferably, the clustering distance threshold is set to 0.3. There is no specific limitation on the setting of the clustering distance threshold, and it can be set according to the actual application scenario.

[0075] If the clustering distance between a certain target document and other clustering clusters is greater than the clustering distance threshold, it indicates that the distance between the target document and other clustering clusters is far. At this time, it can be considered that the clustering result of the target document is reasonable.

[0076] If the clustering distance between a certain target document and other clustering clusters is not greater than the clustering distance threshold, it indicates that the distance between the target document and other clustering clusters is close. At this time, it is necessary to further re-cluster the divergent documents to ensure the accuracy of their clustering results.

[0077] As an alternative implementation manner, in the technical solution provided in step S108 above, the system can re-cluster each document to be corrected according to the following steps, including:

[0078] Step S1081, for each document to be corrected in each clustering cluster, determine the second average distance between the target summary vector of the target document and the target summary vectors of each other target document in other clustering clusters that are different; sort the second average distances corresponding to each other clustering cluster in ascending order, and sequentially determine a preset number of second target average distances from the beginning of the obtained sorting result, and use the other clustering clusters corresponding to each second target average clustering as the candidate clustering clusters for the document to be corrected.

[0079] Step S1082, input the document to be corrected, each candidate clustering cluster, and the topic type corresponding to each candidate clustering cluster into the first large language model, and obtain the classification result of the document to be corrected output by the first large language model.

[0080] In the technical solution provided in step S1082, the system performs secondary classification attribution on the document to be corrected by calling a pre-trained first large language model and combining corresponding prompt words. Among them, if there are 5 candidate clustering clusters, the prompt words input into the first large language model can be in the form of "Suppose you are an article classification decision-maker. Please make a decision based on the provided article abstract and the categories to be classified as follows: (target text abstract vector, 0: the theme type of the currently affiliated clustering cluster, 1: the theme type of candidate cluster 1, 2: the theme type of candidate cluster 2, 3: the theme type of candidate cluster 3, 4: the theme type of candidate cluster 4, 5: the theme type of candidate cluster 5). Please analyze which category the target document should be classified into and directly select and output the serial number subscript". Therefore, this step is actually relying on the in-depth understanding ability and decision-making ability of the first large language model to perform secondary classification attribution on the document to be corrected, improving the accuracy and robustness of document classification.

[0081] Step S1083: Update each clustering cluster according to the classification results of each document to be corrected within each clustering cluster. This means that the target documents with unclear or incorrect original classifications will be reallocated to clustering clusters that better match their semantics.

[0082] By continuously repeating this process until all documents to be corrected have been reasonably classified, the entire clustering result is effectively corrected and optimized, significantly improving the accuracy and efficiency of document clustering. Especially when dealing with large-scale and variable document data sets, it can effectively handle problems such as fuzzy classification boundaries or clustering result deviations.

[0083] Therefore, in the document clustering method provided in the embodiments of the present application, by integrating the in-depth understanding ability and decision-making ability of the large language model, re-analyzing and re-classifying the abnormal points and uncertain points in the clustering result, the stability and robustness of the clustering result are improved, and the situations of misclassification and missed classification are reduced. Especially in terms of understanding and processing complex and large-scale document data, and in terms of improving the accuracy and robustness of the clustering result, it shows the potential to surpass existing methods.

[0084] Embodiment 2

[0085] According to the embodiments of the present application, there is also provided a document clustering system for implementing the document clustering method in Embodiment 1, as Figure 2 shown. The document clustering system at least includes: a first determination module 22, a clustering module 24, a second determination module 26, and a secondary clustering module 28, where:

[0086] The first determination module 22 is used to determine the target summary vectors of multiple target documents to be classified respectively;

[0087] The clustering module 24 is used to cluster multiple target documents based on the target summary vectors, obtain multiple clusters, and determine the topic type corresponding to each cluster;

[0088] The second determination module 26 is used to, for each cluster, determine the clustering distance between each target document in the cluster and other clusters, and use the target documents with a clustering distance lower than the preset clustering distance threshold as the documents to be corrected, where the clustering distance is determined by the distances between each other document in the same cluster and the distances between each other document in other clusters;

[0089] The secondary clustering module 28 is used to reclassify each document to be corrected using the first large language model and update each cluster according to the obtained classification results.

[0090] The functions of each module of the document clustering system will be described below in combination with the specific implementation process.

[0091] As an optional implementation manner, the first determination module 22 can determine the target summary vectors of each target document according to the following method, including:

[0092] Step S11, obtain multiple target documents to be classified.

[0093] Among them, these target documents can be from various information sources, including but not limited to news, academic papers, social media posts, etc.

[0094] Step S12, for each target document, use the second large language model to extract the content of the target document to obtain the target summary vector of the target document.

[0095] Among them, the above-mentioned second large language model is trained in advance on a large amount of text data to understand and capture the semantic features in the document, so as to learn the mapping relationship between the text content and the semantic representation. Therefore, the system can obtain the target summary vector of the target document by directly calling the pre-trained second large language model and using the corresponding prompt words to extract the content of the paragraph. Among them, the prompt word can be in the form of "Please extract the core content of the following content for summary, with no more than 500 words, and the content is as follows:".

[0096] Optionally, in the technical solution provided in the above step S12, the second large language model can extract the content of the target document according to the following steps, including:

[0097] Step 1: Divide the target document into multiple paragraphs according to the preset division rules. Among them, the above division rules include but are not limited to: division based on the number of sentences, division based on paragraph structure, and division based on keywords. Through these division rules, a long document or a document with a complex structure can be decomposed into smaller and easier-to-process units in advance, facilitating subsequent in-depth content extraction.

[0098] Step 2: Use the second large language model to extract key content from each paragraph to obtain the paragraph summary vector of each paragraph. That is to say, traverse each paragraph obtained by division, use the second large language model to deeply understand the text of the paragraph, identify the core viewpoints, keywords, etc. in the paragraph, and generate a semantic feature, that is, the paragraph summary vector.

[0099] Step 3: Concatenate the paragraph summary vectors of each paragraph in the order of their appearance in the document to obtain the initial summary vector of the document.

[0100] Step 4: Use the second large language model to extract key content from the initial summary vector to obtain the target summary vector of the target document. In this step, when the system calls the pre-trained second large language model for summary extraction, it can combine the corresponding prompt words to extract the document summary, and the target summary vector of the target document can be obtained. Among them, the prompt words can be in the form of "Please generate the summary information of the following article based on the summary of each paragraph of the article, not exceeding 500 words. The summary content is as follows". This re-extraction process is based on the model's understanding of the semantic coherence of the entire document, ensuring that the generated summary vector not only contains the paragraph summary information of each paragraph but also reflects the overall structure of the target document.

[0101] Therefore, through the above solution, the key content of long texts and documents with complex structures can be accurately extracted and converted into semantic vectors, and a much more accurate and representative summary vector can be obtained. This provides high-quality input for subsequent document classification and significantly improves the accuracy and efficiency of document clustering.

[0102] Optionally, the clustering module 24 can cluster multiple target documents according to the following method, including:

[0103] Step S21: Calculate the first distance between the target summary vectors of any two target documents in multiple target documents, and divide the target documents with the first distance less than the preset first distance threshold into the same clustering cluster to obtain multiple clustering clusters.

[0104] Step S22: For each clustering cluster, perform the following steps:

[0105] Step 1: Determine the clustering center point of the clustering cluster based on the target summary vectors of all target documents in the clustering cluster.

[0106] Among them, the average vector of the target summary vectors of all target documents within the clustering cluster can be used as the clustering center point of the clustering cluster. That is to say, the clustering center point is the average value of the eigenvalue of all target documents within the clustering cluster in multiple dimensions; alternatively, the density peak point within the clustering cluster can be used as the clustering center point of the clustering cluster, where the density peak point is the target document with the highest density within the clustering cluster, that is, the target summary vector with the most neighbors in the surrounding neighborhood of the target document; or, the most central point within the clustering cluster can be used as the clustering center point of the clustering cluster, where the most central point is the point with the smallest average distance from other documents within the same clustering cluster.

[0107] In addition to the above-listed several schemes for determining the clustering center point, those skilled in the art can also determine the clustering center point of the clustering cluster through other technical solutions. For example, using the median vector within the clustering cluster as the clustering center point, using the centroid vector within the clustering cluster as the clustering center point, etc., which should also be within the protection scope of the present invention.

[0108] The second step: Determine the second distance between the clustering center point of the clustering cluster and the target summary vectors of each target document within the clustering cluster, and use the target documents with the second distance less than the preset second distance threshold as the central documents within the clustering cluster. Among them, the second distance threshold is less than the first distance threshold.

[0109] The third step: Determine the topic type corresponding to the clustering cluster according to the target summary vectors corresponding to each central document. Since each central document determined through the above second step is semantically closer to the center of the clustering cluster, it can often better represent the characteristics of the clustering cluster. Therefore, the embodiments of the present application can determine the topic type corresponding to the clustering cluster according to the target summary vectors corresponding to these central documents.

[0110] Optionally, in the technical solution provided in the above third step, the clustering module 24 can determine the topic type corresponding to the clustering cluster according to the following method, including:

[0111] First, use the first large language model to analyze the target summary vectors corresponding to each central document to obtain the document topic type of each central document. After identifying the document topic types of all central documents, check the unity of the document topic types of each central document within the clustering cluster, where:

[0112] In the case where the document topic types of all central documents within the clustering cluster are the same, use the document topic type as the topic type of the clustering cluster. This is because the central documents are selected as the most representative part within the clustering cluster. If their topic types are the same, then this topic can accurately reflect the core content of the entire clustering cluster.

[0113] In the case where the document theme types of all central documents within a clustering cluster are inconsistent, the majority voting principle is adopted to determine the final theme type. Specifically, the number of central documents included in each document theme type is determined, and the document theme type with the largest number is used as the theme type of the clustering cluster. This method takes into account the theme diversity within the clustering cluster, but determines the dominant theme based on numerical superiority, ensuring the consistency and representativeness of the clustering results.

[0114] It should be noted that the document unique identifier, target document abstract, and target abstract vector of the above target document can also be combined into a triple for subsequent clustering analysis, which can avoid duplication and confusion of documents during the clustering process and facilitate subsequent retrieval, analysis, and update operations.

[0115] Optionally, the second determination module 26 can traverse each target document within the clustering cluster and determine the clustering distance between each target document and other clustering clusters according to the following steps, including:

[0116] For each target document within the clustering cluster, determine the first average distance between the target abstract vectors of the target document and each other target document within the same clustering cluster, and the second average distance between the target abstract vector of the target document and the target abstract vectors of each other target document within different other clustering clusters; according to the first average distance and the second average distance, determine the clustering distance between the target document and other clustering clusters according to the following formula:

[0117]

[0118] In the formula, s i represents the clustering quality of the i-th target document within the clustering cluster, d i represents the first average distance, c i represents the second average distance, and i ∈ [1, n], where n represents the total number of target documents included in the clustering cluster. In addition, s i ∈ [-1, 1], where when s i is close to 1, it indicates that the clustering result of the i-th target document is reasonable; on the contrary, when s i is close to -1, it indicates that the clustering result of the i-th target document is unreasonable.

[0119] Furthermore, the second determination module 26 determines the magnitude relationship between the clustering distance between each target document within the clustering cluster and other clustering clusters and a preset clustering distance threshold. Among them, in the embodiments of the present application, the clustering distance threshold is preferably set to 0.3, and there is no specific limitation on the setting of the clustering distance threshold, which can be set according to the actual application scenario.

[0120] If the clustering distance between a target document and other clustering clusters is greater than the clustering distance threshold, it indicates that the distance between the target document and other clustering clusters is relatively far. At this time, the clustering result of the target document can be considered reasonable.

[0121] If the clustering distance between a target document and other clustering clusters is not greater than the clustering distance threshold, it indicates that the distance between the target document and other clustering clusters is relatively close. At this time, it is necessary to further re-cluster the divergent documents to ensure the accuracy of their clustering results.

[0122] Optionally, the secondary clustering module 28 can re-cluster each document to be corrected according to the following steps, including:

[0123] Step S31, for each document to be corrected within each clustering cluster, determine the second average distance between the target summary vector of the target document and the target summary vectors of each other target document in different other clustering clusters; sort the second average distances corresponding to each other clustering cluster in ascending order, and sequentially determine a preset number of second target average distances from the beginning of the obtained sorting result, and use the other clustering clusters corresponding to each second target average clustering as the candidate clustering clusters for the document to be corrected.

[0124] Step S32, input the document to be corrected, each candidate clustering cluster, and the topic type corresponding to each candidate clustering cluster into the first large language model, and obtain the classification result of the document to be corrected output by the first large language model.

[0125] In the technical solution provided in step S32, the system performs secondary classification attribution on the document to be corrected by calling a pre-trained first large language model and combining corresponding prompt words. Among them, if there are 5 candidate clustering clusters, the prompt words input into the first large language model can be in the form of "Suppose you are an article classification decision maker, please make a decision based on the provided article abstract and the categories to be classified as follows: (target text summary vector, 0: the topic type of the currently belonging clustering cluster, 1: the topic type of candidate cluster 1, 2: the topic type of candidate cluster 2, 3: the topic type of candidate cluster 3, 4: the topic type of candidate cluster 4, 5: the topic type of candidate cluster 5), please analyze which category the target document should be classified into and directly select and output the serial number subscript". Therefore, this step is actually performing secondary classification attribution on the document to be corrected by relying on the in-depth understanding ability and decision-making ability of the first large language model, improving the accuracy and robustness of document classification.

[0126] Step S33, update each clustering cluster according to the classification results of each document to be corrected within each clustering cluster. This means that the target documents with unclear or incorrect original classifications will be re-allocated to the clustering clusters that better match their semantics.

[0127] By continuously repeating this process until all the documents to be corrected are reasonably classified, the entire clustering result is effectively corrected and optimized, significantly improving the accuracy and efficiency of document clustering. Especially when dealing with large-scale and variable document datasets, it can effectively handle the problems of fuzzy classification boundaries or clustering result deviations.

[0128] It should be noted that each module in the document clustering system in the embodiments of the present application corresponds one by one to each implementation step of the document clustering method in Embodiment 1. Since detailed descriptions have been made in Embodiment 1, some details not shown in this embodiment can be referred to Embodiment 1 and will not be elaborated here.

[0129] Embodiment 3

[0130] According to the embodiments of the present application, there is also provided a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the document clustering method in Embodiment 1.

[0131] According to the embodiments of the present application, there is also provided a non-volatile storage medium, which includes a stored computer program. The device where the non-volatile storage medium is located executes the document clustering method in Embodiment 1 by running the computer program.

[0132] According to the embodiments of the present application, there is also provided a processor, which is used to run a computer program. When the computer program runs, it executes the document clustering method in Embodiment 1.

[0133] According to the embodiments of the present application, there is also provided an electronic device, which includes: a memory and a processor. The memory stores a computer program, and the processor is configured to execute the document clustering method in Embodiment 1 through the computer program.

[0134] Specifically, when the computer program runs, it executes the following steps: determining the target summary vectors of multiple target documents to be classified; clustering the multiple target documents based on the target summary vectors to obtain multiple clustering clusters, and determining the corresponding topic types of each clustering cluster; for each clustering cluster, determining the clustering distances between each target document in the clustering cluster and other clustering clusters, and taking the target documents with clustering distances lower than the preset clustering distance threshold as the documents to be corrected, where the clustering distance is determined by the distances between each other document in the same clustering cluster and the distances between each other document in other clustering clusters; using the first large language model to reclassify each document to be corrected, and updating each clustering cluster according to the obtained classification results.

[0135] As an alternative implementation, the above-mentioned electronic device may exist in the form of a mobile terminal, a computer terminal, or a similar computing device. Figure 3 The following shows a hardware structure block diagram of an electronic device for implementing a document clustering method. As Figure 3 shown, the electronic device 30 may include one or more processors 302 (shown as 302a, 302b,..., 302n in the figure) (the processor 302 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 304 for storing data, and a transmission device 306 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports in the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 3 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the electronic device 30 may further include more or fewer components than Figure 3 shown, or have a different configuration from Figure 3 shown.

[0136] It should be noted that the above one or more processors 302 and / or other data processing circuits are generally referred to as "data processing circuits" in this article. The data processing circuit may be embodied in software, hardware, firmware, or any combination thereof, in whole or in part. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the electronic device 30. As involved in the embodiments of the present application, the data processing circuit is a processor control (such as the selection of a variable resistance terminal path connected to an interface).

[0137] The memory 304 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the document clustering method in the embodiments of the present application. The processor 302 executes various functional applications and data processing by running the software programs and modules stored in the memory 304, that is, implements the vulnerability detection method of the above-mentioned application program. The memory 304 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 304 may further include a memory remotely set relative to the processor 302, and these remote memories may be connected to the electronic device 30 through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0138] The transmission device 306 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by the communication provider of the electronic device 30. In one example, the transmission device 306 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the transmission device 306 can be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0139] The display can be, for example, a touch-screen liquid crystal display (LCD), which enables users to interact with the user interface of the electronic device 30.

[0140] The above-mentioned embodiment numbers are only for description and do not represent the superiority or inferiority of the embodiments.

[0141] In the above embodiments of the present application, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0142] In several embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of units or modules can be in electrical or other forms.

[0143] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] In addition, the functional units in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0145] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs.

[0146] The above are only the preferred embodiments of this application. It should be noted that for those of ordinary skill in the art of this technology, without departing from the principle of this application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of this application.

Claims

1. A document clustering method, characterized in that: include: Determining a target summary vector for each of a plurality of target documents to be classified; Clustering the multiple target documents based on the target summary vector to obtain multiple clusters, and determining a topic type corresponding to each of the clusters; For each cluster, determine the cluster distance between each target document in the cluster and other clusters, and use the target document whose cluster distance is lower than a preset cluster distance threshold as a document to be corrected, wherein the cluster distance is determined by the distance between each other document in the same cluster and the distance between each other document in other clusters; The first language model is used to reclassify each of the documents to be corrected, and each of the clusters is updated according to the classification results obtained.

2. The method according to claim 1, characterized in that Determine the target summary vectors of each of the multiple target documents to be classified, including: Acquire the plurality of target documents to be classified; For each of the target documents, the second largest language model is used to extract content from the target document to obtain a target summary vector of the target document.

3. The method according to claim 2, characterized in that The target document is extracted using the second largest language model to obtain a target summary vector of the target document, including: Dividing the target document into multiple paragraphs according to a preset division rule, wherein the division rule includes at least one of the following: division based on the number of sentences, division based on paragraph structure, and division based on keywords; Using the second largest language model to extract key content from each of the paragraphs, to obtain a paragraph summary vector for each of the paragraphs; According to the sequence of the paragraphs in the document, the paragraph summary vectors of the paragraphs are concatenated to obtain an initial summary vector of the document; The second largest language model is used to extract key content from the initial summary vector to obtain a target summary vector of the target document.

4. The method according to claim 1, characterized in that: Clustering the plurality of target documents according to the target summary vector to obtain a plurality of clusters, and determining a topic type corresponding to each of the clusters, including: Calculating a first distance between target summary vectors of any two target documents in the plurality of target documents, and classifying target documents whose first distance is less than a preset first distance threshold into the same cluster, to obtain the plurality of clusters; For each of the clusters, the cluster center of the cluster is determined according to the target summary vectors of all the target documents in the cluster; a second distance between the cluster center of the cluster and the target summary vectors of each target document in the cluster is determined, and a target document whose second distance is less than a preset second distance threshold is used as a central document in the cluster, wherein the second distance threshold is less than the first distance threshold; and a topic type corresponding to the cluster is determined according to the target summary vectors corresponding to each of the central documents.

5. The method according to claim 4, characterized in that Determining the topic type corresponding to the cluster according to the target summary vector corresponding to each of the central documents includes: Analyzing the target summary vector corresponding to each of the central documents using the first language model to obtain the document topic type of each of the central documents; When the document subject types of all the central documents in the cluster are consistent, the document subject type is used as the subject type of the cluster; In the case that the document subject types of all the central documents in the cluster are inconsistent, the number of central documents contained in each document subject type is determined, and the document subject type with the largest number is used as the subject type of the cluster.

6. The method according to claim 1, characterized in that Determining the cluster distance between each target document in the cluster and other clusters includes: For each target document in the cluster, determine a first average distance between the target document and target summary vectors of other target documents in the same cluster, and a second average distance between the target summary vector of the target document and target summary vectors of other target documents in different other clusters; and determine the cluster distance between the target document and other clusters according to the following formula based on the first average distance and the second average distance: Where s i represents the clustering quality of the i-th target document in the cluster, d i represents the first average distance, c i represents the second average distance, and i∈[1,n], where n represents the total number of target documents contained in the cluster.

7. The method according to claim 1, characterized in that Reclassifying each of the documents to be corrected using the first language model, and updating each of the clusters according to the classification results, including: For each document to be corrected in each of the clusters, determine the second average distance between the target summary vector of the target document and the target summary vectors of each other target document in different other clusters; sort the second average distances corresponding to each other cluster in ascending order, and determine a preset number of second target average distances from the beginning of the sorting result, and use the other clusters corresponding to each of the second target average clusters as candidate clusters for the document to be corrected; Inputting the document to be corrected, each of the candidate clusters, and the topic type corresponding to each of the candidate clusters into the first large language model, and obtaining a classification result to which the document to be corrected belongs output by the first large language model; Each of the clusters is updated according to the classification results of each of the to-be-corrected documents in each of the clusters.

8. A document clustering system, characterized in that: include: A first determination module, used to determine a target summary vector of each of a plurality of target documents to be classified; A clustering module, used for clustering the plurality of target documents based on the target summary vector to obtain a plurality of clusters, and determining a topic type corresponding to each of the clusters; A second determination module is used to determine, for each cluster, a cluster distance between each target document in the cluster and other clusters, and to use a target document whose cluster distance is lower than a preset cluster distance threshold as a document to be corrected, wherein the cluster distance is determined by the distance between each other document in the same cluster and the distance between each other document in other clusters; The secondary clustering module is used to reclassify each of the documents to be corrected by using the first language model, and update each of the clusters according to the obtained classification results.

9. A computer program product, characterized in that include: A computer program, wherein when the computer program is executed by a processor, the document clustering method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the document clustering method according to any one of claims 1 to 7 through the computer program.

Images