Document clustering and ranking method, system, device and medium based on language large model

Abstract

The application discloses a document clustering and sorting method, system, device and medium based on a language large model, wherein the method comprises the following steps: collecting document data for structured processing and preprocessing; inputting the document content into the language large model to obtain vectorized representation; using a clustering algorithm on the vectorized document content to obtain a document cluster and a similarity matrix in the document cluster, sorting the documents in each document cluster according to the weighted sum of the similarity matrix, and taking the top ten document titles as seed document titles; counting the number of each level document in the document cluster, the total number of documents and the weighted sum of the document cluster correlation coefficient, and calculating the weighted sum of the three indexes to obtain the final score of each document cluster, and sorting according to the score; inputting the seed document title and the set prompt into the language large model to generate a short sentence as the class label of the document cluster. The application can make the document vectorization more accurate, the class sorting more scientific, and the generation of the class label more specific and automatic.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, and specifically relates to a document clustering and sorting method, system, device and medium based on a large language model. Background Technology

[0002] Traditional document classification requires experts to categorize documents based on business scenarios and experience, typically using alphabetical order, which proved effective in the era of paper documents. However, with continuous technological advancements and increasing digitization of documents, more and more organizations and individuals are adopting digital tools and solutions to manage, store, and share documents. This digitization has created new demands for document searching, often requiring a focus on specific categories. Furthermore, with the sheer volume of documents, manual classification has become extremely difficult. Simultaneously, the accumulation of large amounts of digitized documents can lead to information overload and processing difficulties, making it challenging for users to find the information they need. Therefore, effective management, backup, and categorized storage are crucial to ensure long-term data preservation and accessibility. Without a robust document management system, data loss or unavailability may occur, making the organization, classification, and sorting of these digitized documents particularly important. The rapid development of language large model technologies, such as GhatGPT, has provided new solutions for document classification and sorting.

[0003] Current document classification and ranking methods mainly include: 1. Keyword-based document search classification and text similarity-based document ranking: Relevant documents are queried using keywords. The retrieved documents are input into a pre-trained similarity score prediction model to obtain their similarity scores. Based on the similarity scores of each candidate document, the document ranking result is determined. 2. Knowledge graph-based document clustering: Building upon keyword technology, a keyword-based knowledge graph is generated and divided into several subgraphs. Clustering algorithms are used to cluster documents into events within the knowledge graph. 3. Bidirectional clustering based on term frequency-inverse document matrix: Documents are directly vectorized, and clustering is achieved by calculating the distance between text vectors.

[0004] Chinese patent publication number "CN116304071A" discloses a two-layer webpage document clustering algorithm based on knowledge graphs. This invention searches, categorizes, and sorts relevant documents based on provided keywords. The process involves: 1) sorting and categorizing relevant candidate documents from a document library; 2) concatenating sentences related to keywords in each candidate document to generate a concatenated corpus; 3) predicting similarity scores by inputting the generated concatenated corpus into a pre-trained similarity score prediction model to calculate the similarity score for each candidate document; and 4) ranking multiple candidate documents based on their similarity scores. The invention suffers from the following main problems: First, the keywords used for classification are determined entirely based on experience. The quality of keyword selection directly determines the accuracy of document classification. When the number of documents is large, the category coverage is inevitably limited, directly affecting accuracy. Second, the classification identifiers, or class tags, directly use selected keywords, resulting in limited ability to summarize document categories. Classification tags should reflect the main characteristics of all documents in that category as much as possible, generally phrases or short sentences containing multiple keywords. Third, computational complexity: concatenating and calculating the similarity score between documents and queries can require significant computational resources and time, especially with large-scale document databases, potentially leading to latency and performance issues. Fourth, the traditional similarity calculation model used performs worse than large-scale language models, which may result in inaccurate ranking.

[0005] Chinese patent publication number "CN111680131B" discloses a semantic-based document clustering method, including: acquiring input documents and preprocessing them; performing word frequency statistics and inverse document frequency calculation on each word contained in the processed input document to construct a word frequency-inverse document matrix; inputting the words used in the word frequency statistics as objects into a pre-stored natural language processing model to obtain a similarity matrix adapted to the word frequency-inverse document matrix; performing semantic propagation on the word frequency-inverse document matrix based on the similarity matrix to obtain a second word frequency-inverse document matrix; performing bidirectional clustering on the second word frequency-inverse document matrix to obtain at least one bi-cluster, each bi-cluster containing a document cluster and a word cluster, and assigning labels to each document in the document cluster based on the feature words contained in the word cluster. Unlike keyword methods, the keywords used as class labels in this invention are no longer manually generated, but generated through clustering. The above invention mainly has the following problems: First, although using bidirectional clustering eliminates the need for manually providing keywords, it still belongs to the method of using keywords as class labels, and using only keywords to describe the feature information of document clusters is still insufficient. Second, while the term frequency-inverse document frequency (TF-IF) algorithm is simple to implement, easy to understand, and highly interpretable, using term frequency to measure the importance of a word in an article is not comprehensive enough, resulting in low classification accuracy. Third, this calculation fails to reflect word positional information and the differences between the same word in different contexts. Fourth, this invention does not address the ranking of documents after clustering, which is crucial in document organization. Summary of the Invention

[0006] This invention provides a document clustering and ranking method, system, device, and medium based on a large language model, aiming to solve the problems of existing text vectorization methods relying on feature engineering, having weak representation capabilities, needing to rely on human experience to determine class labels or to select them from existing texts, and having limited expressive power in class labels.

[0007] To address the aforementioned technical problems, this invention provides a document clustering and ranking method based on a large language model, comprising the following steps:

[0008] S1: Collect document data and perform structured processing. The document data includes document content and document information, and the document information includes document title and document level. Preprocess the structured document content.

[0009] S2: Input the preprocessed document content into the language big model to calculate the vectorized representation of the document content.

[0010] S3: The vectorized document content is processed using a clustering algorithm to obtain multiple document clusters. The documents in each document cluster are used to calculate a similarity matrix based on vector distance. The documents in each document cluster are sorted within the cluster according to the weighted sum of the similarity matrices. The top ten documents in the cluster are used as seed documents. The title of the seed document is the seed document title.

[0011] S4: Calculate the correlation coefficient within each document cluster using the seed document title to obtain the weighted sum of the correlation coefficients. Based on the document levels in the structured document information, count the number of documents at each level in the document cluster. Calculate the weighted sum of the three indicators: the number of documents at each level, the total number of documents, and the weighted sum of the correlation coefficients within the document cluster. Obtain the final score for each document cluster. Sort the document clusters according to their scores.

[0012] S5: Generate short sentences summarizing the information features of document clusters by combining the titles of seed documents within each document cluster with the set prompt input language model, and use the short sentences as class labels for document clusters.

[0013] Preferably, in step S3, sorting documents within each document cluster according to the weighted sum of the similarity matrices specifically involves:

[0014] S31: The set of vectorized representations of documents within each document cluster is V = {v1, v2, ..., v...} p}, where p is the total number of documents in the document cluster, and the similarity matrix between the nth document and the mth document in the document cluster is d(v n ,v m ), similarity matrix d(v n ,v m The specific formula for calculating ) is as follows:

[0015]

[0016] In the formula, v n v is the vectorized representation of the nth document within a document cluster. m This is the vectorized representation of the m-th document within the document cluster.

[0017] S32: The formula for calculating the weighted sum of the similarity matrices of the nth document within a document cluster is as follows:

[0018]

[0019] In the formula, S n The weighted sum of the similarity matrix of the nth document, α i Let v be the weight of the i-th similarity matrix. i This is the vectorized representation of the i-th document within the document cluster.

[0020] S33: Sort the documents within each document cluster according to the weighted sum of the similarity matrix.

[0021] Preferably, in step S4, the relevance coefficients within each document cluster are calculated using seed documents to obtain a weighted sum of relevance coefficients. Specifically, the seed document titles are input into the language big data model to calculate a vectorized representation T = {t1, t2, ..., t...}. 10 The formula for calculating the weighted sum of the relevance coefficients of the k-th seed document titles within each document cluster is as follows:

[0022]

[0023] In the formula, WSCC k β is the weighted sum of the relevance coefficients of the title of the k-th seed document. i Let t be the weight of the i-th similarity matrix. k This is the vectorized representation of the title of the k-th seed document.

[0024] The specific formula for calculating the weighted sum of correlation coefficients within each document cluster is as follows:

[0025]

[0026] In the formula, WSCC is the weighted sum of correlation coefficients within a document cluster, and γ k For the k-th WSCC k The weight.

[0027] Preferably, the formula for calculating the final score of the document cluster in step S4 is as follows:

[0028] CFS=0.3*DC1+0.2*DC2+0.1*DC+0.4*WSCC

[0029] In the formula, CFS is the final score of the document cluster, DC1 is the number of documents at level 1, DC2 is the number of documents at level 2, DC is the total number of documents, and WSCC is the weighted sum of correlation coefficients within the document cluster.

[0030] On the other hand, the present invention provides a document clustering and ranking system based on a large language model, including a document acquisition module, a text vectorization module, a document clustering module, a class ranking module, and a class label generation module.

[0031] The document acquisition module is used to collect document data and perform structured processing. The document data includes document content and document information, and the document information includes document title and document level. The structured document content is preprocessed.

[0032] The text vectorization module is used to input the preprocessed document content into the language big model to calculate the vectorized representation of the document content.

[0033] The document clustering module is used to obtain multiple document clusters by using a clustering algorithm on the vectorized document content. The documents in each document cluster are used to calculate a similarity matrix based on vector distance. The documents in each document cluster are sorted within the cluster according to the weighted sum of the similarity matrix. The top ten documents in the cluster are used as seed documents, and the title of the seed document is called the seed document title.

[0034] The sorting module is used to calculate the correlation coefficient within each document cluster using the seed document title to obtain the weighted sum of the correlation coefficients. Based on the document levels in the structured document information, it counts the number of documents at each level in the document cluster. Based on the number of documents at each level, the total number of documents, and the weighted sum of the correlation coefficients within the document cluster, it calculates the weighted sum of the three indicators to obtain the final score of each document cluster. The document clusters are then sorted according to their scores.

[0035] The class tag generation module is used to generate short sentences that summarize the information features of document clusters by combining the titles of seed documents within each document cluster with the set prompt input language model, and these short sentences are used as class tags for document clusters.

[0036] Preferably, the document clustering module performs intra-cluster sorting of documents within each document cluster based on the weighted sum of the similarity matrices as follows:

[0037] S31: The set of vectorized representations of documents within each document cluster is V = {v1, v2, ..., v...} p}, where p is the total number of documents in the document cluster, and the similarity matrix between the nth document and the mth document in the document cluster is d(v n ,v m ), similarity matrix d(v n ,v m The specific formula for calculating ) is as follows:

[0038]

[0039] In the formula, v n v is the vectorized representation of the nth document within a document cluster. m This is the vectorized representation of the m-th document within the document cluster.

[0040] S32: The formula for calculating the weighted sum of the similarity matrices of the nth document within a document cluster is as follows:

[0041]

[0042] In the formula, S n The weighted sum of the similarity matrix of the nth document, α i c represents the weight of the i-th similarity matrix. i This is the vectorized representation of the i-th document within the document cluster.

[0043] S33: Sort the documents within each document cluster according to the weighted sum of the similarity matrix.

[0044] Preferably, the class sorting module uses seed documents to calculate the correlation coefficient within each document cluster to obtain a weighted sum of correlation coefficients. Specifically, this involves inputting the seed document titles into the language big data model to calculate a vectorized representation T = {t1, t2, ..., t}. 10 The formula for calculating the weighted sum of the relevance coefficients of the k-th seed document titles within each document cluster is as follows:

[0045]

[0046] In the formula, WSCC k β is the weighted sum of the relevance coefficients of the title of the k-th seed document. i Let t be the weight of the i-th similarity matrix. k This is the vectorized representation of the title of the k-th seed document.

[0047] The specific formula for calculating the weighted sum of correlation coefficients within each document cluster is as follows:

[0048]

[0049] In the formula, WSCC is the weighted sum of correlation coefficients within a document cluster, and γ k For the k-th WSCC k The weight.

[0050] Preferably, the formula for calculating the final score of the document cluster in the class sorting module is as follows:

[0051] CFS=0.3*DC1+0.2*DC2+0.1*DC+0.4*WSCC

[0052] In the formula, CFS is the final score of the document cluster, DC1 is the number of documents at level 1, DC2 is the number of documents at level 2, DC is the total number of documents, and WSCC is the weighted sum of correlation coefficients within the document cluster.

[0053] In another aspect, the present invention also provides an electronic device, the electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the document clustering and ranking method based on a large language model as described in any embodiment of the present invention.

[0054] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the document clustering and ranking method based on a large language model as described in any embodiment of the present invention.

[0055] Compared with the prior art, the present invention has the following technical effects:

[0056] 1. More accurate document vectorization: This invention eliminates the need for preprocessing of the original document, such as word segmentation. Document vectorization based on the generative language model can dynamically vectorize the document, thereby solving the problem of polysemy and making subsequent vectorization-based process steps more reasonable and the results more accurate.

[0057] 2. More scientific sorting criteria: This invention not only relies on the distance between document vectors, but also considers multiple dimensions such as document source level and the number of documents in a class, so as to comprehensively determine the class sorting through more scientific and appropriate sorting criteria.

[0058] 3. Automated and specific generation of class tags: This invention uses a large language model combined with the top ten document titles in a document cluster to generate titles representing that document cluster, which can more specifically express and highlight the characteristics of the document cluster. Attached Figure Description

[0059] Figure 1 This is an overall flowchart of the document clustering and ranking method based on a large language model as described in this invention;

[0060] Figure 2 This is an overall structural diagram of the document clustering and ranking system based on a large language model as described in this invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present application and with reference to the accompanying drawings.

[0062] Example 1

[0063] This embodiment provides a document clustering and ranking method based on a large language model. It uses the ELMO method within the large language model for text vectorization; it generates class labels based on short phrases generated by the large language model's capabilities, rather than pre-given or clustered keywords; after applying the clustering algorithm, document ranking uses not only document vector distance but also features such as document quantity and document level for comprehensive ranking. This embodiment improves the accuracy of document clustering and the level of organization, classification, and ranking, resulting in more specific class label representations and more reasonable document ranking rules within each class. See also... Figure 1 As shown, it includes the following steps:

[0064] S1: Collect document data and perform structured processing for structured storage. The collected document data can be various rules and regulations documents issued by the company, etc. The document data includes document content and document information, including document title and document level. The structured storage typically involves storing the document level information in a relational database. The structured document content is then preprocessed, including deleting and cleaning special, meaningless symbols and punctuation, removing HTML tags from web documents, de-identifying document data, and anonymizing document data. Unlike traditional techniques based on keywords or word frequency-inverse document algorithms, this embodiment uses a large language model for text vectorization, thus eliminating the need for preprocessing operations such as word segmentation, low-frequency word removal, and stop word removal.

[0065] S2: Input the preprocessed document content into the language big data model to calculate the vectorized representation of the document content. The language big data model can use online interfaces such as ChatGPT, or a locally deployed language big data model service. The preprocessed document content is then used with the embedding module of the language big data model to calculate its vectorized representation. Compared to the term frequency-inverse document algorithm, the language big data model's vectorization is based on the ELMO method, resulting in more accurate vectorization, higher dimensionality, and more informational features. It can also incorporate textual context information to address polysemy.

[0066] S3: Use a clustering algorithm to obtain multiple document clusters from the vectorized document content. The clustering algorithm includes K-means clustering, DBSCA, Hierarchical Clustering, etc. The appropriate algorithm or model can be selected according to the actual scenario and needs.

[0067] K-means clustering is an iterative clustering analysis algorithm. As a preferred implementation in this embodiment, K-means clustering is used to obtain document clusters. Specifically, the implementation steps are as follows: First, the documents are pre-divided into K groups. K documents (vectorized representation) are randomly selected as initial cluster centers. Then, the distance between each document and each cluster center is calculated, and each document is assigned to the cluster center with the nearest Euclidean distance. Each cluster center document and the documents assigned to it represent a document cluster. Whenever a sample document is assigned, the cluster centers of the class are recalculated based on the existing document vectors in the class. This process is repeated until a termination condition is met. The termination condition may be that no (or a minimum number of) objects are reassigned to different clusters, no (or a minimum number of) cluster centers change, or the sum of squared errors reaches a local minimum.

[0068] DBSCA is an algorithm that utilizes the concept of density-based clustering, which requires that the number of objects (points or other spatial objects) contained in a certain region of the cluster space be no less than a given threshold. The significant advantages of the DBSCAN algorithm are its fast clustering speed and its ability to effectively handle noisy points and discover spatial clusters of arbitrary shapes. However, when the density of the spatial clusters is uneven and the difference in the distance between clusters is large, the clustering quality is poor.

[0069] Hierarchical clustering creates a hierarchical nested clustering tree by calculating the similarity between data points of different categories. The smaller the distance, the higher the similarity. In the clustering tree, the original data points of different categories form the lowest level, and the top level is the root node of a cluster. There are two methods for creating clustering trees: bottom-up merging and top-down splitting.

[0070] Documents in each document cluster are used to calculate a similarity matrix based on vector distance. Documents within each document cluster are sorted by the weighted sum of the similarity matrices. The top ten documents in each cluster are used as seed documents. A document cluster, i.e., a document category, contains multiple documents. The title of the seed document is called the seed document title.

[0071] In a preferred embodiment of this example, step S3, which involves sorting documents within each document cluster according to the weighted sum of the similarity matrices, specifically involves:

[0072] S31: The set of vectorized representations of documents within each document cluster is V = {v1, v2, ..., v...} p}, where p is the total number of documents in the document cluster, and the similarity matrix between the nth document and the mth document in the document cluster is d(v n ,v m ), similarity matrix d(v n ,v m The specific formula for calculating ) is as follows:

[0073]

[0074] In the formula, v n v is the vectorized representation of the nth document within a document cluster. m This is the vectorized representation of the m-th document within the document cluster.

[0075] S32: The formula for calculating the weighted sum of the similarity matrices of the nth document within a document cluster is as follows:

[0076]

[0077] In the formula, S n The weighted sum of the similarity matrix of the nth document, α iLet v be the weight of the i-th similarity matrix. i This is the vectorized representation of the i-th document within the document cluster.

[0078] S33: Sort the documents within each document cluster according to the weighted sum of the similarity matrix.

[0079] S4: Calculate the correlation coefficient within each document cluster using the seed document title to obtain the weighted sum of the correlation coefficients. Based on the document levels in the structured document information, count the number of documents at each level in the document cluster. Calculate the weighted sum of the three indicators: the number of documents at each level, the total number of documents, and the weighted sum of the correlation coefficients within the document cluster. Obtain the final score for each document cluster. Sort the document clusters according to their scores.

[0080] In a preferred embodiment of this example, step S4, which uses seed documents to calculate the correlation coefficient within each document cluster to obtain a weighted sum of correlation coefficients, specifically involves inputting the seed document titles into a large language model to calculate a vectorized representation T = {t1, t2, ..., t...}. 10 The formula for calculating the weighted sum of the relevance coefficients of the k-th seed document titles within each document cluster is as follows:

[0081]

[0082] In the formula, WSCC k β is the weighted sum of the relevance coefficients of the title of the k-th seed document. i Let t be the weight of the i-th similarity matrix. k This is the vectorized representation of the title of the k-th seed document.

[0083] The specific formula for calculating the weighted sum of correlation coefficients within each document cluster is as follows:

[0084]

[0085] In the formula, WSCC is the weighted sum of correlation coefficients within a document cluster, and γ k For the k-th WSCC k The weight.

[0086] In a preferred embodiment of this example, the formula for calculating the final score of the document cluster in step S4 is as follows:

[0087] CFS=0.3*DC1+0.2*DC2+0.1*DC+0.4*WSCC

[0088] In the formula, CFS is the final score of the document cluster, DC1 is the number of documents at level 1, DC2 is the number of documents at level 2, DC is the total number of documents, and WSCC is the weighted sum of correlation coefficients within the document cluster.

[0089] S5: The seed document titles within each document cluster are used in conjunction with the set prompt input to the language model to generate short sentences summarizing the information features of the document cluster. These short sentences are then used as class labels for the document clusters. The seed document titles of the document clusters obtained based on the language model and the preceding steps are the top ten most relevant documents in the cluster, weighted by their correlation coefficients with other documents. Compared to pre-defined keywords, these titles are more representative of the information features of the document clusters. The prompt can be set to something like this: Given the following ten document titles, one or more words or phrases are given based on their similarity. The features of these titles are summarized, as follows… (list the ten document titles obtained).

[0090] Example 2

[0091] Accordingly, see Figure 2 As shown, this embodiment provides a document clustering and ranking system based on a large language model, including a document acquisition module, a text vectorization module, a document clustering module, a class ranking module, and a class label generation module.

[0092] The document acquisition module is used to collect document data and perform structured processing. The document data includes document content and document information. The document information includes document title and document level. The structured document content is preprocessed. This module is used to implement the function of step S1 in Embodiment 1, which will not be described again here.

[0093] The text vectorization module is used to input the preprocessed document content into the language big model to calculate the vectorized representation of the document content. This module is used to implement the function of step S2 in Example 1, which will not be described again here.

[0094] The document clustering module is used to obtain multiple document clusters by using a clustering algorithm on the vectorized document content. The documents in each document cluster are used to calculate a similarity matrix based on vector distance. The documents in each document cluster are sorted within the cluster according to the weighted sum of the similarity matrix. The top ten documents in the cluster are used as seed documents. The title of the seed document is called the seed document title. This module is used to implement the function of step S3 in Embodiment 1, which will not be described again here.

[0095] The sorting module is used to calculate the correlation coefficient within each document cluster using the seed document title to obtain the weighted sum of the correlation coefficients. Based on the document levels in the structured document information, it counts the number of documents at each level in the document cluster. Based on the number of documents at each level, the total number of documents, and the weighted sum of the correlation coefficients within the document cluster, it calculates the weighted sum of the three indicators to obtain the final score of each document cluster. The document clusters are sorted according to their scores. This module is used to implement the function of step S4 in Embodiment 1, which will not be described again here.

[0096] The class tag generation module is used to generate short sentences that summarize the information features of document clusters by combining the seed document titles within each document cluster with the set prompt input language model. These short sentences are then used as class tags for the document clusters. This module is used to implement the function of step S5 in Embodiment 1, which will not be described in detail here.

[0097] Example 3

[0098] This embodiment provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the document clustering and ranking method based on a large language model as described in any embodiment of the present invention.

[0099] Example 4

[0100] This embodiment provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the document clustering and ranking method based on a large language model as described in any embodiment of the present invention.

[0101] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0102] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0103] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0104] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A document cluster ranking method based on a language large model, characterized by, The method comprises the following steps: S1: collecting document data and performing structured processing, wherein the document data comprises document content and document information, the document information comprises a document title and a document level, and the document content after the structured processing is preprocessed; S2: inputting the preprocessed document content into a language large model to obtain a vectorized representation of the document content; S3: obtaining a plurality of document clusters by using a clustering algorithm on the vectorized document content, calculating a similarity matrix based on vector distance for the documents in each document cluster through vectors, performing intra-cluster sorting on the documents in each document cluster according to the weighted sum of the similarity matrix, taking the top ten documents in the intra-cluster sorting as seed documents, and taking the document title of the seed documents as a seed document title; S4: calculating a correlation coefficient weighted sum in each document cluster by using the seed document title, counting the number of documents of each level in the document cluster based on the document level in the structured document information, and calculating a weighted sum of three indexes, i.e., the number of documents of each level, the total number of documents, and the correlation coefficient weighted sum in the document cluster, to obtain a final score of each document cluster, and performing document sorting on the document clusters according to the scores; S5: inputting the seed document title in each document cluster and a set prompt into the language large model to generate a short sentence summarizing the information features of the document cluster, and taking the short sentence as a class label of the document cluster. The intra-cluster sorting of the documents in each document cluster according to the weighted sum of the similarity matrix in the step S3 is specifically: S31: the set of vectorized representations of the documents within each cluster of documents is wherein is the total number of documents within the cluster of documents, the similarity matrix between the th document and the th document within the cluster of documents is and the similarity matrix is calculated according to the formula: wherein, is a vectorized representation of the i-th document in the document cluster, is a vectorized representation of the i-th document in the document cluster, is a vectorized representation of the i-th document in the document cluster, is a vectorized representation of the i-th document in the document cluster. S32: Calculation of the weighted sum of the similarity matrices of the first and second documents in the document cluster The calculation formula of the weighted sum of the similarity matrices of the first and second documents in the document cluster is specifically as follows: wherein is the similarity matrix of the first document, is the weighted sum of the similarity matrices of the first documents, is the weight of the first similarity matrix within the cluster of documents; S33: performing intra-cluster sorting on the documents in each document cluster according to the weighted sum of the similarity matrix. 2.The language model-based document clustering and ranking method of claim 1, wherein, The correlation coefficient weighted sum obtained by calculating the correlation coefficient in each document cluster in the step S4 is specifically: inputting the seed document title into the language large model to obtain the vectorized representation The calculation formula of the correlation coefficient weighted sum of the first seed document title in each document cluster is specifically: ​ wherein is the correlation coefficient of the th seed document title, is the weight of the th similarity matrix, is the vectorized representation of the th seed document title; The calculation formula of the correlation coefficient weighted sum in each document cluster is specifically: wherein is the correlation coefficient within the document cluster, is the weight of the th document.

3. A document cluster ranking system based on a language large model, characterized by, The method comprises a document collection module, a text vectorization module, a document clustering module, a class sorting module, and a class label generation module. The document collection module is used for collecting document data and performing structured processing, wherein the document data comprises document content and document information, the document information comprises a document title and a document level, and the document content after the structured processing is preprocessed; The text vectorization module is used for inputting the preprocessed document content into a language large model to obtain a vectorized representation of the document content; The document clustering module is used for obtaining a plurality of document clusters by using a clustering algorithm on the vectorized document content, calculating a similarity matrix based on vector distance for the documents in each document cluster through vectors, performing intra-cluster sorting on the documents in each document cluster according to the weighted sum of the similarity matrix, taking the top ten documents in the intra-cluster sorting as seed documents, and taking the document title of the seed documents as a seed document title; The class sorting module is used for calculating a correlation coefficient weighted sum in each document cluster by using the seed document title, counting the number of documents of each level in the document cluster based on the document level in the structured document information, and calculating a weighted sum of three indexes, i.e., the number of documents of each level, the total number of documents, and the correlation coefficient weighted sum in the document cluster, to obtain a final score of each document cluster, and performing document sorting on the document clusters according to the scores; and The class label generation module is used for inputting the seed document title in each document cluster and a set prompt into the language large model to generate a short sentence summarizing the information features of the document cluster, and taking the short sentence as a class label of the document cluster. The class label generation module is configured to input the title of the seed document in each document cluster and a set prompt into the language large model to generate a short sentence summarizing the information features of the document cluster, and use the short sentence as the class label of the document cluster. The in-cluster sorting of the documents in each document cluster according to the weighted sum of the similarity matrix in the document clustering module is specifically as follows: S31: the set of vectorized representations of the documents within each cluster of documents is wherein is the total number of documents within the cluster of documents, the similarity matrix between the th document and the th document within the cluster of documents is and the similarity matrix is calculated according to the formula: wherein, is a vectorized representation of the i-th document in the document cluster, is a vectorized representation of the i-th document in the document cluster, is a vectorized representation of the i-th document in the document cluster, is a vectorized representation of the i-th document in the document cluster. S32: Calculation of the weighted sum of the similarity matrices of the first and second documents in the document cluster The calculation formula of the weighted sum of the similarity matrices of the first and second documents in the document cluster is specifically as follows: wherein is the similarity matrix of the first document, is the weight of the first similarity matrix, is the vectorized representation of the first document within the cluster of documents; S33: The in-cluster sorting of the documents in each document cluster according to the weighted sum of the similarity matrix. 4.The language model-based document cluster ranking system of claim 3, wherein, The correlation coefficient weighted sum of the seed document in the class sorting module is calculated, and the correlation coefficient weighted sum of each document cluster is obtained, which is specifically: inputting the seed document title into a language large model to obtain a vectorized representation The calculation formula of the correlation coefficient weighted sum of the first seed document title in each document cluster is specifically: wherein is the correlation coefficient of the th seed document title, is the weight of the th similarity matrix, is the vectorized representation of the th seed document title; The calculation formula of the weighted sum of the correlation coefficients in each document cluster is specifically as follows: wherein is the correlation coefficient within the document cluster, is the weight of the th document.

5. An electronic device, comprising: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the processor implements the language large model-based document clustering and sorting method according to any one of claims 1 to 2 when executing the computer program.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the language large model-based document clustering and sorting method according to any one of claims 1 to 2.

Images